Skip to content

An always-on second brain you talk to. Voice notes in Telegram → typed, linked knowledge in your Obsidian vault. Runs 24/7 on the Claude subscription you already have.

feat: vendor autograph v0.2.0 as the memory core skill

smixs679c98613312
diff --git a/vault/.claude/skills/autograph/SKILL.md b/vault/.claude/skills/autograph/SKILL.md
new file mode 100644
index 0000000..4d8759d
--- /dev/null
+++ b/vault/.claude/skills/autograph/SKILL.md
@@ -0,0 +1,336 @@
+---
+name: autograph
+description: >-
+ Schema-as-code enforcement for any Obsidian vault. Zero hardcoded domains.
+ Use when creating vault cards, checking vault health, running schema compliance,
+ deduplicating entities, generating MOC indexes, running decay cycles, bootstrapping
+ a vault, fixing wikilinks, finding orphans or backlinks, extracting entities from
+ daily files, or touching/promoting cards.
+ Do NOT use for content generation or non-vault file operations.
+ For quick health checks without schema enforcement, prefer vault-health skill.
+---
+
+# autograph — typed vault engine
+
+One schema. One graph. Works on any vault.
+
+## Overview
+
+No hardcoded domains, types, or paths. The agent discovers structure from data, builds a schema, then enforces it. All scripts share `common.py`. Zero external dependencies (stdlib only, API calls via urllib).
+
+## Quick Reference: 5 Workflows
+
+| Workflow | When to use | Entry point |
+|----------|-------------|-------------|
+| **BOOTSTRAP** | New vault / after import / first setup | `discover.py` → `enforce.py` → `graph.py health` |
+| **HEALTH** | Daily maintenance / on request | `graph.py health` → fix → moc → decay |
+| **CREATE** | New knowledge card | Schema lookup → write file → link → touch |
+| **SEARCH & LINK** | Find info + strengthen connections | Hub → links → target; `graph.py orphans` → connect |
+| **ORCHESTRATE** | Automated multi-agent workflows (no API keys) | `orchestrate.py health\|bootstrap` |
+
+---
+
+## Workflow 1: BOOTSTRAP (raw vault → structured graph)
+
+**When to use:** New vault, bulk import, first setup. Run once, then switch to HEALTH.
+
+**Full guide:** `references/bootstrap-workflow.md`
+
+### Summary (10 phases)
+
+1. **Discover:** `uv run scripts/discover.py <vault-dir> --verbose > /tmp/discovery.json`
+2. **Generate schema:** Script baseline (`generate_schema.py`) + **agent swarm** (`swarm_prepare.py` → Wave 1 haiku → `swarm_reduce.py` → Wave 2 sonnet). **NEVER skip the swarm.**
+3. **Review:** Human approves schema. Never auto-apply.
+4. **Bootstrap + Enforce:** `engine.py init` + `enforce.py --apply`
+5. **Link cleanup:** `link_cleanup.py --apply` (before enrichment)
+6. **Tag enrich:** `enrich.py tags --apply` (via OpenRouter API)
+7. **Deduplicate:** `dedup.py --apply` (before link enrichment)
+8. **Link enrich:** `enrich.py swarm-links --apply` (**always swarm-links, never links**)
+9. **MOC generation:** `moc.py generate`
+10. **Verify:** `graph.py health` + `enforce.py` → target 90+/100
+
+### Critical Rules
+
+- **Always run Phase 2B (agent swarm).** Script alone cannot classify unstructured content.
+- **Always use `swarm-links`**, not `links` (0.3% vs 81.6% match rate).
+- **Always dry-run first** — run without `--apply` before applying.
+- **Dedup before link enrich** — prevents links to merged/trashed files.
+
+---
+
+## Workflow 2: HEALTH (daily graph maintenance)
+
+**When to use:** Daily upkeep, after edits, or when health score drops. This is the most common workflow.
+
+### Decision Logic
+
+```
+1. Run `graph.py health <vault-dir>` → check score
+2. If health < 90 → investigate:
+ a. broken_links > 0 → `graph.py fix <vault-dir> --apply`
+ b. orphans > 5 → connect orphans to hub files (see Workflow 4)
+ c. desc_coverage < 70% → add descriptions to files missing them
+3. Run `moc.py generate <vault-dir>` → regenerate indexes
+4. Run `engine.py decay <vault-dir>` → recalculate relevance + tiers
+5. Run `graph.py health <vault-dir>` → confirm improvement
+```
+
+### Thresholds & Action Triggers
+
+| Metric | Good | Action needed |
+|--------|------|---------------|
+| Health score | ≥90 | <90: investigate broken links, orphans |
+| Broken links | 0 | >0: `graph.py fix --apply` |
+| Orphan files | <5 | ≥5: connect to hubs (Workflow 4) |
+| Description coverage | ≥80% | <70%: add descriptions |
+| Stale cards (>90d) | <20% | >30%: `engine.py creative` to resurface |
+
+### Commands
+
+```bash
+uv run scripts/graph.py health <vault-dir> # health check
+uv run scripts/graph.py fix <vault-dir> --apply # fix broken links
+uv run scripts/moc.py generate <vault-dir> # regenerate MOCs
+uv run scripts/engine.py decay <vault-dir> # decay cycle (Ebbinghaus)
+uv run scripts/engine.py decay <vault-dir> --dry-run # preview decay changes
+uv run scripts/engine.py stats <vault-dir> # tier distribution
+uv run scripts/engine.py creative 5 <vault-dir> # resurface forgotten cards
+```
+
+---
+
+## Workflow 3: CREATE (new card with immediate linking)
+
+**When to use:** Creating any new vault card. Always link immediately — orphan cards are wasted knowledge.
+
+### Steps
+
+1. **Type:** Pick from schema `node_types`
+2. **Path:** Reverse-lookup `domain_inference` to find target folder:
+ ```python
+ # domain_inference maps path→domain. To find folder for domain "crm":
+ for path_prefix, domain in schema['domain_inference'].items():
+ if domain == 'crm':
+ target_folder = path_prefix # e.g. "work/crm/"
+ break
+ ```
+3. **Frontmatter:** Write description (search snippet, not title repeat), tags (2-5, lowercase, kebab-case), status from type's enum
+4. **LINKING PROTOCOL (mandatory):**
+ a. Add `## Related` section with `[[hub]]` file of the domain
+ - Hub = `_index.md` or `MEMORY.md` of that domain
+ b. Find 2-3 sibling cards of same type+domain → add `[[links]]`
+ - `uv run scripts/graph.py backlinks <vault> <hub>` → find siblings
+ - Or: read vault-graph.json → filter nodes by type+domain
+ c. Run `uv run scripts/engine.py touch <new-file>`
+5. **Verify checklist:**
+ - [ ] Hub linked?
+ - [ ] 2+ related cards found?
+ - [ ] description ≠ title repeat?
+ - [ ] tags: 2-5, lowercase, kebab-case?
+ - [ ] status ∈ schema enum?
+
+Templates: `references/card-templates.md`
+
+---
+
+## Workflow 4: SEARCH & LINK (find + strengthen connections)
+
+**When to use:** Looking up information in the vault, or strengthening weak areas of the graph.
+
+### Navigation (Hub → Links → Target)
+
+1. **Determine domain** from the topic (work, personal, research, etc. — whatever your schema defines)
+2. **Start at hub:** `_index.md` or `MEMORY.md` of that domain
+3. **Follow links** — max 2 hops from hub to target
+4. **Fallback:** `uv run scripts/graph.py backlinks <vault> <target>` for reverse links
+
+### Orphan Rescue
+
+```bash
+uv run scripts/graph.py orphans <vault-dir> # find orphans
+# For each orphan: connect to nearest hub or sibling card
+```
+
+### Link Strengthening
+
+```bash
+# Files with <2 links → enrich
+OPENROUTER_API_KEY=sk-... uv run scripts/enrich.py swarm-links <vault-dir> --apply
+uv run scripts/graph.py health <vault-dir> # verify improvement
+```
+
+---
+
+## Workflow 5: ORCHESTRATE (automated multi-agent workflows)
+
+**When to use:** Instead of running scripts manually. No API keys — the Claude Code agent does all judgment directly.
+
+### Phase 0: Script sequencing
+
+```bash
+python3 scripts/orchestrate.py health <vault-dir> # automated health workflow
+python3 scripts/orchestrate.py bootstrap <vault-dir> # full bootstrap (one command)
+```
+
+`health` runs: graph check > fix broken links > link cleanup > MOC > decay > verify.
+`bootstrap` runs: enforce > cleanup > tags > dedup > swarm-links > MOC > verify.
+
+### Phases 1-3: Agent judgment (no API keys)
+
+The agent (you) does the judgment directly — read prepared data, decide, write results.
+
+```bash
+# Phase 1: prep dedup clusters for YOUR review
+python3 scripts/orchestrate.py dedup-prepare <vault-dir>
+# -> writes .graph/dedup-review-input.json
+# -> YOU read clusters, mark approved=true, then: dedup.py --apply-manifest
+
+# Phase 2: prep domain catalogs for YOUR link suggestions
+python3 scripts/orchestrate.py link-prepare <vault-dir>
+# -> writes .graph/link-review-input.json
+# -> YOU read catalogs, suggest links per domain, write batch results
+
+# Phase 3: prep graph data for YOUR semantic analysis
+python3 scripts/orchestrate.py graph-prepare <vault-dir>
+# -> writes .graph/graph-analysis-input.json
+# -> YOU analyze contradictions, missing links, stale hubs, write findings
+```
+
+For Phases 1-3: run the prep command, read the output JSON, do the analysis yourself (you ARE the LLM), write results back. Use Agent tool for parallel domain work in Phase 2.
+
+---
+
+---
+
+## Decay Engine (Ebbinghaus)
+
+The decay system models memory with three key mechanisms:
+
+### 1. Access count (spacing effect)
+
+Each `touch` increments `access_count` in frontmatter. More retrievals = slower forgetting:
+
+```
+strength = 1 + ln(access_count)
+effective_rate = base_rate / strength
+relevance = max(floor, 1.0 - effective_rate * days_since_access)
+```
+
+Example: a card touched 5 times has `strength = 1 + ln(5) ≈ 2.6`, decaying ~2.6x slower than a card touched once.
+
+### 2. Domain-specific rates
+
+Different content types decay at different rates. Configure in schema `decay.domain_rates`:
+
+| Type | Rate | Half-life (~) | Rationale |
+|------|------|---------------|-----------|
+| contact | 0.005 | 100 days | People don't become irrelevant quickly |
+| crm | 0.008 | 62 days | Deals have medium lifecycle |
+| learning | 0.010 | 50 days | Knowledge fades moderately |
+| project | 0.012 | 42 days | Projects have defined timelines |
+| daily | 0.020 | 25 days | Daily notes lose relevance fast |
+| (default) | 0.015 | 33 days | Fallback for unlisted types |
+
+### 3. Graduated recall
+
+Touch promotes one tier at a time, not a direct jump to active:
+
+```
+archive → cold → warm → active
+```
+
+Each promotion sets `last_accessed` to a midpoint date, so without re-touch the card naturally drifts back.
+
+### Backward compatibility
+
+- Files without `access_count` → default=1 → `1+ln(1)=1.0` → rate unchanged
+- Files without `type` → default rate applies
+- Existing calls `calc_relevance(days, schema)` → work unchanged (new params optional)
+
+---
+
+## Maintenance Commands
+
+```bash
+uv run scripts/moc.py generate <vault-dir> # MOC generation
+uv run scripts/engine.py decay <vault-dir> # decay cycle
+uv run scripts/engine.py touch <vault-dir>/path/card.md # touch (graduated)
+uv run scripts/engine.py creative 5 <vault-dir> # creative recall
+uv run scripts/engine.py stats <vault-dir> # stats
+uv run scripts/graph.py backlinks <vault-dir> path/to/card # backlinks
+uv run scripts/graph.py orphans <vault-dir> # orphans
+uv run scripts/graph.py fix <vault-dir> --apply # fix links
+uv run scripts/daily.py extract <memory-dir> <vault-dir> # entity extraction
+uv run scripts/engine.py init <vault-dir> --dry-run # bootstrap bare files
+OPENROUTER_API_KEY=sk-... uv run scripts/enrich.py swarm-links <vault-dir> --apply # link enrichment
+OPENROUTER_API_KEY=sk-... uv run scripts/enrich.py tags <vault-dir> --apply # tag enrichment
+uv run scripts/link_cleanup.py <vault-dir> --apply # link cleanup
+```
+
+## Scripts
+
+| Script | Purpose |
+|--------|---------|
+| common.py | Shared: parse FM, walk, domain, decay (Ebbinghaus), wikilinks |
+| discover.py | Workflow 1: scan vault, output enum candidates |
+| generate_schema.py | Workflow 1: turn discovery JSON into draft schema |
+| swarm_prepare.py | Workflow 1: bin-pack vault into agent batches |
+| swarm_reduce.py | Workflow 1: consolidate + validate schema |
+| enforce.py | Workflow 1: validate + autofix against schema |
+| link_cleanup.py | Workflow 1/4: remove phantom wikilinks from ## Related |
+| enrich.py | Workflow 1/4: tags + swarm-links (catalog-oriented link enrichment) |
+| dedup.py | Workflow 1: safe merge + .trash/ |
+| graph.py | Workflow 2/4: health score, link repair, backlinks, orphans |
+| moc.py | Workflow 2: MOC generation per domain |
+| orchestrate.py | Workflow 5: multi-agent orchestration (health, bootstrap, dedup-review, link-enrich, graph-analyze) |
+| engine.py | Workflow 2/3: decay (Ebbinghaus), touch (graduated), creative, stats, init |
+| daily.py | Entity extraction from memory files |
+| test_autograph.py | Self-contained tests (~193 cases, temp fixtures) |
+
+## Files
+
+| File | In package? | Purpose |
+|------|------------|---------|
+| schema.example.json | Yes | Template — copy and customize (includes domain_rates) |
+| schema.json | No | Your vault's schema (generated) |
+| schema.local.json | No | Local override (gitignored) |
+| references/ | Yes | Bootstrap workflow, schema docs, card templates, linking protocol |
+
+## Common Mistakes
+
+| Mistake | Fix |
+|---------|-----|
+| **Skipping agent swarm in Phase 2** | **CRITICAL: always run Step 2B. Script alone cannot classify unstructured content. No exceptions.** |
+| **Using deprecated `links` subcommand** | **`links` was removed (0.3% match rate). Only `swarm-links` is available — 81.6% match rate.** |
+| **Creating cards without linking** | **Always follow Workflow 3 — link to hub + 2 siblings immediately. Orphan cards are wasted knowledge.** |
+| **Touching archive cards to active directly** | **Use graduated recall — touch promotes one tier at a time (archive→cold→warm→active).** |
+| Sending full vault to one agent | Use `swarm_prepare.py` — bin-packs into ~50K token batches. |
+| Running Wave 2 without Wave 1 | `swarm_reduce.py prepare` needs JSONL in `.graph/swarm/classifications/`. |
+| Using schema.example.json directly | Run discover → generate your own schema.json |
+| Description = title repeat | Write specific search snippet |
+| Status not in enum | Check schema's node_types |
+| Skip dry run | Always run without --apply first |
+| Running link enrich before dedup | Creates links to files that get merged/trashed. Dedup first. |
+| Missing OPENROUTER_API_KEY | `enrich.py` reads from `OPENROUTER_API_KEY` env var. |
+| Only running swarm-links once | Run again with `--force` to enrich ALL files. |
+
+## Default Models
+
+| Command | Default model | Override |
+|---------|--------------|----------|
+| tags | google/gemini-3-flash-preview | `--model` flag |
+| swarm-links | google/gemini-2.0-flash-001 | `--model` flag |
+
+Both are production-tested. Do not change defaults without benchmarking.
+
+## Troubleshooting
+
+**Error: Schema not found** → Create schema.json from discover output, or pass path: `enforce.py vault/ my-schema.json`
+
+**Score drops after enforce** → New files without frontmatter. Run `engine.py init vault/`
+
+**Dedup picks wrong canonical** → Content richness wins. Enrich the right file first, re-run.
+
+**Low match rate on swarm-links (<60%)** → Check if LLM returns paths instead of stems. Try `--force` for second pass.
+
+**swarm-links shows 0 matched for some batches** → Usually network errors. Results are cached — rerun and only failed batches retry.
diff --git a/vault/.claude/skills/autograph/evals/evals.json b/vault/.claude/skills/autograph/evals/evals.json
new file mode 100644
index 0000000..994683e
--- /dev/null
+++ b/vault/.claude/skills/autograph/evals/evals.json
@@ -0,0 +1,23 @@
+{
+ "skill_name": "autograph",
+ "evals": [
+ {
+ "id": 1,
+ "prompt": "Need to create a CRM card for a new client — ACME Corp (consumer electronics), primary contact John Doe, CEO. We had a discovery call, they want a brand campaign in Eastern Europe for $15K. Create the card in the vault.",
+ "expected_output": "Card written to work/crm/acme-corp.md with valid frontmatter (type: crm, status from schema enum, description not repeating title, 2-5 tags), a ## Related section with a hub link to work/_index and 2+ sibling links, followed by a touch to initialize recall metadata.",
+ "files": []
+ },
+ {
+ "id": 2,
+ "prompt": "Check the vault health. If the score is below 90, show me what to fix. Repair any broken links you find. Show the tier distribution.",
+ "expected_output": "Runs graph.py health, analyses metrics, fixes broken links when present, runs engine.py stats for tier distribution, runs moc.py generate if MOCs are stale, and runs engine.py decay to refresh relevance scores.",
+ "files": []
+ },
+ {
+ "id": 3,
+ "prompt": "Find everything in the vault related to Sample Inc — the client card, projects, linked contacts. Show backlinks, and report any orphans in the 'work' domain.",
+ "expected_output": "Navigates via the hub work/_index.md → wikilinks → work/crm/sample-inc.md. Uses graph.py backlinks for reverse lookups. Checks orphans in the 'work' domain via graph.py orphans. Re-links any orphan files when sensible targets exist.",
+ "files": []
+ }
+ ]
+}
diff --git a/vault/.claude/skills/autograph/references/bootstrap-workflow.md b/vault/.claude/skills/autograph/references/bootstrap-workflow.md
new file mode 100644
index 0000000..41d5e7d
--- /dev/null
+++ b/vault/.claude/skills/autograph/references/bootstrap-workflow.md
@@ -0,0 +1,261 @@
+# Workflow 1: BOOTSTRAP (raw vault → structured graph)
+
+**When to use:** New vault, bulk import, first setup. Run once, then switch to HEALTH workflow for maintenance.
+
+## Phase 1: Discover (scan vault, find natural patterns)
+
+```bash
+python3 scripts/discover.py <vault-dir> --verbose > /tmp/discovery.json
+```
+
+Outputs JSON: all frontmatter field values, folder structure, wikilink patterns, tag frequency. This is raw data — not yet a schema.
+
+## Phase 2: Generate schema (CRITICAL — TWO STEPS, NEVER SKIP)
+
+**THIS PHASE HAS TWO MANDATORY PARTS. BOTH MUST RUN. NEVER SKIP THE AGENT SWARM.**
+
+### Step 2A: Script baseline
+
+```bash
+python3 scripts/generate_schema.py /tmp/discovery.json /tmp/schema-draft.json
+```
+
+Produces a mechanical draft from enum data. For vaults with existing frontmatter this may be sufficient. For vaults without frontmatter (imports, fresh vaults, mixed dumps) the script output will be nearly empty — this is expected.
+
+### Step 2B: Agent swarm enrichment (MANDATORY — map-reduce)
+
+**NEVER SKIP THIS STEP.** No deterministic script can classify unstructured notes. Only LLM agents can read actual content and propose meaningful types, domains, and enums.
+
+Uses a map-reduce architecture: `swarm_prepare.py` splits vault into non-overlapping batches, parallel haiku agents classify (Wave 1), `swarm_reduce.py` consolidates, a single sonnet agent produces the final schema (Wave 2).
+
+#### Preparation
+
+```bash
+python3 scripts/swarm_prepare.py <vault-dir> /tmp/discovery.json [--budget 50000]
+```
+
+Walks vault, estimates tokens per file (~4 bytes/token), greedy bin-packs into batches of ~50K tokens. Writes manifests to `.graph/swarm/manifests/batch-NNN.json`. Each batch contains a non-overlapping file list + seed types extracted from discovery + generate_schema.
+
+**Key guarantees:**
+- Files are never duplicated across batches
+- Each batch stays within token budget
+- Seed types minimize novel type proliferation
+
+#### Wave 1: Classify (parallel haiku agents)
+
+For each manifest in `.graph/swarm/manifests/`:
+
+1. Read the manifest JSON to get file list + seed types
+2. Launch a haiku agent with the Wave 1 prompt (embedded in swarm-meta.json)
+3. Agent reads each file, outputs one JSONL line per file
+4. Save output to `.graph/swarm/classifications/batch-NNN.jsonl`
+
+JSONL format per line:
+```json
+{"path":"<path>","proposed_type":"<type>","proposed_domain":"<domain>","summary":"<120 chars>","seed_match":true,"confidence":"high|medium|low"}
+```
+
+#### Consolidation
+
+```bash
+python3 scripts/swarm_reduce.py prepare <vault-dir> [discovery.json] [draft-schema.json]
+```
+
+Reads all `.graph/swarm/classifications/*.jsonl`, counts type/domain frequencies, identifies novel types (not in seed), writes `consolidation.json` for Wave 2.
+
+#### Wave 2: Reduce (single sonnet agent)
+
+Give the sonnet agent `consolidation.json` contents. It produces a complete schema.json with all 11 sections. Rules: target 5-10 node_types, merge types with <5 files, novel types need >10 files to survive.
+
+Save output to `/tmp/wave2-schema.json`.
+
+#### Finalization
+
+```bash
+python3 scripts/swarm_reduce.py finalize /tmp/wave2-schema.json schema.json
+```
+
+Validates the schema: all 11 sections present, each node_type has description/required/status, aliases point to real types, status_order covers all statuses, 3-15 types (sanity check). If valid → writes schema.json. If invalid → saves draft to `.graph/swarm/schema-draft-invalid.json` with errors.
+
+#### Intermediate files
+
+```
+.graph/swarm/
+ manifests/batch-001.json ... # Wave 1 inputs (file lists per batch)
+ classifications/batch-001.jsonl ... # Wave 1 outputs (JSONL per batch)
+ consolidation.json # Wave 2 input (frequencies + all classifications)
+ swarm-meta.json # Run metadata + prompt templates
+```
+
+**Why this step is critical:**
+- Script works on enum frequency only — if vault has no frontmatter, it produces nothing useful
+- Agents actually READ file content and understand what type of note it is
+- A vault with 1000 imported notes and zero frontmatter needs intelligent classification, not pattern matching
+- Skipping this step = applying a near-empty schema = useless enforce results
+- **Map-reduce prevents context overflow: each agent sees only its batch (~50K tokens)**
+
+**Schema structure (11 required sections):**
+- `node_types` — valid types with status enums per type
+- `type_aliases` — old→new type mappings (auto-fixed on enforce)
+- `field_fixes` — typo corrections per field
+- `domain_inference` — folder prefix→domain mapping
+- `path_type_hints` — folder→type inference
+- `status_order` — sort priority for MOC
+- `status_defaults` — default status by type
+- `richness_fields` — dedup scoring fields
+- `entity_extraction` — daily.py config
+- `decay` — rate, floor, tier thresholds, domain_rates
+- `ignore_tags` — tags to skip
+
+Full reference: `references/schema-reference.md`
+
+## Phase 3: Review (human approves schema)
+
+Present schema to user. They approve, adjust, or reject. Never auto-apply a generated schema.
+
+## Phase 4: Bootstrap + Enforce (frontmatter + structural fields)
+
+```bash
+python3 scripts/engine.py init <vault-dir> --dry-run # bootstrap bare files
+python3 scripts/engine.py init <vault-dir> # apply
+python3 scripts/enforce.py <vault-dir> schema.json # dry run
+python3 scripts/enforce.py <vault-dir> schema.json --apply # apply
+```
+
+Bootstrap adds frontmatter to bare files. Enforce auto-fixes: type aliases, missing types (inferred from path), status typos, missing domains, missing system fields. Flags for review: missing descriptions, missing tags, unknown statuses.
+
+## Phase 5: Link Cleanup (remove phantom wikilinks)
+
+```bash
+python3 scripts/link_cleanup.py <vault-dir> # dry run
+python3 scripts/link_cleanup.py <vault-dir> --apply
+```
+
+Cleans historical phantom wikilinks from `## Related` sections. Checks every link target against real vault stems. Broken links are removed; if all links in a section are broken, the section is deleted. Body links outside `## Related` are never touched. Report written to `.graph/link-cleanup-report.json`.
+
+Run BEFORE tag/link enrichment so the enricher works on clean files.
+
+## Phase 6: Tag Enrich (tags via OpenRouter API)
+
+```bash
+OPENROUTER_API_KEY=sk-... python3 scripts/enrich.py tags <vault-dir> # dry run
+OPENROUTER_API_KEY=sk-... python3 scripts/enrich.py tags <vault-dir> --apply # apply
+```
+
+Collects seed tags from vault, finds files with empty/missing tags, batches them into API calls via OpenRouter (default model: `google/gemini-3-flash-preview`). LLM assigns 1-5 lowercase hyphenated tags per file, strongly preferring seed tags. Results cached in `.graph/enrich/tags/`. Concurrent via `ThreadPoolExecutor` (default 3 workers).
+
+Options: `--budget N` (token budget per batch), `--model MODEL`, `--force` (re-process all), `--delay N` (seconds between calls), `--workers N`.
+
+## Phase 7: Deduplicate
+
+```bash
+python3 scripts/dedup.py <vault-dir> # report
+python3 scripts/dedup.py <vault-dir> --apply # merge + trash
+```
+
+Finds same-slug files in different folders. Picks canonical by content richness (primary) + path depth (tiebreaker). Merges unique content, redirects wikilinks, moves extras to `.trash/`. Run BEFORE link enrichment so links point to canonical files.
+
+## Phase 8: Link Enrich — Swarm Links (CATALOG-ORIENTED)
+
+**USE `swarm-links`, NOT `links`.** This is the proven approach.
+
+```bash
+OPENROUTER_API_KEY=sk-... python3 scripts/enrich.py swarm-links <vault-dir> # dry run
+OPENROUTER_API_KEY=sk-... python3 scripts/enrich.py swarm-links <vault-dir> --apply # apply
+OPENROUTER_API_KEY=sk-... python3 scripts/enrich.py swarm-links <vault-dir> --apply --force # re-enrich all files
+```
+
+### How it works
+
+1. **Build catalog** — `build_catalog()` scans vault, groups files by domain into `{domain: [{stem, type, tags, desc}]}`
+2. **Format for LLM** — `format_catalog()` renders entries as text table (stem | tags | first line of content), max 800 entries per domain
+3. **Cross-domain bridging** — Each domain catalog gets top-30 stems from other domains to prevent domain silos
+4. **LLM picks from catalog** — Prompt explicitly says "ONLY return stems from the catalog, do NOT invent new names"
+5. **Strict set validation** — No fuzzy matching. Response stems checked against `all_stems` set. Also extracts stem from path-like responses (`Path(s).stem`) since LLMs sometimes return full paths
+6. **Self-link & duplicate filtering** — Removes file's own stem and already-existing links
+7. **Apply** — Appends `## Related` section with matched wikilinks (reuses `apply_links()`)
+
+### Defaults
+
+| Parameter | Value | Why |
+|-----------|-------|-----|
+| Model | `google/gemini-2.0-flash-001` | Cheap, fast, follows catalog instructions well |
+| Files per batch | 8 | Catalog takes ~5K tokens, leave room for files |
+| Workers | 5 | Flash handles high parallelism |
+| Delay | 0.3s | Lower latency, Flash is cheap |
+| Validation | Strict set membership | Zero phantom links — only exact stems pass |
+
+### Why NOT `links` (legacy)
+
+The old `enrich.py links` approach asks LLM to "generate natural, descriptive names" WITHOUT providing a stem catalog. Then Python fuzzy-matches suggestions against real vault stems (SequenceMatcher + Jaccard, threshold 0.75).
+
+**Results in practice:**
+- `links`: **10 out of 3260 suggestions matched (0.3%)** — catastrophic failure
+- `swarm-links`: **3488 out of 4277 matched (81.6%)** — works
+
+The root cause: LLM invents names like "machine learning fundamentals" but the vault stem is "ml-basics" or in Russian. Fuzzy matching can't bridge that gap. Catalog-based approach eliminates hallucination entirely.
+
+**`links` is kept for backward compatibility but should NOT be used. Always use `swarm-links`.**
+
+### Key lesson: path-to-stem extraction
+
+LLMs sometimes return full paths (`Inbox/imports/DESCRIBE`) instead of bare stems (`DESCRIBE`). The validation layer handles this:
+1. Try raw value against stems set
+2. If no match, extract `Path(s).stem` and try again
+3. If `.md` suffix present, strip it and retry
+
+This raised match rate from 52% to 81.6% in testing.
+
+### Intermediate files
+
+```
+.graph/enrich/
+ swarm-links/
+ batch-001-results.json # {batch_id, results: [{path, links_raw, matched_links}], validation}
+ run-meta.json # {mode, total_files, match_rate, model, validation method}
+ tags/
+ batch-001-results.json
+ run-meta.json
+ errors/
+ batch-NNN-error.json # Only on failures
+```
+
+### Recommended run order
+
+1. `swarm-links .` — dry run, check match rate in output
+2. `swarm-links . --apply` — apply to files with <2 links
+3. `swarm-links . --apply --force` — re-enrich ALL files (second pass catches stragglers)
+4. `link_cleanup.py . --apply` — clean any broken links after enrichment
+5. `graph.py health .` — verify health score improved
+
+## Phase 9: MOC Generation
+
+```bash
+python3 scripts/moc.py generate <vault-dir>
+python3 scripts/moc.py generate <vault-dir> --domain work
+```
+
+Generates index files per domain with wikilinks grouped by type and sorted by status.
+
+## Phase 10: Verify
+
+```bash
+python3 scripts/graph.py health <vault-dir>
+python3 scripts/enforce.py <vault-dir> schema.json
+```
+
+Target: 90+/100 on both scores.
+
+## Proven Results (production run, 864 files)
+
+| Metric | Before enrichment | After swarm-links |
+|--------|------------------|-------------------|
+| Health Score | 66.4 | **80.6** |
+| Total links | ~1750 | **4963** |
+| Avg links/file | 2.03 | **5.74** |
+| Dead-ends | 164+ | **55** |
+| Orphan files | 9 | **7** |
+| Files with ## Related | ~200 | **777 (90%)** |
+| Match rate (links) | 0.3% | N/A (deprecated) |
+| Match rate (swarm-links) | N/A | **81.6%** |
+| API cost (Gemini Flash) | — | ~$0.10 total |
diff --git a/vault/.claude/skills/autograph/references/card-templates.md b/vault/.claude/skills/autograph/references/card-templates.md
new file mode 100644
index 0000000..5e212fa
--- /dev/null
+++ b/vault/.claude/skills/autograph/references/card-templates.md
@@ -0,0 +1,68 @@
+# Card Templates
+
+Generic templates. Adapt types/statuses to your schema.json.
+
+## Note
+
+```yaml
+---
+type: note
+description: >-
+ [One-line search snippet — what this knowledge is about]
+tags: [topic, subtopic]
+status: active
+source: article
+created: 2026-01-01
+---
+```
+
+## Contact
+
+```yaml
+---
+type: contact
+description: >-
+ [Who they are, relationship context]
+tags: [network, role]
+status: active
+---
+```
+
+## Project
+
+```yaml
+---
+type: project
+description: >-
+ [What the project delivers, for whom]
+tags: [client, type]
+status: active
+---
+```
+
+## Linking Protocol (ОБЯЗАТЕЛЬНО при создании карточки)
+
+После создания файла — СРАЗУ свяжи:
+
+1. **Hub link:** Добавь `## Related` с [[hub]] файлом домена
+ - Определи домен из path → schema `domain_inference`
+ - Hub = _index.md или MEMORY.md домена
+2. **Sibling links:** Найди 2-3 карточки того же type+domain
+ - `python3 scripts/graph.py backlinks <vault> <hub>` → найди siblings
+ - Или: прочитай vault-graph.json → filter nodes by type+domain
+3. **Touch:** `python3 scripts/engine.py touch <new-file>`
+4. **Verify:** Карточка должна иметь ≥2 links в `## Related`
+
+### Checklist
+- [ ] Hub linked?
+- [ ] 2+ related cards found?
+- [ ] description ≠ title repeat?
+- [ ] tags: 2-5, lowercase, kebab-case?
+- [ ] status ∈ schema enum?
+
+## Anti-Patterns
+
+❌ `description: "Contact"` — useless for search, write a real snippet
+❌ `status: "interested"` — not in enum, use what your schema defines
+❌ `tags: []` — empty tags add nothing, pick 2-5 relevant ones
+❌ No frontmatter — every file needs `---` block
diff --git a/vault/.claude/skills/autograph/references/schema-reference.md b/vault/.claude/skills/autograph/references/schema-reference.md
new file mode 100644
index 0000000..01c9f08
--- /dev/null
+++ b/vault/.claude/skills/autograph/references/schema-reference.md
@@ -0,0 +1,127 @@
+# Schema Reference
+
+Full documentation for `schema.json` — generated per vault, never shipped with hardcoded values.
+
+## Structure
+
+```json
+{
+ "node_types": { ... },
+ "type_aliases": { ... },
+ "field_fixes": { ... },
+ "domain_inference": { ... },
+ "path_type_hints": { ... },
+ "status_order": { ... },
+ "status_defaults": { ... },
+ "richness_fields": { ... },
+ "entity_extraction": { ... },
+ "decay": { ... },
+ "ignore_tags": [ ... ]
+}
+```
+
+## node_types
+
+Map of type name → definition. Each type has:
+- `description` — what this type represents
+- `required` — fields that must be present (e.g. `["description", "tags", "status"]`)
+- `status` — valid status enum values for this type
+- Optional: `priority`, `potential`, `source` — additional enum fields
+
+Example:
+```json
+"project": {
+ "description": "Project with deliverables",
+ "required": ["description", "tags", "status"],
+ "status": ["draft", "active", "done", "paused", "cancelled"]
+}
+```
+
+## type_aliases
+
+Map of old type values → canonical type. Auto-fixed on enforce.
+
+Example: `"blog-post": "note"` — any card with `type: blog-post` gets rewritten to `type: note`.
+
+## field_fixes
+
+Map of field name → {old_value: fixed_value}. Catches typos.
+
+Example: `"status": {"actve": "active"}` — fixes a common typo across the vault.
+
+## domain_inference
+
+Map of folder prefix → domain name. Used when a card has no explicit `domain:` field.
+
+Example: `"projects/": "work"` — any file under `projects/` gets `domain: work`.
+
+## decay
+
+Controls relevance decay over time.
+
+| Parameter | Description | Default |
+|-----------|-------------|---------|
+| rate | Relevance loss per day | 0.015 |
+| floor | Minimum relevance | 0.1 |
+| tiers.active | Days threshold for "active" | 7 |
+| tiers.warm | Days threshold for "warm" | 21 |
+| tiers.cold | Days threshold for "cold" | 60 |
+
+Formula: `relevance = max(floor, 1.0 - rate × days_since_access)`
+
+Tier beyond cold = "archive".
+
+Special tier `core` = manual only, never auto-demoted.
+
+## path_type_hints
+
+Map of folder substring → type name. Used by `infer_type()` when a card has no explicit `type:` field.
+
+Example: `"leads/": "lead"` — files in a `leads/` folder get `type: lead`.
+
+## status_order
+
+Map of status name → sort priority (integer). Used by `moc.py` for MOC generation ordering. Lower number = higher priority.
+
+Example: `"active": 0, "draft": 9, "cancelled": 11`
+
+## status_defaults
+
+Default status assigned when a card has no `status:` field. Key = type name, `"default"` = fallback.
+
+Example: `"default": "active"` — all types default to `active` unless overridden per type.
+
+## richness_fields
+
+Configuration for dedup content scoring.
+
+- `bonus_fields` — list of frontmatter field names that indicate content richness. Each present field adds 15 points to the richness score.
+
+Example: `"bonus_fields": ["telegram", "email", "company", "role"]`
+
+## entity_extraction
+
+Configuration for `daily.py` entity extraction.
+
+- `noise_words` — list of words to filter out when detecting **Bold Name** patterns in daily files (e.g. `"TODO"`, `"FIX"`).
+
+## ignore_tags
+
+List of tags to skip during processing (e.g. bulk import artifacts).
+
+## System Fields (auto-managed by engine.py)
+
+| Field | Values | Description |
+|-------|--------|-------------|
+| tier | core, active, warm, cold, archive | Decay tier |
+| relevance | 0.0–1.0 | Decay score |
+| last_accessed | ISO date | Last touched |
+| domain | (from domain_inference) | Auto-inferred from path |
+
+## Generating Your Schema
+
+1. Run `discover.py <vault>` → save discovery JSON
+2. Run `generate_schema.py discovery.json schema.json` → draft schema
+3. Review the generated schema, adjust as needed
+4. Run `enforce.py <vault> schema.json` to validate
+5. Iterate: add missing values, fix typos, re-run
diff --git a/vault/.claude/skills/autograph/schema.example.json b/vault/.claude/skills/autograph/schema.example.json
new file mode 100644
index 0000000..a109202
--- /dev/null
+++ b/vault/.claude/skills/autograph/schema.example.json
@@ -0,0 +1,90 @@
+{
+ "_comment": "Example schema — generate yours with: discover.py <vault> → review → save as schema.json",
+
+ "node_types": {
+ "note": {
+ "description": "Knowledge note, article, thought",
+ "required": ["description", "tags"],
+ "status": ["active", "draft", "archived"]
+ },
+ "contact": {
+ "description": "Person",
+ "required": ["description", "tags", "status"],
+ "status": ["active", "inactive"]
+ },
+ "project": {
+ "description": "Project with deliverables",
+ "required": ["description", "tags", "status"],
+ "status": ["active", "done", "paused", "cancelled"]
+ }
+ },
+
+ "type_aliases": {},
+ "field_fixes": {},
+ "region_fixes": {},
+
+ "domain_inference": {
+ "projects/": "work",
+ "personal/": "personal",
+ "knowledge/": "knowledge"
+ },
+
+ "path_type_hints": {
+ "_comment": "folder substring → type name (used when card has no type field)",
+ "leads/": "lead",
+ "contacts/": "contact",
+ "people/": "contact"
+ },
+
+ "status_order": {
+ "_comment": "sort order for MOC generation (lower = higher priority)",
+ "active": 0,
+ "prospect": 1,
+ "negotiation": 2,
+ "won": 3,
+ "standby": 4,
+ "pending": 5,
+ "lost": 6,
+ "inactive": 7,
+ "done": 8,
+ "draft": 9,
+ "paused": 10,
+ "cancelled": 11
+ },
+
+ "status_defaults": {
+ "_comment": "default status for new cards by type (when status is missing)",
+ "default": "active"
+ },
+
+ "richness_fields": {
+ "_comment": "frontmatter fields that indicate content richness (used by dedup)",
+ "bonus_fields": ["telegram", "email", "company", "role", "source", "deal_status", "responsible"]
+ },
+
+ "entity_extraction": {
+ "_comment": "settings for daily.py entity extraction",
+ "noise_words": ["TODO", "FIX", "NEW", "UPDATED", "Phase", "Score", "Vault",
+ "Health", "Pricing"]
+ },
+
+ "decay": {
+ "rate": 0.015,
+ "floor": 0.1,
+ "tiers": {
+ "active": 7,
+ "warm": 21,
+ "cold": 60
+ },
+ "domain_rates": {
+ "_comment": "type-specific decay rates (override default rate)",
+ "contact": 0.005,
+ "crm": 0.008,
+ "learning": 0.010,
+ "project": 0.012,
+ "daily": 0.020
+ }
+ },
+
+ "ignore_tags": []
+}
diff --git a/vault/.claude/skills/autograph/scripts/common.py b/vault/.claude/skills/autograph/scripts/common.py
new file mode 100644
index 0000000..07c1234
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/common.py
@@ -0,0 +1,493 @@
+"""
+autograph common — shared utilities for all scripts.
+Single source of truth. NO hardcoded domains, types, statuses, paths.
+Everything reads from schema.json.
+"""
+
+import re
+import json
+from pathlib import Path
+from datetime import date, datetime
+from collections import defaultdict
+
+# ─── CONSTANTS ─────────────────────────────────────────────
+IGNORE_DIRS = frozenset({
+ '.obsidian', 'attachments', '.git', '.graph',
+ '.claude', '.trash', 'backup', 'archive', '__pycache__'
+})
+
+SCHEMA_FILENAME = 'schema.json'
+
+
+# ─── SCHEMA ────────────────────────────────────────────────
+_schema_cache = {}
+
+def load_schema(schema_path: Path | str | None = None) -> dict:
+ """Load schema.json. Caches after first load by resolved path.
+ If no path given, looks in: CWD/schema.json → skill dir/schema.local.json → schema.json → schema.example.json"""
+ if schema_path is not None:
+ schema_path = Path(schema_path)
+ key = str(schema_path.resolve())
+ if key in _schema_cache:
+ return _schema_cache[key]
+ else:
+ skill_dir = Path(__file__).parent.parent
+ candidates = [
+ Path.cwd() / SCHEMA_FILENAME, # user's schema in CWD
+ skill_dir / 'schema.local.json', # local override FIRST
+ skill_dir / SCHEMA_FILENAME, # schema.json in skill dir
+ skill_dir / 'schema.example.json', # fallback example
+ ]
+ for c in candidates:
+ if c.exists():
+ schema_path = c
+ break
+ if schema_path is None:
+ schema_path = skill_dir / SCHEMA_FILENAME # will raise FileNotFoundError
+ key = str(schema_path.resolve())
+ if key in _schema_cache:
+ return _schema_cache[key]
+
+ if not schema_path.exists():
+ raise FileNotFoundError(f"Schema not found: {schema_path}")
+
+ schema = json.loads(schema_path.read_text())
+ _schema_cache[key] = schema
+ return schema
+
+
+def get_node_types(schema: dict) -> list[str]:
+ """All valid node types."""
+ return list(schema.get('node_types', {}).keys())
+
+
+def get_valid_statuses(schema: dict, node_type: str) -> list[str]:
+ """Valid statuses for a node type."""
+ return schema.get('node_types', {}).get(node_type, {}).get('status', [])
+
+
+def get_type_aliases(schema: dict) -> dict:
+ """Type alias map (old → new)."""
+ return schema.get('type_aliases', {})
+
+
+def get_domain_map(schema: dict) -> dict:
+ """Folder → domain mapping."""
+ return schema.get('domain_inference', {})
+
+
+def get_decay_config(schema: dict) -> dict:
+ """Decay rate, floor, tier thresholds."""
+ return schema.get('decay', {
+ 'rate': 0.015, 'floor': 0.1,
+ 'tiers': {'active': 7, 'warm': 21, 'cold': 60}
+ })
+
+
+def get_ignore_tags(schema: dict) -> set:
+ """Tags to ignore (e.g. bulk import artifacts)."""
+ return set(schema.get('ignore_tags', []))
+
+
+def get_field_fixes(schema: dict) -> dict:
+ """Field value normalizations."""
+ return schema.get('field_fixes', {})
+
+
+def get_path_type_hints(schema: dict) -> dict:
+ """Folder substring → type name mapping for type inference."""
+ hints = schema.get('path_type_hints', {})
+ return {k: v for k, v in hints.items() if k != '_comment'}
+
+
+def get_status_order(schema: dict) -> dict:
+ """Status sort order for MOC generation."""
+ order = schema.get('status_order', {})
+ return {k: v for k, v in order.items() if k != '_comment'}
+
+
+def get_status_defaults(schema: dict) -> dict:
+ """Default status by type when status is missing."""
+ defaults = schema.get('status_defaults', {})
+ return {k: v for k, v in defaults.items() if k != '_comment'}
+
+
+def get_richness_fields(schema: dict) -> list:
+ """Frontmatter fields that indicate content richness (for dedup)."""
+ cfg = schema.get('richness_fields', {})
+ return cfg.get('bonus_fields', [])
+
+
+def get_entity_extraction_config(schema: dict) -> dict:
+ """Entity extraction settings for daily.py."""
+ cfg = schema.get('entity_extraction', {})
+ return {k: v for k, v in cfg.items() if k != '_comment'}
+
+
+# ─── FRONTMATTER ───────────────────────────────────────────
+def parse_frontmatter(content: str) -> tuple[dict, str, list[str]]:
+ """Parse YAML frontmatter from markdown content.
+ Returns: (fields_dict, body_after_fm, original_fm_lines)
+ If no frontmatter: (None, full_content, [])
+ """
+ content = content.replace('\r\n', '\n').replace('\r', '\n')
+ m = re.match(r'^---\n(.*?)\n---\n?(.*)', content, re.DOTALL)
+ if not m:
+ return None, content, []
+
+ raw_lines = m.group(1).split('\n')
+ body = m.group(2)
+ fields = {}
+ multiline_key = None
+ multiline_mode = None # fold | literal | list | pending
+ multiline_sep = ' ' # >- fold (space), |- literal (newline)
+
+ for line in raw_lines:
+ stripped = line.strip()
+ indented = line.startswith(' ') or line.startswith('\t')
+
+ # Multi-line continuation
+ if multiline_key and indented:
+ if multiline_mode == 'pending':
+ if stripped.startswith('- '):
+ multiline_mode = 'list'
+ fields[multiline_key] = []
+ else:
+ multiline_mode = 'fold'
+ multiline_sep = ' '
+ fields[multiline_key] = ''
+
+ if multiline_mode == 'list':
+ item = stripped[2:].strip() if stripped.startswith('- ') else stripped
+ if item:
+ fields[multiline_key].append(item.strip("'\""))
+ else:
+ prev = fields.get(multiline_key, '') or ''
+ fields[multiline_key] = (prev + multiline_sep + stripped).strip()
+ continue
+
+ if multiline_key and not indented:
+ if fields.get(multiline_key) is None:
+ fields[multiline_key] = ''
+ multiline_key = None
+ multiline_mode = None
+ multiline_sep = ' '
+
+ if not stripped or stripped.startswith('#'):
+ continue
+
+ if ':' not in stripped:
+ continue
+
+ key, _, val = stripped.partition(':')
+ key = key.strip()
+ val = val.strip()
+
+ if val in ('>-', '>'):
+ multiline_key = key
+ multiline_mode = 'fold'
+ multiline_sep = ' '
+ fields[key] = ''
+ continue
+ if val in ('|-', '|'):
+ multiline_key = key
+ multiline_mode = 'literal'
+ multiline_sep = '\n'
+ fields[key] = ''
+ continue
+ if val.startswith('>-') or val.startswith('>'):
+ multiline_key = key
+ multiline_mode = 'fold'
+ multiline_sep = ' '
+ fields[key] = val.lstrip('>-').strip()
+ continue
+ if val.startswith('|-') or val.startswith('|'):
+ multiline_key = key
+ multiline_mode = 'literal'
+ multiline_sep = '\n'
+ fields[key] = val.lstrip('|-').strip()
+ continue
+ if val == '':
+ multiline_key = key
+ multiline_mode = 'pending'
+ multiline_sep = ' '
+ fields[key] = None
+ continue
+
+ if val.startswith('[') and val.endswith(']'):
+ items = [x.strip().strip("'\"") for x in val[1:-1].split(',') if x.strip()]
+ fields[key] = items
+ else:
+ fields[key] = val.strip("'\"")
+
+ if multiline_key and fields.get(multiline_key) is None:
+ fields[multiline_key] = ''
+
+ return fields, body, raw_lines
+
+
+def write_frontmatter(fields: dict, original_lines: list[str]) -> str:
+ """Rebuild frontmatter, preserving original order, updating values, appending new.
+ Handles multiline YAML values (>-, |-, >, |) — when a key is rewritten,
+ its continuation lines are skipped to prevent duplication."""
+ written = set()
+ out = []
+ skip_continuation = False
+
+ for line in original_lines:
+ stripped = line.strip()
+
+ if not stripped or stripped.startswith('#'):
+ skip_continuation = False
+ out.append(line)
+ continue
+ if ':' not in stripped:
+ if not skip_continuation:
+ out.append(line)
+ continue
+
+ skip_continuation = False
+ key = stripped.partition(':')[0].strip()
+ val_part = stripped.partition(':')[2].strip()
+ written.add(key)
+ if key in fields:
+ out.append(format_field(key, fields[key]))
+ if val_part in ('>-', '>', '|-', '|'):
+ skip_continuation = True
+ else:
+ out.append(line)
+
+ for key, val in fields.items():
+ if key not in written:
+ out.append(format_field(key, val))
+
+ return '\n'.join(out)
+
+
+YAML_SPECIAL = re.compile(r'[:#\[\]{}"\',|>!&*?]')
+
+def format_field(key: str, val) -> str:
+ """Format a single frontmatter field."""
+ if isinstance(val, list):
+ return f"{key}: [{', '.join(str(v) for v in val)}]"
+ if isinstance(val, (int, float)):
+ return f"{key}: {val}"
+ s = str(val)
+ if key == 'description' and s and len(s) > 80:
+ return f'{key}: >-\n {s}'
+ if YAML_SPECIAL.search(s):
+ escaped = s.replace('"', '\\"')
+ return f'{key}: "{escaped}"'
+ return f"{key}: {s}"
+
+
+# ─── FILE OPERATIONS ───────────────────────────────────────
+def walk_vault(vault_dir: Path) -> list[Path]:
+ """Walk vault, yield all .md files, skipping IGNORE_DIRS."""
+ results = []
+ for md in sorted(vault_dir.rglob('*.md')):
+ parts = set(md.relative_to(vault_dir).parts)
+ if parts & IGNORE_DIRS:
+ continue
+ results.append(md)
+ return results
+
+
+def rel_path(md_file: Path, vault_dir: Path) -> str:
+ """Relative path as string."""
+ return str(md_file.relative_to(vault_dir))
+
+
+def is_hub_path(path: str) -> bool:
+ """Return True for hub notes like _index or MEMORY at any depth."""
+ return Path(path).name in {'_index', 'MEMORY'}
+
+
+def build_link_index(vault_dir: Path, files: list[Path] | None = None) -> dict:
+ """Build deterministic indexes for wikilink resolution."""
+ files = files or walk_vault(vault_dir)
+ exact = {}
+ suffix_map = defaultdict(set)
+ stem_map = defaultdict(set)
+
+ for md in files:
+ rp = rel_path(md, vault_dir)
+ rp_noext = rp[:-3] if rp.endswith('.md') else rp
+ exact[rp_noext] = rp_noext
+ stem_map[md.stem].add(rp_noext)
+
+ parts = rp_noext.split('/')
+ for i in range(1, len(parts) - 1):
+ suffix_map['/'.join(parts[i:])].add(rp_noext)
+
+ return {
+ 'exact': exact,
+ 'unique_suffix': {k: next(iter(v)) for k, v in suffix_map.items() if len(v) == 1},
+ 'ambiguous_suffix': {k: sorted(v) for k, v in suffix_map.items() if len(v) > 1},
+ 'unique_stem': {k: next(iter(v)) for k, v in stem_map.items() if len(v) == 1},
+ 'ambiguous_stem': {k: sorted(v) for k, v in stem_map.items() if len(v) > 1},
+ }
+
+
+def normalize_link_target(target: str) -> str:
+ """Normalize a wikilink target before resolution."""
+ target = target.replace('\\', '').strip()
+ if '#' in target:
+ target = target.split('#', 1)[0].strip()
+ if target.endswith('.md'):
+ target = target[:-3]
+ if target.startswith('vault/'):
+ target = target[6:]
+ return target
+
+
+def resolve_link_target(target: str, link_index: dict) -> tuple[str | None, str]:
+ """Resolve a target using exact path, unique suffix, then unique stem."""
+ target = normalize_link_target(target)
+ if not target:
+ return None, 'empty'
+
+ exact = link_index.get('exact', {})
+ unique_suffix = link_index.get('unique_suffix', {})
+ ambiguous_suffix = link_index.get('ambiguous_suffix', {})
+ unique_stem = link_index.get('unique_stem', {})
+ ambiguous_stem = link_index.get('ambiguous_stem', {})
+
+ if target in exact:
+ return exact[target], 'exact'
+ if target in unique_suffix:
+ return unique_suffix[target], 'unique_suffix'
+ if target in ambiguous_suffix:
+ return None, 'ambiguous_suffix'
+
+ stem = target.split('/')[-1]
+ if '/' in target:
+ return None, 'missing'
+ if stem in unique_stem:
+ return unique_stem[stem], 'unique_stem'
+ if stem in ambiguous_stem:
+ return None, 'ambiguous_stem'
+ return None, 'missing'
+
+
+# ─── DOMAIN INFERENCE ──────────────────────────────────────
+def infer_domain(file_rel_path: str, schema: dict) -> str:
+ """Infer domain from file path using schema's domain_inference map."""
+ domain_map = get_domain_map(schema)
+ for pattern, domain in domain_map.items():
+ if file_rel_path.startswith(pattern):
+ return domain
+ return 'personal' # default
+
+
+# ─── TYPE INFERENCE ────────────────────────────────────────
+def infer_type(file_rel_path: str, schema: dict) -> str:
+ """Infer card type from file path using schema's node_types + path_type_hints.
+ No hardcoded values — uses schema data only."""
+ valid_types = set(get_node_types(schema))
+ rp = file_rel_path.lower()
+
+ # Try to match path keywords against known type names
+ for type_name in valid_types:
+ if f'{type_name}/' in rp or rp.startswith(f'{type_name}/'):
+ return type_name
+
+ # Check path_type_hints from schema
+ for pattern, hint_type in get_path_type_hints(schema).items():
+ if f'/{pattern}' in rp or rp.startswith(pattern):
+ return hint_type if hint_type in valid_types else 'note'
+
+ # Default to first type or 'note'
+ return 'note' if 'note' in valid_types else (list(valid_types)[0] if valid_types else 'note')
+
+
+def collect_duplicate_groups(vault_dir: Path, schema: dict | None = None,
+ files: list[Path] | None = None,
+ ignored_stems: set[str] | None = None) -> dict[tuple[str, str, str], list[str]]:
+ """Group only compatible duplicates: same stem, domain, and type."""
+ schema = schema or {}
+ files = files or walk_vault(vault_dir)
+ ignored_stems = ignored_stems or {'_index', 'MEMORY'}
+ groups = defaultdict(list)
+
+ for md in files:
+ if md.stem in ignored_stems:
+ continue
+
+ rp = rel_path(md, vault_dir)
+ try:
+ content = md.read_text(errors='replace')
+ except Exception:
+ content = ''
+
+ fm, _, _ = parse_frontmatter(content)
+ if fm is None:
+ fm = {}
+
+ domain = str(fm.get('domain') or infer_domain(rp, schema))
+ card_type = str(fm.get('type') or infer_type(rp, schema))
+ groups[(md.stem, domain, card_type)].append(rp)
+
+ return {key: paths for key, paths in groups.items() if len(paths) > 1}
+
+
+# ─── WIKILINKS ─────────────────────────────────────────────
+def extract_wikilinks(text: str) -> list[tuple[str, str]]:
+ """Extract wikilinks as [(target, display_name), ...].
+ Handles [[target]], [[target|display]], and [[target#heading]]."""
+ results = []
+ for m in re.finditer(r'\[\[([^\]|]+?)(?:\|([^\]]+))?\]\]', text):
+ target = m.group(1).strip()
+ # Strip #anchor — file target only
+ if '#' in target:
+ target = target.split('#')[0].strip()
+ if not target:
+ continue
+ display = m.group(2) or target
+ results.append((target, display.strip()))
+ return results
+
+
+# ─── DECAY ─────────────────────────────────────────────────
+def calc_relevance(days_since_access: int, schema: dict,
+ access_count: int = 1, file_type: str = '') -> float:
+ """Ebbinghaus-inspired: more retrievals = slower forgetting.
+ strength = 1 + ln(access_count) → effective_rate = rate / strength
+ """
+ from math import log
+ config = get_decay_config(schema)
+ # Domain-specific rate or default (filter _comment keys)
+ domain_rates = {k: v for k, v in config.get('domain_rates', {}).items()
+ if k != '_comment'}
+ rate = domain_rates.get(file_type, config.get('rate', 0.015))
+ floor = config.get('floor', 0.1)
+ # Ebbinghaus spacing effect
+ strength = 1.0 + log(max(access_count, 1))
+ effective_rate = rate / strength
+ return max(floor, round(1.0 - effective_rate * days_since_access, 3))
+
+
+def calc_tier(days_since_access: int, schema: dict, current_tier: str = '') -> str:
+ """Calculate tier based on days since last access."""
+ if current_tier == 'core':
+ return 'core' # never auto-demoted
+ config = get_decay_config(schema)
+ tiers = config.get('tiers', {'active': 7, 'warm': 21, 'cold': 60})
+ if days_since_access <= tiers.get('active', 7):
+ return 'active'
+ if days_since_access <= tiers.get('warm', 21):
+ return 'warm'
+ if days_since_access <= tiers.get('cold', 60):
+ return 'cold'
+ return 'archive'
+
+
+def days_since(date_str: str) -> int:
+ """Days between a date string and today."""
+ if not date_str:
+ return 999
+ try:
+ dt = datetime.strptime(date_str[:10], '%Y-%m-%d').date()
+ return (date.today() - dt).days
+ except (ValueError, TypeError):
+ return 999
diff --git a/vault/.claude/skills/autograph/scripts/daily.py b/vault/.claude/skills/autograph/scripts/daily.py
new file mode 100755
index 0000000..3653eec
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/daily.py
@@ -0,0 +1,283 @@
+#!/usr/bin/env python3
+"""
+autograph daily — extract entities from daily memory files.
+No hardcoded project/company names. Pattern-based extraction only.
+
+Commands:
+ daily.py extract <memory-dir> <vault-dir> [date|full]
+"""
+
+import re
+import sys
+import json
+from pathlib import Path
+from datetime import datetime, date
+from collections import defaultdict
+
+from common import (
+ load_schema, parse_frontmatter, walk_vault, rel_path,
+ extract_wikilinks, IGNORE_DIRS, get_entity_extraction_config,
+ build_link_index, resolve_link_target, infer_type, infer_domain
+)
+
+LEGACY_ENTITY_BUCKETS = {
+ 'projects': 'project',
+ 'companies': 'company',
+}
+
+
+def build_output_meta() -> dict:
+ """Expose migration hints for legacy daily entity buckets."""
+ return {
+ 'primary_entity_field': 'entities.linked_entities',
+ 'deprecated_fields': [
+ 'entities.projects',
+ 'entities.companies',
+ ],
+ }
+
+
+def resolve_vault_entity(target: str, vault_index: dict) -> dict | None:
+ """Resolve a wikilink target to a typed vault entity."""
+ if not vault_index:
+ return None
+ resolved, _ = resolve_link_target(target, vault_index.get('link_index', {}))
+ if not resolved:
+ return None
+ return vault_index.get('entities', {}).get(resolved)
+
+
+def derive_legacy_buckets(linked_entities: list[dict]) -> dict:
+ """Build legacy buckets from the canonical linked_entities list."""
+ buckets = {bucket: [] for bucket in LEGACY_ENTITY_BUCKETS}
+ for entity in linked_entities:
+ for bucket, entity_type in LEGACY_ENTITY_BUCKETS.items():
+ if entity.get('type') == entity_type:
+ buckets[bucket].append({
+ 'name': entity.get('name', ''),
+ 'link': entity.get('link', ''),
+ })
+ return buckets
+
+
+def extract_entities(text: str, vault_index: dict, noise_words: set | None = None) -> dict:
+ """Extract entities from daily file text. No hardcoded names."""
+ entities = {
+ 'people': [],
+ 'companies': [],
+ 'projects': [],
+ 'linked_entities': [],
+ 'wikilinks': [],
+ 'financial': [],
+ 'events': [],
+ 'decisions': [],
+ }
+
+ # @mentions (filter common bots dynamically — just skip @-refs < 3 chars)
+ for m in re.finditer(r'@([A-Za-z_]\w{2,})', text):
+ handle = m.group(1)
+ if len(handle) >= 3:
+ entities['people'].append({'handle': f'@{handle}', 'type': 'mention'})
+
+ # **Bold Names** (Cyrillic + Latin, 2-4 words starting with uppercase)
+ noise = noise_words or set()
+ for m in re.finditer(r'\*\*([A-ZА-ЯЁ][a-zа-яё]+(?:\s+[A-ZА-ЯЁ][a-zа-яё]+){0,3})\*\*', text):
+ name = m.group(1)
+ if name.split()[0] not in noise and len(name) > 4:
+ entities['people'].append({'name': name, 'type': 'bold'})
+
+ # [[wikilinks]] — resolve against vault index
+ for target, display in extract_wikilinks(text):
+ entities['wikilinks'].append({'target': target, 'display': display})
+ entity = resolve_vault_entity(target, vault_index)
+ if entity:
+ linked = {
+ 'name': display,
+ 'link': entity['path'],
+ 'type': entity['type'],
+ 'domain': entity['domain'],
+ }
+ entities['linked_entities'].append(linked)
+
+ # $amounts
+ for m in re.finditer(r'\$[\d,]+(?:\.\d+)?(?:\s*[KkМм])?', text):
+ entities['financial'].append({'amount': m.group(0), 'context': text[max(0,m.start()-40):m.end()+40].strip()})
+
+ # Emoji events (🚀🔥💀✅❌🎂🎉⚠️)
+ for m in re.finditer(r'[🚀🔥💀✅❌🎂🎉⚠️]\s*(.+?)(?:\n|$)', text):
+ entities['events'].append(m.group(0).strip()[:150])
+
+ # Decisions (✅ lines, "решено", "approved", "decided")
+ for line in text.split('\n'):
+ line_lower = line.strip().lower()
+ if any(kw in line_lower for kw in ['✅', 'решено', 'approved', 'decided', 'договорились']):
+ if len(line.strip()) > 5:
+ entities['decisions'].append(line.strip()[:200])
+
+ entities.update(derive_legacy_buckets(entities['linked_entities']))
+ return entities
+
+
+def build_vault_index(vault_dir: Path, schema: dict | None = None) -> dict:
+ """Build deterministic typed index for vault entity resolution."""
+ index = {'link_index': {}, 'entities': {}}
+ if not vault_dir.is_dir():
+ return index
+ schema = schema or {}
+ files = walk_vault(vault_dir)
+ index['link_index'] = build_link_index(vault_dir, files)
+ for md in files:
+ rp = rel_path(md, vault_dir)
+ rp_noext = rp.replace('.md', '')
+ try:
+ content = md.read_text(errors='replace')
+ except Exception:
+ content = ''
+ fm, _, _ = parse_frontmatter(content)
+ if fm is None:
+ fm = {}
+ index['entities'][rp_noext] = {
+ 'path': rp_noext,
+ 'type': fm.get('type') or infer_type(rp, schema),
+ 'domain': fm.get('domain') or infer_domain(rp, schema),
+ }
+ return index
+
+
+def build_relationships(entities: dict) -> list:
+ """Find co-mentions (entities appearing in same file = related)."""
+ relationships = []
+ all_entities = []
+ for person in entities.get('people', []):
+ name = person.get('handle') or person.get('name', '')
+ if name:
+ all_entities.append(('person', name))
+ for linked in entities.get('linked_entities', []):
+ name = linked.get('name', '')
+ if name:
+ all_entities.append((linked.get('type', 'entity'), name))
+
+ # Co-mention pairs
+ seen = set()
+ for i, (t1, n1) in enumerate(all_entities):
+ for j, (t2, n2) in enumerate(all_entities):
+ if i >= j:
+ continue
+ pair = tuple(sorted([n1, n2]))
+ if pair not in seen:
+ seen.add(pair)
+ relationships.append({
+ 'from': n1, 'from_type': t1,
+ 'to': n2, 'to_type': t2,
+ 'type': 'co_mentioned'
+ })
+ return relationships
+
+
+def process_date(memory_dir: Path, vault_index: dict, date_str: str, noise_words: set | None = None) -> dict:
+ """Process a single daily file."""
+ md_file = memory_dir / f"{date_str}.md"
+ if not md_file.exists():
+ return {'date': date_str, 'status': 'not_found'}
+
+ content = md_file.read_text(errors='replace')
+ entities = extract_entities(content, vault_index, noise_words)
+ relationships = build_relationships(entities)
+
+ return {
+ 'date': date_str,
+ 'status': 'ok',
+ '_meta': build_output_meta(),
+ 'entities': entities,
+ 'relationships': relationships,
+ 'summary': {
+ 'people': len(entities['people']),
+ 'companies': len(entities['companies']),
+ 'projects': len(entities['projects']),
+ 'linked_entities': len(entities['linked_entities']),
+ 'wikilinks': len(entities['wikilinks']),
+ 'financial': len(entities['financial']),
+ 'events': len(entities['events']),
+ 'decisions': len(entities['decisions']),
+ 'relationships': len(relationships),
+ }
+ }
+
+
+def process_full(memory_dir: Path, vault_index: dict, noise_words: set | None = None) -> list:
+ """Process all daily files."""
+ results = []
+ for md in sorted(memory_dir.glob('????-??-??.md')):
+ date_str = md.stem
+ result = process_date(memory_dir, vault_index, date_str, noise_words)
+ if result['status'] == 'ok':
+ results.append(result)
+ return results
+
+
+def main():
+ args = sys.argv[1:]
+ if not args or args[0] in ('-h', '--help'):
+ print(__doc__)
+ sys.exit(0)
+
+ cmd = args[0]
+ if cmd != 'extract':
+ print(f"Unknown command: {cmd}", file=sys.stderr)
+ sys.exit(1)
+
+ memory_dir = Path(args[1]) if len(args) > 1 else None
+ vault_dir = Path(args[2]) if len(args) > 2 else None
+ mode = args[3] if len(args) > 3 else 'today'
+
+ if not memory_dir or not memory_dir.is_dir():
+ print("Error: memory directory required", file=sys.stderr)
+ sys.exit(1)
+
+ # Load schema once for both type inference and noise words
+ try:
+ schema = load_schema()
+ ee_config = get_entity_extraction_config(schema)
+ noise_words = set(ee_config.get('noise_words', []))
+ except FileNotFoundError:
+ schema = {}
+ noise_words = set()
+
+ vault_index = build_vault_index(vault_dir, schema) if vault_dir and vault_dir.is_dir() else {}
+
+ if mode == 'full':
+ results = process_full(memory_dir, vault_index, noise_words)
+ # Save all extracts
+ if vault_dir:
+ out_dir = vault_dir / '.graph'
+ out_dir.mkdir(parents=True, exist_ok=True)
+ for r in results:
+ out_file = out_dir / f"daily-extract-{r['date']}.json"
+ out_file.write_text(json.dumps(r, indent=2, ensure_ascii=False))
+
+ # Summary
+ total_people = sum(r['summary']['people'] for r in results)
+ total_financial = sum(r['summary']['financial'] for r in results)
+ total_events = sum(r['summary']['events'] for r in results)
+ print(f"\n daily extract — full mode")
+ print(f" Files processed: {len(results)}")
+ print(f" People found: {total_people}")
+ print(f" Financial: {total_financial}")
+ print(f" Events: {total_events}")
+ else:
+ # Single date
+ if mode == 'today':
+ mode = date.today().isoformat()
+ result = process_date(memory_dir, vault_index, mode, noise_words)
+
+ if vault_dir:
+ out_dir = vault_dir / '.graph'
+ out_dir.mkdir(parents=True, exist_ok=True)
+ out_file = out_dir / f"daily-extract-{mode}.json"
+ out_file.write_text(json.dumps(result, indent=2, ensure_ascii=False))
+
+ print(json.dumps(result.get('summary', result), indent=2))
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/dedup.py b/vault/.claude/skills/autograph/scripts/dedup.py
new file mode 100644
index 0000000..758030c
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/dedup.py
@@ -0,0 +1,678 @@
+#!/usr/bin/env python3
+"""
+autograph dedup — find duplicate entities and write reproducible cleanup
+manifests before any destructive-ish move.
+
+Usage:
+ python3 dedup.py <vault-dir> [schema.json] # report only
+ python3 dedup.py <vault-dir> [schema.json] --manifest /tmp/m.json # write dry-run manifest
+ python3 dedup.py <vault-dir> [schema.json] --apply-manifest m.json # apply approved entries
+ python3 dedup.py <vault-dir> [schema.json] --apply # legacy apply, blocked by policy schemas
+
+Safety:
+ - NEVER deletes files. Moves extras to <vault>/.trash/dedup-YYYY-MM-DD/
+ - Merges unique content from extras INTO canonical before moving
+ - Logs every action to <vault>/.graph/dedup-log.jsonl
+ - Policy schemas require manifest approval before apply
+"""
+
+import argparse
+import re
+import sys
+import json
+from pathlib import Path
+from collections import defaultdict
+from datetime import datetime
+
+from common import (
+ parse_frontmatter, walk_vault, rel_path,
+ load_schema, get_richness_fields, collect_duplicate_groups,
+ write_frontmatter, infer_domain, infer_type
+)
+
+
+def content_richness(content: str, bonus_fields: list | None = None) -> int:
+ """Score content by information density. PRIMARY signal for canonical pick."""
+ fm, body, _ = parse_frontmatter(content)
+ if fm is None:
+ fm = {}
+ score = 0
+ # Body length (main signal)
+ score += len(body.strip())
+ # Description quality
+ desc = fm.get('description', '')
+ if isinstance(desc, str):
+ score += len(desc) * 2
+ # Tags
+ tags = fm.get('tags', [])
+ if isinstance(tags, list):
+ score += len(tags) * 10
+ # Status presence
+ if fm.get('status'):
+ score += 20
+ # Rich fields from schema
+ for field in (bonus_fields or []):
+ if fm.get(field):
+ score += 15
+ return score
+
+
+def pick_canonical(paths: list[str], vault_dir: Path, bonus_fields: list | None = None) -> tuple[str, list[str]]:
+ """Pick richest file as canonical. Content richness = primary, folder depth = tiebreaker."""
+ scored = []
+ for p in paths:
+ try:
+ content = (vault_dir / p).read_text(errors='replace')
+ except Exception:
+ content = ''
+ richness = content_richness(content, bonus_fields)
+ # Tiebreaker: shallower paths preferred (fewer '/' = more canonical location)
+ depth_score = -p.count('/')
+ scored.append((richness, depth_score, p))
+
+ scored.sort(key=lambda x: (-x[0], -x[1]))
+ canonical = scored[0][2]
+ extras = [s[2] for s in scored[1:]]
+ return canonical, extras
+
+
+def dedup_policy(schema: dict) -> dict:
+ """Return optional policy config from schema."""
+ return schema.get('dedup_policy', {}) or {}
+
+
+def is_policy_schema(schema: dict) -> bool:
+ return bool(dedup_policy(schema).get('path_rules'))
+
+
+def ignored_by_policy(path: str, policy: dict) -> bool:
+ """Skip known inactive namespaces such as ignored backup folders."""
+ for prefix in policy.get('ignored_path_prefixes', []):
+ if path.startswith(prefix):
+ return True
+ return False
+
+
+def path_rule_for(path: str, policy: dict) -> dict:
+ """Find the first ontology rule matching a relative vault path."""
+ for rule in policy.get('path_rules', []):
+ if path.startswith(rule.get('prefix', '')):
+ return rule
+ return {}
+
+
+def policy_kind(path: str, domain: str, card_type: str, policy: dict) -> str:
+ rule = path_rule_for(path, policy)
+ return str(rule.get('kind') or f'{domain}/{card_type}')
+
+
+def policy_domain(path: str, fallback: str, policy: dict) -> str:
+ rule = path_rule_for(path, policy)
+ return str(rule.get('domain') or fallback)
+
+
+def policy_type(path: str, fallback: str, policy: dict) -> str:
+ rule = path_rule_for(path, policy)
+ return str(rule.get('type') or fallback)
+
+
+def policy_canonical(
+ records: list[dict],
+ vault_dir: Path,
+ bonus_fields: list | None,
+ policy: dict
+) -> tuple[str, list[str]]:
+ """Pick canonical by ontology priority first, richness second."""
+ priority = policy.get('canonical_priority', [])
+ ranked = []
+ for rec in records:
+ path = rec['path']
+ prefix_rank = len(priority)
+ for idx, prefix in enumerate(priority):
+ if path.startswith(prefix):
+ prefix_rank = idx
+ break
+ try:
+ content = (vault_dir / path).read_text(errors='replace')
+ except Exception:
+ content = ''
+ ranked.append((prefix_rank, -content_richness(content, bonus_fields), path))
+ ranked.sort()
+ canonical = ranked[0][2]
+ extras = [item[2] for item in ranked[1:]]
+ return canonical, extras
+
+
+def file_record(vault_dir: Path, path: str, schema: dict, policy: dict, bonus_fields: list | None) -> dict:
+ """Build a manifest record for one markdown file."""
+ full = vault_dir / path
+ try:
+ content = full.read_text(errors='replace')
+ except Exception:
+ content = ''
+ fm, body, _ = parse_frontmatter(content)
+ fm = fm or {}
+ inferred_domain = str(fm.get('domain') or infer_domain(path, schema))
+ inferred_type = str(fm.get('type') or infer_type(path, schema))
+ domain = policy_domain(path, inferred_domain, policy)
+ card_type = policy_type(path, inferred_type, policy)
+ return {
+ 'path': path,
+ 'stem': Path(path).stem,
+ 'domain': domain,
+ 'type': card_type,
+ 'kind': policy_kind(path, domain, card_type, policy),
+ 'richness': content_richness(content, bonus_fields),
+ 'title': first_heading(body) or Path(path).stem,
+ 'status': fm.get('status', ''),
+ 'source': fm.get('source', ''),
+ }
+
+
+def first_heading(body: str) -> str:
+ m = re.search(r'^#\s+(.+)$', body, re.MULTILINE)
+ return m.group(1).strip() if m else ''
+
+
+def same_title(records: list[dict]) -> bool:
+ titles = {str(r.get('title') or '').casefold() for r in records if r.get('title')}
+ return len(titles) <= 1
+
+
+def classify_policy_cluster(records: list[dict], policy: dict) -> tuple[str, str, str]:
+ """Return (action, risk, reason) for a same-stem cluster."""
+ stems = {r['stem'] for r in records}
+ kinds = {r['kind'] for r in records}
+ paths = [r['path'] for r in records]
+ manual_stems = set(policy.get('manual_hold_stems', []))
+ high_risk_stems = set(policy.get('high_risk_stems', []))
+
+ if stems & manual_stems:
+ return 'manual_hold', 'high', 'stem is in manual_hold_stems'
+ if stems & high_risk_stems:
+ return 'manual_hold', 'high', 'stem is in high_risk_stems'
+ if len({p.casefold() for p in paths}) != len(paths):
+ return 'manual_hold', 'high', 'case-only path collision'
+ if not same_title(records):
+ return 'manual_hold', 'high', 'same stem but different headings'
+
+ if len(kinds) == 1:
+ return 'merge_duplicate', 'low', 'same stem and same ontology kind'
+
+ crm_kinds = set(policy.get('crm_overlay_kinds', ['crm']))
+ non_crm = kinds - crm_kinds
+ if non_crm and crm_kinds & kinds:
+ return 'thin_crm_overlay', 'medium', 'crm overlay duplicates canonical ontology layers'
+
+ return 'keep_linked_layers', 'medium', 'same stem spans multiple ontology layers'
+
+
+def build_manifest(vault_dir: Path, schema: dict, schema_path: Path | None = None) -> dict:
+ """Build dry-run manifest without changing vault content."""
+ policy = dedup_policy(schema)
+ bonus_fields = get_richness_fields(schema)
+ by_stem = defaultdict(list)
+
+ for md in walk_vault(vault_dir):
+ path = rel_path(md, vault_dir)
+ if ignored_by_policy(path, policy):
+ continue
+ if md.stem in {'_index', 'MEMORY'}:
+ continue
+ by_stem[md.stem].append(file_record(vault_dir, path, schema, policy, bonus_fields))
+
+ clusters = []
+ for slug, records in sorted(by_stem.items()):
+ if len(records) < 2:
+ continue
+ action, risk, reason = classify_policy_cluster(records, policy)
+ canonical = ''
+ extras = []
+ if action == 'merge_duplicate':
+ canonical, extras = policy_canonical(records, vault_dir, bonus_fields, policy)
+ elif action == 'thin_crm_overlay':
+ crm_kinds = set(policy.get('crm_overlay_kinds', ['crm']))
+ non_crm_records = [r for r in records if r['kind'] not in crm_kinds]
+ canonical, _ = policy_canonical(non_crm_records, vault_dir, bonus_fields, policy)
+ extras = [r['path'] for r in records if r['kind'] in crm_kinds]
+ clusters.append({
+ 'id': slug,
+ 'slug': slug,
+ 'action': action,
+ 'risk': risk,
+ 'reason': reason,
+ 'canonical': canonical,
+ 'extras': extras,
+ 'approved': False,
+ 'records': records,
+ })
+
+ return {
+ 'generated_at': datetime.now().isoformat(),
+ 'vault': str(vault_dir),
+ 'schema': str(schema_path) if schema_path else '',
+ 'mode': 'policy',
+ 'clusters': clusters,
+ 'summary': summarize_manifest(clusters),
+ }
+
+
+def summarize_manifest(clusters: list[dict]) -> dict:
+ actions = defaultdict(int)
+ risks = defaultdict(int)
+ extras = 0
+ for cluster in clusters:
+ actions[cluster['action']] += 1
+ risks[cluster['risk']] += 1
+ extras += len(cluster.get('extras', []))
+ return {
+ 'clusters': len(clusters),
+ 'extra_files': extras,
+ 'actions': dict(sorted(actions.items())),
+ 'risks': dict(sorted(risks.items())),
+ }
+
+
+def merge_content(canonical_path: Path, extra_paths: list[Path]) -> bool:
+ """Merge unique info from extras into canonical. Returns True if changed."""
+ canon_content = canonical_path.read_text(errors='replace')
+ canon_fm, canon_body, canon_lines = parse_frontmatter(canon_content)
+ if canon_fm is None:
+ canon_fm = {}
+ changed = False
+
+ for extra_path in extra_paths:
+ try:
+ extra_content = extra_path.read_text(errors='replace')
+ except Exception:
+ continue
+ extra_fm, extra_body, _ = parse_frontmatter(extra_content)
+ if extra_fm is None:
+ extra_fm = {}
+
+ # Merge frontmatter: take richer/non-empty values
+ for key, val in extra_fm.items():
+ canon_val = canon_fm.get(key, '')
+ if not canon_val and val:
+ canon_fm[key] = val
+ changed = True
+ elif isinstance(val, str) and isinstance(canon_val, str) and len(val) > len(canon_val) * 1.5:
+ canon_fm[key] = val
+ changed = True
+ elif isinstance(val, list) and isinstance(canon_val, list) and len(val) > len(canon_val):
+ merged = list(dict.fromkeys(canon_val + val))
+ if merged != canon_val:
+ canon_fm[key] = merged
+ changed = True
+
+ # Merge unique body sections
+ extra_sections = set(re.findall(r'^## (.+)$', extra_body, re.MULTILINE))
+ canon_sections = set(re.findall(r'^## (.+)$', canon_body, re.MULTILINE))
+ for section_name in extra_sections - canon_sections:
+ pattern = rf'^## {re.escape(section_name)}\n(.*?)(?=\n## |\Z)'
+ m = re.search(pattern, extra_body, re.MULTILINE | re.DOTALL)
+ if m:
+ canon_body = canon_body.rstrip() + '\n\n' + m.group(0).strip() + '\n'
+ changed = True
+
+ if changed:
+ new_fm = write_frontmatter(canon_fm, canon_lines)
+ canonical_path.write_text(f"---\n{new_fm}\n---\n{canon_body}")
+
+ return changed
+
+
+def redirect_links(vault_dir: Path, old_paths: list[str], canonical: str) -> int:
+ """Update wikilinks pointing to old paths → canonical."""
+ canonical_noext = canonical.replace('.md', '')
+
+ old_refs = set()
+ for old in old_paths:
+ old_noext = old.replace('.md', '')
+ old_refs.add(old_noext)
+ old_refs.add(Path(old).stem)
+ parts = old_noext.split('/')
+ for i in range(len(parts)):
+ old_refs.add('/'.join(parts[i:]))
+
+ # Don't redirect canonical references
+ canon_parts = canonical_noext.split('/')
+ for i in range(len(canon_parts)):
+ old_refs.discard('/'.join(canon_parts[i:]))
+ old_refs.discard(Path(canonical).stem)
+
+ if not old_refs:
+ return 0
+
+ count = 0
+ for md in walk_vault(vault_dir):
+ try:
+ content = md.read_text(errors='replace')
+ except Exception:
+ continue
+
+ new_content = content
+ for old_ref in old_refs:
+ pattern = re.compile(r'\[\[' + re.escape(old_ref) + r'(\|[^\]]+)?\]\]')
+ if pattern.search(new_content):
+ new_content = pattern.sub(
+ lambda m: f'[[{canonical_noext}{m.group(1) or ""}]]',
+ new_content
+ )
+ count += 1
+
+ if new_content != content:
+ md.write_text(new_content)
+
+ return count
+
+
+def thin_crm_overlay(
+ vault_dir: Path, crm_path: str, canonical: str, schema: dict | None = None
+) -> bool:
+ """Replace a CRM duplicate body with a compact status overlay."""
+ full = vault_dir / crm_path
+ if not full.exists():
+ return False
+ content = full.read_text(errors='replace')
+ fm, body, fm_lines = parse_frontmatter(content)
+ fm = fm or {}
+ canonical_noext = canonical.replace('.md', '')
+ title = first_heading(body) or Path(crm_path).stem
+ fm['canonical'] = f'[[{canonical_noext}]]'
+ fm['type'] = 'crm'
+ # Domain comes from the schema's domain_inference — never hardcoded.
+ domain = infer_domain(crm_path, schema) if schema else ''
+ if domain:
+ fm['domain'] = domain
+ new_fm = write_frontmatter(fm, fm_lines)
+ status = fm.get('status', 'active')
+ new_body = (
+ f"# {title}\n\n"
+ "## CRM Overlay\n"
+ f"- Canonical: [[{canonical_noext}]]\n"
+ f"- Status: {status}\n"
+ )
+ new_content = f"---\n{new_fm}\n---\n{new_body}"
+ if new_content == content:
+ return False
+ full.write_text(new_content)
+ return True
+
+
+def apply_manifest(
+ vault_dir: Path, manifest_path: Path, verbose=False, schema: dict | None = None
+) -> dict:
+ """Apply only entries explicitly marked approved in a manifest."""
+ manifest = json.loads(manifest_path.read_text())
+ today = datetime.now().strftime('%Y-%m-%d')
+ trash_dir = vault_dir / '.trash' / f'dedup-{today}'
+ log_path = vault_dir / '.graph' / 'dedup-log.jsonl'
+ log_path.parent.mkdir(exist_ok=True)
+
+ applied = {
+ 'approved': 0,
+ 'merged': 0,
+ 'thinned': 0,
+ 'moved': 0,
+ 'links_redirected': 0,
+ 'skipped': 0,
+ }
+
+ with open(log_path, 'a') as log_file:
+ for cluster in manifest.get('clusters', []):
+ if not cluster.get('approved'):
+ continue
+ applied['approved'] += 1
+ action = cluster.get('action')
+ canonical = cluster.get('canonical')
+ extras = list(cluster.get('extras') or [])
+ if not canonical:
+ applied['skipped'] += 1
+ continue
+
+ if action == 'merge_duplicate':
+ trash_dir.mkdir(parents=True, exist_ok=True)
+ extra_full = [vault_dir / e for e in extras]
+ merge_content(vault_dir / canonical, extra_full)
+ lr = redirect_links(vault_dir, extras, canonical)
+ moved = []
+ for extra in extras:
+ src = vault_dir / extra
+ if not src.exists():
+ continue
+ dest = trash_dir / extra
+ dest.parent.mkdir(parents=True, exist_ok=True)
+ src.rename(dest)
+ moved.append(extra)
+ applied['merged'] += 1
+ applied['moved'] += len(moved)
+ applied['links_redirected'] += lr
+ log_entry = {
+ 'ts': datetime.now().isoformat(),
+ 'action': action,
+ 'slug': cluster.get('slug'),
+ 'canonical': canonical,
+ 'moved': moved,
+ 'links_redirected': lr,
+ 'manifest': str(manifest_path),
+ }
+ log_file.write(json.dumps(log_entry, ensure_ascii=False) + '\n')
+ if verbose:
+ print(f"MERGED {cluster.get('slug')}: {len(moved)} moved")
+
+ elif action == 'thin_crm_overlay':
+ # build_manifest puts only overlay-kind paths (crm_overlay_kinds)
+ # into extras, so trust the approved manifest — no path hardcode.
+ crm_paths = list(extras)
+ if not crm_paths:
+ applied['skipped'] += 1
+ continue
+ changed = 0
+ for crm_path in crm_paths:
+ if thin_crm_overlay(vault_dir, crm_path, canonical, schema):
+ changed += 1
+ applied['thinned'] += changed
+ log_entry = {
+ 'ts': datetime.now().isoformat(),
+ 'action': action,
+ 'slug': cluster.get('slug'),
+ 'canonical': canonical,
+ 'thinned': crm_paths,
+ 'manifest': str(manifest_path),
+ }
+ log_file.write(json.dumps(log_entry, ensure_ascii=False) + '\n')
+ if verbose:
+ print(f"THINNED {cluster.get('slug')}: {changed} overlays")
+ else:
+ applied['skipped'] += 1
+
+ out = vault_dir / '.graph' / 'dedup-apply-report.json'
+ out.write_text(json.dumps({
+ 'date': today,
+ 'manifest': str(manifest_path),
+ **applied,
+ }, indent=2, ensure_ascii=False))
+ return applied
+
+
+def write_manifest(path: Path, manifest: dict) -> None:
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text(json.dumps(manifest, indent=2, ensure_ascii=False))
+
+
+def print_policy_manifest(manifest: dict, verbose=False) -> None:
+ summary = manifest['summary']
+ print(f"\n{'='*55}")
+ print(" AUTOGRAPH DEDUP - POLICY DRY RUN")
+ print(f"{'='*55}")
+ print(f" Duplicate clusters: {summary['clusters']}")
+ print(f" Extra files: {summary['extra_files']}")
+ print(f" Actions: {summary['actions']}")
+ print(f" Risks: {summary['risks']}")
+ if verbose:
+ for cluster in manifest['clusters']:
+ print(f"\n {cluster['slug']} [{cluster['action']}/{cluster['risk']}]:")
+ if cluster.get('canonical'):
+ print(f" CANONICAL: {cluster['canonical']}")
+ for rec in cluster.get('records', []):
+ marker = ' '
+ if rec['path'] == cluster.get('canonical'):
+ marker = ' * '
+ print(f"{marker}{rec['path']} ({rec['kind']}, richness={rec['richness']})")
+ if cluster.get('extras'):
+ print(f" EXTRAS: {', '.join(cluster['extras'])}")
+ print(f" REASON: {cluster['reason']}")
+
+
+def dedup(vault_dir: Path, schema_path: Path | None = None, apply=False,
+ verbose=False, manifest_path: Path | None = None,
+ apply_manifest_path: Path | None = None):
+ today = datetime.now().strftime('%Y-%m-%d')
+ trash_dir = vault_dir / '.trash' / f'dedup-{today}'
+ log_path = vault_dir / '.graph' / 'dedup-log.jsonl'
+
+ try:
+ schema = load_schema(schema_path)
+ except FileNotFoundError:
+ schema = {}
+
+ if apply_manifest_path:
+ result = apply_manifest(vault_dir, apply_manifest_path, verbose, schema=schema)
+ print(f"Applied manifest: {result}")
+ return
+
+ if is_policy_schema(schema):
+ if apply:
+ print("ERROR: policy schema requires --apply-manifest with approved entries.", file=sys.stderr)
+ sys.exit(2)
+ manifest = build_manifest(vault_dir, schema, schema_path)
+ print_policy_manifest(manifest, verbose)
+ if manifest_path:
+ write_manifest(manifest_path, manifest)
+ print(f" Manifest: {manifest_path}")
+ out = vault_dir / '.graph' / 'dedup-report.json'
+ out.parent.mkdir(exist_ok=True)
+ summary = manifest['summary']
+ out.write_text(json.dumps({
+ 'date': today,
+ 'mode': 'policy',
+ 'duplicates': summary['clusters'],
+ 'extra_files': summary['extra_files'],
+ 'actions': summary['actions'],
+ 'risks': summary['risks'],
+ 'manifest': str(manifest_path) if manifest_path else '',
+ }, indent=2, ensure_ascii=False))
+ print(f" Report: {out}")
+ return
+
+ bonus_fields = get_richness_fields(schema)
+
+ # Find only safe duplicates: same stem + domain + type
+ dupes = collect_duplicate_groups(vault_dir, schema)
+
+ if not dupes:
+ print("No duplicates found.")
+ return
+
+ total_extra = sum(len(paths) - 1 for paths in dupes.values())
+
+ print(f"\n{'='*55}")
+ print(f" AUTOGRAPH DEDUP — {'APPLY' if apply else 'DRY RUN'}")
+ print(f"{'='*55}")
+ print(f" Duplicate groups: {len(dupes)}")
+ print(f" Extra files: {total_extra}")
+
+ log_file = None
+ if apply:
+ trash_dir.mkdir(parents=True, exist_ok=True)
+ log_path.parent.mkdir(exist_ok=True)
+ log_file = open(log_path, 'a')
+
+ merged_count = 0
+ content_merged = 0
+ links_redirected = 0
+ moved_count = 0
+
+ for (slug, domain, card_type), paths in sorted(dupes.items(), key=lambda x: (-len(x[1]), x[0])):
+ canonical, extras = pick_canonical(paths, vault_dir, bonus_fields)
+
+ if verbose:
+ canon_rich = content_richness((vault_dir / canonical).read_text(errors='replace'), bonus_fields)
+ print(f"\n {slug} [{domain}/{card_type}] ({len(paths)}x):")
+ print(f" KEEP: {canonical} (richness={canon_rich})")
+ for e in extras:
+ e_rich = content_richness((vault_dir / e).read_text(errors='replace'), bonus_fields)
+ print(f" MOVE: {e} (richness={e_rich})")
+
+ if apply:
+ extra_full = [vault_dir / e for e in extras]
+ did_merge = merge_content(vault_dir / canonical, extra_full)
+ if did_merge:
+ content_merged += 1
+
+ lr = redirect_links(vault_dir, extras, canonical)
+ links_redirected += lr
+
+ for extra in extras:
+ src = vault_dir / extra
+ dest = trash_dir / extra
+ dest.parent.mkdir(parents=True, exist_ok=True)
+ src.rename(dest)
+ moved_count += 1
+
+ if log_file:
+ log_file.write(json.dumps({
+ 'ts': datetime.now().isoformat(), 'slug': slug,
+ 'domain': domain, 'type': card_type,
+ 'canonical': canonical, 'moved': extras,
+ 'content_merged': did_merge, 'links_redirected': lr,
+ }, ensure_ascii=False) + '\n')
+
+ merged_count += 1
+
+ if log_file:
+ log_file.close()
+
+ print(f"\n Results:")
+ print(f" Slugs processed: {merged_count}")
+ if apply:
+ print(f" Content merged: {content_merged}")
+ print(f" Links redirected: {links_redirected}")
+ print(f" Files moved: {moved_count} → {trash_dir}")
+ print(f" Log: {log_path}")
+ else:
+ print(f" Would move: {total_extra} files to .trash/")
+
+ out = vault_dir / '.graph' / 'dedup-report.json'
+ out.parent.mkdir(exist_ok=True)
+ out.write_text(json.dumps({
+ 'date': today, 'duplicates': len(dupes), 'extra_files': total_extra,
+ 'merged': merged_count, 'moved': moved_count if apply else 0,
+ 'content_merged': content_merged if apply else 0,
+ }, indent=2, ensure_ascii=False))
+ print(f" Report: {out}")
+
+
+def parse_args(argv: list[str]) -> argparse.Namespace:
+ parser = argparse.ArgumentParser(description='autograph dedup')
+ parser.add_argument('vault_dir')
+ parser.add_argument('schema_path', nargs='?')
+ parser.add_argument('--dry-run', action='store_true', help='explicit no-op dry run')
+ parser.add_argument('--apply', action='store_true', help='legacy apply; blocked for policy schemas')
+ parser.add_argument('--verbose', action='store_true')
+ parser.add_argument('--manifest', help='write dry-run manifest JSON')
+ parser.add_argument('--apply-manifest', help='apply approved entries from manifest JSON')
+ return parser.parse_args(argv)
+
+
+if __name__ == '__main__':
+ ns = parse_args(sys.argv[1:])
+ dedup(
+ Path(ns.vault_dir),
+ Path(ns.schema_path) if ns.schema_path else None,
+ apply=ns.apply,
+ verbose=ns.verbose,
+ manifest_path=Path(ns.manifest) if ns.manifest else None,
+ apply_manifest_path=Path(ns.apply_manifest) if ns.apply_manifest else None,
+ )
diff --git a/vault/.claude/skills/autograph/scripts/discover.py b/vault/.claude/skills/autograph/scripts/discover.py
new file mode 100644
index 0000000..192e893
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/discover.py
@@ -0,0 +1,153 @@
+#!/usr/bin/env python3
+"""
+autograph discover — Phase 1: scan vault, extract natural enums.
+Finds all frontmatter values, wikilink patterns, folder structures.
+Outputs discovered schema candidates to stdout as JSON.
+
+Usage: python3 discover.py <vault-dir> [--verbose]
+"""
+
+import re
+import json
+import sys
+from pathlib import Path
+from collections import defaultdict, Counter
+from datetime import datetime
+
+from common import parse_frontmatter, walk_vault, rel_path, extract_wikilinks
+
+
+def scan_vault(vault_dir: Path, verbose=False):
+ """Scan entire vault, collect all metadata patterns."""
+
+ field_values = defaultdict(Counter)
+ field_presence = Counter()
+ folder_patterns = Counter()
+ wikilink_patterns = defaultdict(Counter)
+ edge_contexts = []
+ section_headers = Counter()
+ tag_frequency = Counter()
+ file_count = 0
+ no_frontmatter = 0
+ missing_required = defaultdict(int)
+
+ for md_file in walk_vault(vault_dir):
+ file_count += 1
+ rp = rel_path(md_file, vault_dir)
+ parts = Path(rp).parts
+ folder = str(Path(rp).parent) if len(parts) > 1 else 'root'
+ top_folder = parts[0] if len(parts) > 1 else 'root'
+
+ folder_patterns[folder] += 1
+
+ try:
+ content = md_file.read_text(errors='replace')
+ except Exception:
+ continue
+
+ fm, body, _ = parse_frontmatter(content)
+ if fm is None:
+ no_frontmatter += 1
+ continue
+
+ for key, val in fm.items():
+ field_presence[key] += 1
+ if isinstance(val, list):
+ for item in val:
+ if isinstance(item, str):
+ field_values[key][item.lower()] += 1
+ if key == 'tags':
+ tag_frequency[item.lower()] += 1
+ elif isinstance(val, str) and len(val) < 100:
+ field_values[key][val.lower()] += 1
+
+ for req in ['type', 'description', 'tags', 'status']:
+ if req not in fm:
+ missing_required[req] += 1
+
+ source_type = fm.get('type', 'unknown')
+ for target, _ in extract_wikilinks(body):
+ target_top = target.split('/')[0] if '/' in target else 'self'
+ wikilink_patterns[top_folder][target_top] += 1
+
+ if verbose and len(edge_contexts) < 200:
+ edge_contexts.append({
+ 'source_type': source_type,
+ 'source_file': rp,
+ 'target': target,
+ })
+
+ for h in re.findall(r'^## (.+)$', body, re.MULTILINE):
+ section_headers[h.strip().lower()] += 1
+
+ # Build discovered schema
+ discovered = {
+ 'meta': {
+ 'vault_dir': str(vault_dir),
+ 'total_files': file_count,
+ 'no_frontmatter': no_frontmatter,
+ 'frontmatter_coverage': round((file_count - no_frontmatter) / max(file_count, 1) * 100, 1),
+ 'scanned_at': datetime.now().isoformat(),
+ },
+ 'enums': {},
+ 'missing_required': dict(missing_required),
+ 'folder_structure': {},
+ 'edge_patterns': {},
+ 'top_tags': [],
+ 'section_headers': [],
+ }
+
+ for field, values in sorted(field_values.items()):
+ unique = len(values)
+ total = sum(values.values())
+ if unique <= 30 and total >= 3:
+ top_vals = values.most_common(30)
+ discovered['enums'][field] = {
+ 'unique_values': unique,
+ 'total_occurrences': total,
+ 'coverage': round(total / max(file_count, 1) * 100, 1),
+ 'values': {v: c for v, c in top_vals}
+ }
+
+ folder_groups = defaultdict(int)
+ for folder, count in folder_patterns.most_common():
+ top = folder.split('/')[0]
+ folder_groups[top] += count
+ discovered['folder_structure'] = dict(sorted(folder_groups.items(), key=lambda x: -x[1])[:20])
+
+ for src, targets in wikilink_patterns.items():
+ top_targets = targets.most_common(10)
+ if top_targets:
+ discovered['edge_patterns'][src] = {t: c for t, c in top_targets}
+
+ discovered['top_tags'] = [{'tag': t, 'count': c} for t, c in tag_frequency.most_common(30)]
+ discovered['section_headers'] = [{'header': h, 'count': c} for h, c in section_headers.most_common(20)]
+ discovered['field_presence'] = {
+ k: {'count': v, 'coverage': round(v / max(file_count, 1) * 100, 1)}
+ for k, v in field_presence.most_common(30)
+ }
+
+ if verbose:
+ discovered['edge_samples'] = edge_contexts[:50]
+
+ return discovered
+
+
+def main():
+ if len(sys.argv) < 2:
+ print("Usage: discover.py <vault-dir> [--verbose]", file=sys.stderr)
+ sys.exit(1)
+
+ vault_dir = Path(sys.argv[1])
+ verbose = '--verbose' in sys.argv
+
+ if not vault_dir.is_dir():
+ print(f"Error: {vault_dir} is not a directory", file=sys.stderr)
+ sys.exit(1)
+
+ result = scan_vault(vault_dir, verbose=verbose)
+ print(json.dumps(result, ensure_ascii=False, indent=2))
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/enforce.py b/vault/.claude/skills/autograph/scripts/enforce.py
new file mode 100644
index 0000000..70b44f3
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/enforce.py
@@ -0,0 +1,241 @@
+#!/usr/bin/env python3
+"""
+autograph enforce — validate and fix vault cards against schema.
+
+Usage:
+ python3 enforce.py <vault-dir> <schema.json> # dry run
+ python3 enforce.py <vault-dir> <schema.json> --apply # apply fixes
+ python3 enforce.py <vault-dir> <schema.json> --verbose
+"""
+
+import json
+import sys
+from pathlib import Path
+from datetime import date
+from collections import defaultdict
+
+from common import (
+ load_schema, parse_frontmatter, write_frontmatter, format_field,
+ walk_vault, rel_path, infer_domain, infer_type, IGNORE_DIRS,
+ get_type_aliases, get_field_fixes, get_node_types, get_status_defaults,
+ collect_duplicate_groups
+)
+
+
+def enforce(vault_dir: Path, schema: dict, apply=False, verbose=False):
+ node_types = schema['node_types']
+ type_aliases = get_type_aliases(schema)
+ field_fixes = get_field_fixes(schema)
+ region_fixes = schema.get('region_fixes', {})
+
+ stats = {
+ 'total': 0, 'valid': 0, 'fixed': 0, 'needs_review': 0,
+ 'no_fm': 0, 'fixes': defaultdict(int), 'review_items': []
+ }
+ for md in walk_vault(vault_dir):
+ rp = rel_path(md, vault_dir)
+ stats['total'] += 1
+
+ try:
+ content = md.read_text(errors='replace')
+ except Exception:
+ continue
+
+ fields, body, orig_lines = parse_frontmatter(content)
+ if fields is None:
+ stats['no_fm'] += 1
+ continue
+
+ changed = False
+ issues = []
+
+ # --- TYPE ---
+ t = fields.get('type', '')
+ if t in type_aliases:
+ fields['type'] = type_aliases[t]
+ changed = True
+ stats['fixes']['type_alias'] += 1
+ t = fields.get('type', '')
+ if not t or t not in node_types:
+ fields['type'] = infer_type(rp, schema)
+ changed = True
+ stats['fixes']['type_inferred'] += 1
+ t = fields['type']
+ tdef = node_types.get(t, {})
+
+ # --- STATUS ---
+ valid_s = tdef.get('status', [])
+ cur_s = fields.get('status', '')
+ # Apply field_fixes for status
+ status_fixes = field_fixes.get('status', {})
+ if cur_s and cur_s.lower() in status_fixes:
+ fields['status'] = status_fixes[cur_s.lower()]
+ changed = True
+ stats['fixes']['status_fix'] += 1
+ cur_s = fields['status']
+ # Validate
+ if valid_s and cur_s and cur_s not in valid_s:
+ # Fall back to first valid status for this type
+ fields['status'] = valid_s[0]
+ changed = True
+ stats['fixes']['status_remap'] += 1
+ if not cur_s and valid_s:
+ # Use status_defaults from schema, or first valid status
+ defaults = get_status_defaults(schema)
+ fields['status'] = defaults.get(t, defaults.get('default', valid_s[0]))
+ changed = True
+ stats['fixes']['status_default'] += 1
+
+ # --- PRIORITY ---
+ pri = fields.get('priority', '')
+ priority_fixes = field_fixes.get('priority', {})
+ if pri and pri.lower() in priority_fixes:
+ fields['priority'] = priority_fixes[pri.lower()]
+ changed = True
+ stats['fixes']['priority_fix'] += 1
+
+ # --- POTENTIAL ---
+ pot = fields.get('potential', '')
+ potential_fixes = field_fixes.get('potential', {})
+ if pot and pot.lower() in potential_fixes:
+ fields['potential'] = potential_fixes[pot.lower()]
+ changed = True
+ stats['fixes']['potential_fix'] += 1
+
+ # --- REGION ---
+ reg = fields.get('region', '')
+ if isinstance(reg, str) and reg in region_fixes:
+ fields['region'] = region_fixes[reg]
+ changed = True
+ stats['fixes']['region_fix'] += 1
+
+ # --- DOMAIN ---
+ if 'domain' not in fields or not fields['domain']:
+ fields['domain'] = infer_domain(rp, schema)
+ changed = True
+ stats['fixes']['domain_add'] += 1
+
+ # --- DESCRIPTION ---
+ desc = fields.get('description', '')
+ if not desc:
+ first = body.strip().split('\n')[0].strip().lstrip('#').strip() if body.strip() else ''
+ if first and 10 < len(first) < 200:
+ fields['description'] = first
+ changed = True
+ stats['fixes']['desc_inferred'] += 1
+ else:
+ issues.append('missing description')
+ elif isinstance(desc, str) and len(desc) > 20:
+ # Detect duplicated description (processor bug: same text repeated)
+ half = len(desc) // 2
+ first_half = desc[:half].strip()
+ second_half = desc[half:].strip()
+ if first_half and first_half == second_half:
+ fields['description'] = first_half
+ changed = True
+ stats['fixes']['desc_dedup'] += 1
+
+ # --- TAGS ---
+ tags = fields.get('tags', '')
+ if not tags or (isinstance(tags, list) and len(tags) == 0):
+ issues.append('missing tags')
+
+ # --- SYSTEM FIELDS ---
+ today_str = date.today().isoformat()
+ for sf, default in [('last_accessed', today_str), ('tier', 'warm'), ('relevance', 0.5)]:
+ if sf not in fields or not fields[sf]:
+ fields[sf] = default
+ changed = True
+ stats['fixes'][f'{sf}_add'] += 1
+
+ # --- WRITE ---
+ if changed and apply:
+ new_fm = write_frontmatter(fields, orig_lines)
+ md.write_text(f"---\n{new_fm}\n---\n{body}")
+
+ if changed:
+ stats['fixed'] += 1
+ if issues:
+ stats['needs_review'] += 1
+ if verbose:
+ for i in issues:
+ stats['review_items'].append(f"{rp}: {i}")
+ if not changed and not issues:
+ stats['valid'] += 1
+
+ dupes = collect_duplicate_groups(vault_dir, schema)
+ return stats, dupes
+
+
+def health_score(stats, dupes):
+ total = max(stats['total'], 1)
+ fm_ok = 1 - stats['no_fm'] / total
+ valid_rate = (stats['valid'] + stats['fixed']) / total
+ desc_miss = stats['fixes'].get('desc_inferred', 0) + len([r for r in stats.get('review_items', []) if 'description' in r])
+ tags_miss = len([r for r in stats.get('review_items', []) if 'tags' in r])
+ dup_count = sum(len(p) - 1 for p in dupes.values())
+
+ score = 100.0
+ score -= (1 - fm_ok) * 10
+ score -= (1 - valid_rate) * 25
+ score -= (desc_miss / total) * 20
+ score -= (tags_miss / total) * 15
+ score -= min(dup_count, 50) * 0.3
+ score -= stats['needs_review'] / total * 10
+ return max(0, round(score, 1))
+
+
+def main():
+ args = sys.argv[1:]
+ if len(args) < 2:
+ print("Usage: enforce.py <vault-dir> <schema.json> [--apply] [--verbose]", file=sys.stderr)
+ sys.exit(1)
+
+ vault_dir = Path(args[0])
+ schema_path = Path(args[1])
+ apply = '--apply' in args
+ verbose = '--verbose' in args
+
+ schema = load_schema(schema_path)
+ stats, dupes = enforce(vault_dir, schema, apply=apply, verbose=verbose)
+ score = health_score(stats, dupes)
+
+ mode = "APPLIED" if apply else "DRY RUN"
+ print(f"\n{'='*55}")
+ print(f" AUTOGRAPH ENFORCE — {mode}")
+ print(f"{'='*55}")
+ print(f" Total: {stats['total']}")
+ print(f" Valid: {stats['valid']}")
+ print(f" Auto-fixed: {stats['fixed']}")
+ print(f" Needs review: {stats['needs_review']}")
+ print(f" No frontmatter: {stats['no_fm']}")
+ print(f"\n Fixes:")
+ for k, v in sorted(stats['fixes'].items(), key=lambda x: -x[1]):
+ print(f" {k}: {v}")
+ if dupes:
+ dup_total = sum(len(p) - 1 for p in dupes.values())
+ print(f"\n Duplicates: {len(dupes)} slugs ({dup_total} extra files)")
+ for (slug, domain, card_type), paths in sorted(dupes.items(), key=lambda x: -len(x[1]))[:15]:
+ print(f" {slug} [{domain}/{card_type}] ({len(paths)}x)")
+ if verbose and stats.get('review_items'):
+ print(f"\n Review ({len(stats['review_items'])}):")
+ for r in stats['review_items'][:20]:
+ print(f" {r}")
+
+ print(f"\n {'='*40}")
+ print(f" SCHEMA COMPLIANCE: {score}/100")
+ print(f" {'='*40}")
+
+ out = vault_dir / '.graph' / 'enforce-report.json'
+ out.parent.mkdir(exist_ok=True)
+ out.write_text(json.dumps({
+ 'score': score, 'total': stats['total'], 'valid': stats['valid'],
+ 'fixed': stats['fixed'], 'review': stats['needs_review'],
+ 'duplicates': len(dupes), 'mode': mode,
+ 'fixes': dict(stats['fixes']),
+ }, indent=2))
+ print(f" Report: {out}")
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/engine.py b/vault/.claude/skills/autograph/scripts/engine.py
new file mode 100755
index 0000000..8126925
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/engine.py
@@ -0,0 +1,325 @@
+#!/usr/bin/env python3
+"""
+autograph engine — decay, touch, creative recall, stats.
+All decay config from schema.json. No hardcoded rates or thresholds.
+
+Commands:
+ engine.py decay <vault-dir> [schema.json] [--dry-run]
+ engine.py touch <filepath> [schema.json]
+ engine.py creative <N> <vault-dir> [schema.json]
+ engine.py stats <vault-dir> [schema.json]
+ engine.py init <vault-dir> [schema.json] [--dry-run]
+"""
+
+import sys
+import json
+import random
+from pathlib import Path
+from datetime import date
+from collections import defaultdict
+
+from common import (
+ load_schema, parse_frontmatter, write_frontmatter, walk_vault, rel_path,
+ infer_domain, infer_type, calc_relevance, calc_tier, days_since,
+ get_decay_config, get_node_types
+)
+
+
+def find_schema(args: list) -> Path | None:
+ for a in args:
+ if a.endswith('.json') and Path(a).exists():
+ return Path(a)
+ default = Path(__file__).parent.parent / 'schema.json'
+ return default if default.exists() else None
+
+
+def cmd_decay(vault_dir: Path, schema: dict, dry_run: bool = False):
+ """Recalculate relevance and tier for all cards."""
+ files = walk_vault(vault_dir)
+ updated = 0
+ today = date.today()
+
+ for md in files:
+ try:
+ content = md.read_text(errors='replace')
+ except Exception:
+ continue
+
+ fm, body, orig_lines = parse_frontmatter(content)
+ if fm is None:
+ continue
+
+ # Skip core tier
+ if fm.get('tier') == 'core':
+ continue
+
+ last = fm.get('last_accessed', fm.get('updated', fm.get('created', '')))
+ d = days_since(last)
+
+ # Domain-specific rate + access_count
+ file_type = fm.get('type', '')
+ try:
+ access_count = int(fm.get('access_count', 1))
+ except (ValueError, TypeError):
+ access_count = 1
+
+ new_rel = calc_relevance(d, schema, access_count, file_type)
+ new_tier = calc_tier(d, schema, fm.get('tier', ''))
+
+ old_rel = fm.get('relevance', '')
+ old_tier = fm.get('tier', '')
+
+ # Check if update needed
+ try:
+ old_rel_f = float(old_rel) if old_rel else -1
+ except (ValueError, TypeError):
+ old_rel_f = -1
+
+ if abs(old_rel_f - new_rel) < 0.005 and old_tier == new_tier:
+ continue
+
+ fm['relevance'] = new_rel
+ fm['tier'] = new_tier
+ updated += 1
+
+ if not dry_run:
+ new_fm = write_frontmatter(fm, orig_lines)
+ md.write_text(f"---\n{new_fm}\n---\n{body}")
+
+ config = get_decay_config(schema)
+ mode = "DRY RUN" if dry_run else "APPLIED"
+ print(f"\n decay — {mode}")
+ print(f" Config: rate={config.get('rate')}, floor={config.get('floor')}")
+ print(f" Cards scanned: {len(files)}")
+ print(f" Cards updated: {updated}")
+
+
+def cmd_touch(filepath: str, schema: dict):
+ """Touch card — graduated recall (archive→cold→warm→active) + access_count."""
+ from datetime import timedelta
+ fp = Path(filepath)
+ if not fp.exists():
+ print(f"Error: {filepath} not found", file=sys.stderr)
+ sys.exit(1)
+
+ content = fp.read_text(errors='replace')
+ fm, body, orig_lines = parse_frontmatter(content)
+ if fm is None:
+ print(f"Error: no frontmatter in {filepath}", file=sys.stderr)
+ sys.exit(1)
+
+ # Increment access_count
+ try:
+ access_count = int(fm.get('access_count', 0)) + 1
+ except (ValueError, TypeError):
+ access_count = 1
+ fm['access_count'] = access_count
+
+ current_tier = fm.get('tier', 'archive')
+ config = get_decay_config(schema)
+ tiers = config.get('tiers', {'active': 7, 'warm': 21, 'cold': 60})
+
+ if current_tier == 'core':
+ # Core: just refresh timestamp
+ fm['last_accessed'] = date.today().isoformat()
+ fm['relevance'] = 1.0
+ elif current_tier in ('active', ''):
+ # Already active: refresh
+ fm['last_accessed'] = date.today().isoformat()
+ fm['relevance'] = 1.0
+ fm['tier'] = 'active'
+ else:
+ # Graduated: promote one tier up
+ promotion = {
+ 'archive': ('cold', (tiers.get('warm', 21) + tiers.get('cold', 60)) // 2),
+ 'cold': ('warm', (tiers.get('active', 7) + tiers.get('warm', 21)) // 2),
+ 'warm': ('active', tiers.get('active', 7) // 2),
+ }
+ new_tier, target_days = promotion.get(current_tier, ('active', 0))
+ new_date = date.today() - timedelta(days=target_days)
+ fm['last_accessed'] = new_date.isoformat()
+ file_type = fm.get('type', '')
+ fm['relevance'] = calc_relevance(target_days, schema, access_count, file_type)
+ fm['tier'] = new_tier
+
+ new_fm = write_frontmatter(fm, orig_lines)
+ fp.write_text(f"---\n{new_fm}\n---\n{body}")
+ new_tier = fm.get('tier', current_tier)
+ print(f" touched: {filepath} → {current_tier}→{new_tier}, "
+ f"relevance={fm['relevance']}, access_count={access_count}")
+
+
+def cmd_creative(n: int, vault_dir: Path, schema: dict):
+ """Random sample from cold+archive tiers for serendipitous discovery."""
+ files = walk_vault(vault_dir)
+ cold_archive = []
+
+ for md in files:
+ try:
+ content = md.read_text(errors='replace')
+ except Exception:
+ continue
+ fm, _, _ = parse_frontmatter(content)
+ if fm is None:
+ continue
+ tier = fm.get('tier', '')
+ if tier in ('cold', 'archive'):
+ desc = fm.get('description', md.stem.replace('-', ' '))
+ cold_archive.append((rel_path(md, vault_dir), desc, tier))
+
+ if not cold_archive:
+ print("No cold/archive cards found.")
+ return
+
+ sample = random.sample(cold_archive, min(n, len(cold_archive)))
+ print(f"\n creative recall — {len(sample)} cards from {len(cold_archive)} cold/archive")
+ print()
+ for rp, desc, tier in sample:
+ desc_short = desc[:80] if isinstance(desc, str) else str(desc)[:80]
+ print(f" [{tier}] {rp}")
+ print(f" {desc_short}")
+ print()
+
+
+def cmd_stats(vault_dir: Path, schema: dict):
+ """Show tier distribution and context budget."""
+ files = walk_vault(vault_dir)
+ tiers = defaultdict(int)
+ total_size = 0
+ active_size = 0
+ no_fm = 0
+ stale_90 = 0
+
+ for md in files:
+ try:
+ content = md.read_text(errors='replace')
+ total_size += len(content)
+ except Exception:
+ continue
+ fm, _, _ = parse_frontmatter(content)
+ if fm is None:
+ no_fm += 1
+ tiers['unknown'] += 1
+ continue
+ tier = fm.get('tier', 'unknown')
+ tiers[tier] += 1
+ if tier in ('active', 'warm'):
+ active_size += len(content)
+ last = fm.get('last_accessed', '')
+ if days_since(last) > 90:
+ stale_90 += 1
+
+ total = len(files)
+ config = get_decay_config(schema)
+
+ print(f"\n memory health — {vault_dir}")
+ print(f" {'─'*40}")
+ print(f" total cards: {total}")
+ print(f" total size: {total_size // 1024} KB")
+ print(f" without yaml: {no_fm}")
+ print(f" stale (>90 days): {stale_90}")
+ print(f" decay rate: {config.get('rate', '?')}/day")
+ print(f" {'─'*40}")
+ print(f" tier distribution:")
+
+ tier_order = ['core', 'active', 'warm', 'cold', 'archive', 'unknown']
+ max_bar = 50
+ for t in tier_order:
+ count = tiers.get(t, 0)
+ if count == 0:
+ continue
+ pct = count / max(total, 1) * 100
+ bar_len = int(pct / 100 * max_bar)
+ bar = '█' * bar_len
+ print(f" {t:<8}: {count:>4} ({pct:>4.1f}%) {bar}")
+
+ print(f" {'─'*40}")
+ print(f" active context: {active_size // 1024} KB (~{active_size // 4:,} tokens)")
+ print(f" total context: {total_size // 1024} KB (~{total_size // 4:,} tokens)")
+
+
+def cmd_init(vault_dir: Path, schema: dict, dry_run: bool = False):
+ """Bootstrap frontmatter on files missing it."""
+ files = walk_vault(vault_dir)
+ added = 0
+
+ for md in files:
+ try:
+ content = md.read_text(errors='replace')
+ except Exception:
+ continue
+ fm, body, _ = parse_frontmatter(content)
+ if fm is not None:
+ continue # already has frontmatter
+
+ rp = rel_path(md, vault_dir)
+ card_type = infer_type(rp, schema)
+ domain = infer_domain(rp, schema)
+ today = date.today().isoformat()
+
+ new_fm = f"---\ntype: {card_type}\ndomain: {domain}\ntier: warm\nrelevance: 0.5\nlast_accessed: {today}\n---\n"
+
+ if not dry_run:
+ md.write_text(new_fm + content)
+ added += 1
+
+ mode = "DRY RUN" if dry_run else "APPLIED"
+ print(f"\n init — {mode}")
+ print(f" Files without frontmatter: {added}")
+ if added and not dry_run:
+ print(f" Added minimal frontmatter to {added} files")
+
+
+def main():
+ args = sys.argv[1:]
+ if not args or args[0] in ('-h', '--help'):
+ print(__doc__)
+ sys.exit(0)
+
+ cmd = args[0]
+ schema_path = find_schema(args)
+ schema = load_schema(schema_path) if schema_path else {}
+
+ if cmd == 'decay':
+ vault_dir = Path(args[1]) if len(args) > 1 else None
+ if not vault_dir:
+ print("Usage: engine.py decay <vault-dir>", file=sys.stderr)
+ sys.exit(1)
+ cmd_decay(vault_dir, schema, dry_run='--dry-run' in args)
+
+ elif cmd == 'touch':
+ filepath = args[1] if len(args) > 1 else None
+ if not filepath:
+ print("Usage: engine.py touch <filepath>", file=sys.stderr)
+ sys.exit(1)
+ cmd_touch(filepath, schema)
+
+ elif cmd == 'creative':
+ n = int(args[1]) if len(args) > 1 else 5
+ vault_dir = Path(args[2]) if len(args) > 2 else None
+ if not vault_dir:
+ print("Usage: engine.py creative <N> <vault-dir>", file=sys.stderr)
+ sys.exit(1)
+ cmd_creative(n, vault_dir, schema)
+
+ elif cmd == 'stats':
+ vault_dir = Path(args[1]) if len(args) > 1 else None
+ if not vault_dir:
+ print("Usage: engine.py stats <vault-dir>", file=sys.stderr)
+ sys.exit(1)
+ cmd_stats(vault_dir, schema)
+
+ elif cmd == 'init':
+ vault_dir = Path(args[1]) if len(args) > 1 else None
+ if not vault_dir:
+ print("Usage: engine.py init <vault-dir>", file=sys.stderr)
+ sys.exit(1)
+ cmd_init(vault_dir, schema, dry_run='--dry-run' in args)
+
+ else:
+ print(f"Unknown command: {cmd}", file=sys.stderr)
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/enrich.py b/vault/.claude/skills/autograph/scripts/enrich.py
new file mode 100644
index 0000000..9d2c9f8
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/enrich.py
@@ -0,0 +1,802 @@
+#!/usr/bin/env python3
+"""
+autograph enrich — tag and link enrichment via OpenRouter API.
+
+Replaces agent swarms with direct API calls. Zero external dependencies (stdlib only).
+
+Usage:
+ enrich.py tags <vault-dir> [--apply] [--budget 50000] [--model MODEL] [--force] [--delay 0.5] [--workers 3]
+ enrich.py swarm-links <vault-dir> [--apply] [--budget 100000] [--model MODEL] [--force] [--delay 0.3] [--workers 5]
+"""
+
+import json
+import sys
+import os
+import time
+import random
+import threading
+from pathlib import Path
+from datetime import datetime
+from urllib.request import Request, urlopen
+from urllib.error import HTTPError, URLError
+from concurrent.futures import ThreadPoolExecutor, as_completed
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from common import (walk_vault, parse_frontmatter, write_frontmatter,
+ rel_path, extract_wikilinks, load_schema, get_ignore_tags)
+from swarm_prepare import estimate_tokens, bin_pack_batches, top_folder
+
+# ─── CONSTANTS ────────────────────────────────────────────
+OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions"
+DEFAULT_MODEL = "google/gemini-3-flash-preview"
+MAX_RETRIES = 3
+BASE_DELAY = 2.0
+FILES_PER_TAGS_CALL = 20
+
+# Thread safety
+_results_lock = threading.Lock()
+_consecutive_auth_errors = 0
+_auth_lock = threading.Lock()
+
+# ─── API LAYER ────────────────────────────────────────────
+
+def get_api_key() -> str:
+ """Get OpenRouter API key from environment."""
+ key = os.environ.get('OPENROUTER_API_KEY', '')
+ if not key:
+ raise RuntimeError("OPENROUTER_API_KEY environment variable not set")
+ return key
+
+
+def call_openrouter(messages: list[dict], response_schema: dict,
+ model: str = DEFAULT_MODEL, schema_name: str = "response") -> dict:
+ """Call OpenRouter API with structured JSON output.
+
+ Retries on 429/5xx with exponential backoff. Extra retry on broken JSON.
+ Returns parsed JSON dict.
+ """
+ global _consecutive_auth_errors
+ api_key = get_api_key()
+
+ body = json.dumps({
+ "model": model,
+ "messages": messages,
+ "response_format": {
+ "type": "json_schema",
+ "json_schema": {
+ "name": schema_name,
+ "strict": True,
+ "schema": response_schema,
+ }
+ }
+ }).encode()
+
+ headers = {
+ "Authorization": f"Bearer {api_key}",
+ "Content-Type": "application/json",
+ }
+
+ last_error = None
+ for attempt in range(MAX_RETRIES):
+ try:
+ req = Request(OPENROUTER_URL, data=body, headers=headers, method="POST")
+ with urlopen(req, timeout=60) as resp:
+ raw = resp.read().decode()
+
+ data = json.loads(raw)
+ content = data["choices"][0]["message"]["content"]
+
+ # Reset auth error counter on success
+ with _auth_lock:
+ _consecutive_auth_errors = 0
+
+ # Try to parse JSON response
+ try:
+ return json.loads(content)
+ except json.JSONDecodeError:
+ # One extra retry for broken JSON
+ if attempt < MAX_RETRIES - 1:
+ time.sleep(BASE_DELAY + random.uniform(0.5, 2.0))
+ continue
+ raise ValueError(f"Invalid JSON in response: {content[:200]}")
+
+ except HTTPError as e:
+ last_error = e
+ if e.code == 401:
+ with _auth_lock:
+ _consecutive_auth_errors += 1
+ if _consecutive_auth_errors >= 3:
+ raise RuntimeError("3 consecutive 401 errors — check OPENROUTER_API_KEY") from e
+ raise
+ if e.code == 429:
+ retry_after = e.headers.get('Retry-After')
+ delay = int(retry_after) if retry_after and retry_after.isdigit() else BASE_DELAY * (2 ** attempt)
+ delay += random.uniform(0.5, 2.0)
+ time.sleep(delay)
+ continue
+ if e.code >= 500:
+ delay = BASE_DELAY * (2 ** attempt) + random.uniform(0.5, 2.0)
+ time.sleep(delay)
+ continue
+ raise
+ except (URLError, TimeoutError) as e:
+ last_error = e
+ if attempt < MAX_RETRIES - 1:
+ delay = BASE_DELAY * (2 ** attempt) + random.uniform(0.5, 2.0)
+ time.sleep(delay)
+ continue
+ raise
+
+ raise RuntimeError(f"All {MAX_RETRIES} retries exhausted: {last_error}")
+
+
+def chunk_list(lst: list, n: int) -> list[list]:
+ """Split list into chunks of size n."""
+ if n <= 0:
+ return [lst] if lst else []
+ return [lst[i:i+n] for i in range(0, len(lst), n)]
+
+
+# ─── TAGS SUBCOMMAND ─────────────────────────────────────
+
+TAGS_SYSTEM_PROMPT = """\
+You are a tag classifier for an Obsidian knowledge vault. Assign 1-5 lowercase hyphenated tags per file.
+Rules:
+- STRONGLY prefer tags from SEED SET below. Only create new if no seed tag fits.
+- New tags: lowercase, hyphenated (machine-learning, not MachineLearning)
+- Content-based only (never tag "note", "markdown", "file")
+- Return fewer tags if file is narrow. Empty array OK if nothing fits from seeds.
+SEED TAGS: {seed_tags}"""
+
+TAGS_RESPONSE_SCHEMA = {
+ "type": "object",
+ "properties": {
+ "results": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "path": {"type": "string"},
+ "tags": {"type": "array", "items": {"type": "string"}}
+ },
+ "required": ["path", "tags"],
+ "additionalProperties": False
+ }
+ }
+ },
+ "required": ["results"],
+ "additionalProperties": False
+}
+
+
+def collect_vault_tags(vault_dir: Path, schema: dict) -> list[str]:
+ """Collect all unique tags from vault, minus ignore_tags."""
+ ignore = get_ignore_tags(schema)
+ tags = set()
+ for md in walk_vault(vault_dir):
+ content = md.read_text(errors='replace')
+ fm, _, _ = parse_frontmatter(content)
+ if fm and isinstance(fm.get('tags'), list):
+ for t in fm['tags']:
+ t_clean = str(t).strip().lower()
+ if t_clean and t_clean not in ignore:
+ tags.add(t_clean)
+ return sorted(tags)
+
+
+def build_tag_entries(vault_dir: Path, schema: dict, force: bool = False) -> list[dict]:
+ """Build entries for files that need tags (empty/missing tags)."""
+ ignore = get_ignore_tags(schema)
+ entries = []
+ for md in walk_vault(vault_dir):
+ content = md.read_text(errors='replace')
+ fm, body, _ = parse_frontmatter(content)
+ rp = rel_path(md, vault_dir)
+
+ # Skip files that already have tags (unless --force)
+ if not force and fm and isinstance(fm.get('tags'), list) and len(fm['tags']) > 0:
+ tags_clean = [t for t in fm['tags'] if str(t).strip().lower() not in ignore]
+ if tags_clean:
+ continue
+
+ tokens = estimate_tokens(md)
+ summary = body[:500].strip() if body else ''
+ ftype = fm.get('type', '') if fm else ''
+ fdomain = fm.get('domain', '') if fm else ''
+
+ entries.append({
+ 'path': rp,
+ 'folder': top_folder(rp),
+ 'tokens': tokens,
+ 'summary': summary,
+ 'type': ftype,
+ 'domain': fdomain,
+ })
+ return entries
+
+
+def _process_tag_batch(batch_id: str, files: list[dict], seed_tags: list[str],
+ model: str, delay: float, output_dir: Path, force: bool) -> dict | None:
+ """Process a single tag batch. Returns result dict or None on error."""
+ result_path = output_dir / f'{batch_id}-results.json'
+
+ # Resume: skip if result exists and not --force
+ if result_path.exists() and not force:
+ return json.loads(result_path.read_text())
+
+ if delay > 0:
+ time.sleep(delay + random.uniform(0, delay * 0.5))
+
+ # Build user message with file summaries
+ file_list = []
+ for f in files:
+ file_list.append(f"### {f['path']}\nType: {f['type']} | Domain: {f['domain']}\n{f['summary'][:300]}")
+
+ user_msg = "Classify these files:\n\n" + "\n\n".join(file_list)
+ system_msg = TAGS_SYSTEM_PROMPT.format(seed_tags=', '.join(seed_tags[:200]))
+
+ try:
+ result = call_openrouter(
+ messages=[
+ {"role": "system", "content": system_msg},
+ {"role": "user", "content": user_msg},
+ ],
+ response_schema=TAGS_RESPONSE_SCHEMA,
+ model=model,
+ schema_name="tag_results",
+ )
+ output = {
+ 'batch_id': batch_id,
+ 'results': result.get('results', []),
+ 'validation': {'file_count': len(files), 'result_count': len(result.get('results', []))},
+ }
+ result_path.write_text(json.dumps(output, ensure_ascii=False, indent=2) + '\n')
+ return output
+ except Exception as e:
+ error_path = output_dir.parent / 'errors' / f'{batch_id}-error.json'
+ error_path.parent.mkdir(parents=True, exist_ok=True)
+ error_path.write_text(json.dumps({
+ 'batch_id': batch_id, 'error': str(e), 'timestamp': datetime.now().isoformat(),
+ }, indent=2) + '\n')
+ print(f" ERROR {batch_id}: {e}", file=sys.stderr)
+ return None
+
+
+def apply_tags(vault_dir: Path, results_dir: Path):
+ """Apply tag results to vault files."""
+ applied = 0
+ for rp_file in sorted(results_dir.glob('batch-*-results.json')):
+ data = json.loads(rp_file.read_text())
+ for item in data.get('results', []):
+ path = vault_dir / item['path']
+ if not path.exists():
+ continue
+ tags = item.get('tags', [])
+ if not tags:
+ continue
+
+ content = path.read_text(errors='replace')
+ fm, body, lines = parse_frontmatter(content)
+ if fm is None:
+ # Create frontmatter
+ fm = {'tags': tags}
+ new_fm = write_frontmatter(fm, [])
+ path.write_text(f"---\n{new_fm}\n---\n{body}")
+ else:
+ fm['tags'] = tags
+ new_fm = write_frontmatter(fm, lines)
+ path.write_text(f"---\n{new_fm}\n---\n{body}")
+ applied += 1
+ return applied
+
+
+def cmd_tags(vault_dir: Path, apply: bool = False, budget: int = 50000,
+ model: str = DEFAULT_MODEL, force: bool = False,
+ delay: float = 0.5, workers: int = 3):
+ """Tags enrichment subcommand."""
+ vault_dir = vault_dir.resolve()
+ schema = load_schema()
+
+ # 1. Collect seed tags
+ seed_tags = collect_vault_tags(vault_dir, schema)
+ print(f"Seed tags: {len(seed_tags)}")
+
+ # 2. Build entries for files needing tags
+ entries = build_tag_entries(vault_dir, schema, force)
+ print(f"Files needing tags: {len(entries)}")
+ if not entries:
+ print("Nothing to do.")
+ return
+
+ # 3. Bin-pack and chunk
+ batches = bin_pack_batches(entries, budget)
+ api_tasks = []
+ batch_num = 0
+ for batch in batches:
+ for chunk in chunk_list(batch, FILES_PER_TAGS_CALL):
+ batch_num += 1
+ api_tasks.append((f'batch-{batch_num:03d}', chunk))
+ print(f"API batches: {len(api_tasks)}")
+
+ # 4. Output dir
+ output_dir = vault_dir / '.graph' / 'enrich' / 'tags'
+ output_dir.mkdir(parents=True, exist_ok=True)
+
+ # 5. Process with ThreadPoolExecutor
+ results = []
+ errors = 0
+ with ThreadPoolExecutor(max_workers=workers) as pool:
+ futures = {
+ pool.submit(_process_tag_batch, bid, files, seed_tags, model, delay, output_dir, force): bid
+ for bid, files in api_tasks
+ }
+ for future in as_completed(futures):
+ bid = futures[future]
+ try:
+ result = future.result()
+ if result:
+ with _results_lock:
+ results.append(result)
+ print(f" OK {bid}: {result['validation']['result_count']} results")
+ else:
+ errors += 1
+ except Exception as e:
+ errors += 1
+ print(f" FAIL {bid}: {e}", file=sys.stderr)
+
+ # 6. Write run metadata
+ total_tagged = sum(r['validation']['result_count'] for r in results)
+ meta = {
+ 'mode': 'tags',
+ 'total_files': len(entries),
+ 'total_batches': len(api_tasks),
+ 'total_tagged': total_tagged,
+ 'errors': errors,
+ 'model': model,
+ 'timestamp': datetime.now().isoformat(),
+ }
+ (output_dir / 'run-meta.json').write_text(json.dumps(meta, indent=2) + '\n')
+ print(f"\nDone: {total_tagged} files tagged, {errors} errors")
+
+ # 7. Apply if requested
+ if apply:
+ applied = apply_tags(vault_dir, output_dir)
+ print(f"Applied tags to {applied} files")
+
+
+# ─── CATALOG (for swarm-links) ────────────────────────────
+
+FILES_PER_SWARM_CALL = 8
+
+
+def scan_vault_for_links(vault_dir: Path, force: bool = False) -> tuple[set, dict, dict, list]:
+ """Single walk: returns (stems_set, stem_to_path, catalog, link_entries).
+
+ Replaces three separate walk_vault() calls with one pass.
+ """
+ stems_set = set()
+ stem_to_path = {}
+ catalog = {} # domain → list of catalog entries
+ link_entries = [] # files needing links (<2 outgoing)
+
+ for md in walk_vault(vault_dir):
+ content = md.read_text(errors='replace')
+ fm, body, _ = parse_frontmatter(content)
+ rp = rel_path(md, vault_dir)
+ stem = md.stem
+
+ # --- stems ---
+ stems_set.add(stem)
+ stem_to_path[stem] = rp
+
+ # --- catalog ---
+ domain = (fm.get('domain', '') if fm else '') or 'general'
+ ftype = fm.get('type', '') if fm else ''
+ tags = fm.get('tags', []) if fm else []
+ first_line = body.strip().split('\n')[0][:120] if body else ''
+
+ catalog.setdefault(domain, []).append({
+ 'stem': stem,
+ 'type': ftype,
+ 'tags': tags[:5] if isinstance(tags, list) else [],
+ 'desc': first_line,
+ })
+
+ # --- link entries ---
+ existing_links = extract_wikilinks(body)
+ if not force and len(existing_links) >= 2:
+ continue
+
+ tokens = estimate_tokens(md)
+ summary = body[:500].strip() if body else ''
+ ftags = tags
+
+ link_entries.append({
+ 'path': rp,
+ 'folder': top_folder(rp),
+ 'tokens': tokens,
+ 'summary': summary,
+ 'type': ftype,
+ 'domain': domain,
+ 'tags': ftags if isinstance(ftags, list) else [],
+ 'existing_links': [t for t, _ in existing_links],
+ })
+
+ return stems_set, stem_to_path, catalog, link_entries
+
+
+def format_catalog(entries: list[dict], max_entries: int = 800) -> str:
+ """Format catalog entries as text table for LLM prompt.
+
+ Each line: stem | tags: t1, t2 | Description...
+ Truncates if > max_entries.
+ """
+ if not entries:
+ return "(empty catalog)"
+ lines = []
+ for e in entries[:max_entries]:
+ tags_str = ', '.join(str(t) for t in e.get('tags', []))
+ desc = e.get('desc', '')[:80]
+ line = f"{e['stem']} | tags: {tags_str} | {desc}"
+ lines.append(line)
+ if len(entries) > max_entries:
+ lines.append(f"(truncated, {len(entries) - max_entries} more)")
+ return '\n'.join(lines)
+
+
+def apply_links(vault_dir: Path, results_dir: Path, stem_to_path: dict[str, str]):
+ """Apply link results to vault files — append ## Related section."""
+ applied = 0
+ for rp_file in sorted(results_dir.glob('batch-*-results.json')):
+ data = json.loads(rp_file.read_text())
+ for item in data.get('results', []):
+ path = vault_dir / item['path']
+ if not path.exists():
+ continue
+ matched = item.get('matched_links', [])
+ if not matched:
+ continue
+
+ content = path.read_text(errors='replace')
+
+ # Build wikilink targets for new stems
+ new_targets = {}
+ for stem in matched:
+ rp_target = stem_to_path.get(stem, stem)
+ wikilink_target = rp_target[:-3] if rp_target.endswith('.md') else rp_target
+ new_targets[stem] = wikilink_target
+
+ if '## Related' in content:
+ # Parse existing links and deduplicate
+ related_idx = content.index('## Related')
+ related_section = content[related_idx:]
+ existing_links = extract_wikilinks(related_section)
+ existing_stems = {Path(link).stem for link, _ in existing_links}
+ new_stems = [s for s in matched if s not in existing_stems]
+ if not new_stems:
+ continue
+ # Find end of ## Related section (next ## or EOF)
+ rest = content[related_idx + len('## Related'):]
+ next_heading = rest.find('\n## ')
+ if next_heading >= 0:
+ insert_pos = related_idx + len('## Related') + next_heading
+ new_lines = '\n'.join(f"- [[{new_targets[s]}]]" for s in new_stems)
+ content = content[:insert_pos] + '\n' + new_lines + content[insert_pos:]
+ else:
+ new_lines = '\n'.join(f"- [[{new_targets[s]}]]" for s in new_stems)
+ content = content.rstrip() + '\n' + new_lines + '\n'
+ else:
+ # Create new ## Related section
+ link_lines = [f"- [[{new_targets[s]}]]" for s in matched]
+ section = "\n\n## Related\n" + '\n'.join(link_lines) + '\n'
+ content = content.rstrip() + section
+
+ path.write_text(content)
+ applied += 1
+ return applied
+
+
+# ─── SWARM-LINKS SUBCOMMAND ──────────────────────────────
+
+SWARM_LINKS_SYSTEM_PROMPT = """\
+You are a link suggestion engine for an Obsidian vault.
+For each file, pick 3-8 related notes FROM THE CATALOG BELOW.
+CRITICAL RULES:
+- ONLY return stems that appear EXACTLY in the catalog. Do NOT invent new names.
+- Pick notes that are semantically related by topic, domain, tags, or content.
+- Do NOT suggest the file's own stem.
+- Do NOT suggest stems already in its existing links.
+
+CATALOG ({domain}):
+{catalog_text}"""
+
+SWARM_LINKS_RESPONSE_SCHEMA = {
+ "type": "object",
+ "properties": {
+ "results": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "path": {"type": "string"},
+ "links": {"type": "array", "items": {"type": "string"}}
+ },
+ "required": ["path", "links"],
+ "additionalProperties": False
+ }
+ }
+ },
+ "required": ["results"],
+ "additionalProperties": False
+}
+
+
+def _process_swarm_link_batch(batch_id: str, files: list[dict],
+ catalog_text: str, domain: str,
+ all_stems: set[str], model: str,
+ delay: float, output_dir: Path,
+ force: bool) -> dict | None:
+ """Process a single swarm-link batch with strict set membership validation."""
+ result_path = output_dir / f'{batch_id}-results.json'
+
+ if result_path.exists() and not force:
+ return json.loads(result_path.read_text())
+
+ if delay > 0:
+ time.sleep(delay + random.uniform(0, delay * 0.5))
+
+ # Build user message with file summaries
+ file_list = []
+ for f in files:
+ tags_str = ', '.join(f.get('tags', [])[:10])
+ links_str = ', '.join(f.get('existing_links', [])[:5])
+ file_list.append(
+ f"### {f['path']}\nType: {f['type']} | Domain: {f['domain']} | Tags: {tags_str}\n"
+ f"Existing links: {links_str}\n{f['summary'][:300]}"
+ )
+
+ user_msg = "Suggest links for these files:\n\n" + "\n\n".join(file_list)
+ system_msg = SWARM_LINKS_SYSTEM_PROMPT.format(
+ domain=domain, catalog_text=catalog_text
+ )
+
+ try:
+ result = call_openrouter(
+ messages=[
+ {"role": "system", "content": system_msg},
+ {"role": "user", "content": user_msg},
+ ],
+ response_schema=SWARM_LINKS_RESPONSE_SCHEMA,
+ model=model,
+ schema_name="swarm_link_results",
+ )
+
+ # Post-process: strict set membership (no fuzzy)
+ enriched = []
+ for item in result.get('results', []):
+ raw_links = item.get('links', [])
+ file_path = item.get('path', '')
+ file_stem = Path(file_path).stem
+
+ # Get existing links for this file
+ existing = set()
+ for f in files:
+ if f['path'] == file_path:
+ existing = set(f.get('existing_links', []))
+ break
+
+ # Match: try raw value first, then extract stem from path
+ valid = []
+ seen = set()
+ for s in raw_links:
+ # Try as-is first (exact stem)
+ matched_stem = s if s in all_stems else None
+ # If not found, try extracting stem from path-like value
+ if not matched_stem:
+ extracted = Path(s).stem
+ if extracted in all_stems:
+ matched_stem = extracted
+ # Also strip .md suffix if present
+ if not matched_stem and s.endswith('.md'):
+ bare = Path(s[:-3]).stem
+ if bare in all_stems:
+ matched_stem = bare
+ if (matched_stem
+ and matched_stem != file_stem
+ and matched_stem not in existing
+ and matched_stem not in seen):
+ valid.append(matched_stem)
+ seen.add(matched_stem)
+
+ enriched.append({
+ 'path': file_path,
+ 'links_raw': raw_links,
+ 'matched_links': valid,
+ })
+
+ output = {
+ 'batch_id': batch_id,
+ 'results': enriched,
+ 'validation': {
+ 'file_count': len(files),
+ 'result_count': len(enriched),
+ 'total_raw': sum(len(r.get('links_raw', [])) for r in enriched),
+ 'total_matched': sum(len(r.get('matched_links', [])) for r in enriched),
+ },
+ }
+ result_path.write_text(json.dumps(output, ensure_ascii=False, indent=2) + '\n')
+ return output
+ except Exception as e:
+ error_path = output_dir.parent / 'errors' / f'{batch_id}-error.json'
+ error_path.parent.mkdir(parents=True, exist_ok=True)
+ error_path.write_text(json.dumps({
+ 'batch_id': batch_id, 'error': str(e), 'timestamp': datetime.now().isoformat(),
+ }, indent=2) + '\n')
+ print(f" ERROR {batch_id}: {e}", file=sys.stderr)
+ return None
+
+
+def cmd_swarm_links(vault_dir: Path, apply: bool = False, budget: int = 100000,
+ model: str = 'google/gemini-2.0-flash-001', force: bool = False,
+ delay: float = 0.3, workers: int = 5):
+ """Swarm-links enrichment: catalog-oriented link suggestions."""
+ vault_dir = vault_dir.resolve()
+
+ # Single vault walk: stems + catalog + link entries
+ all_stems, stem_to_path, catalog, entries = scan_vault_for_links(vault_dir, force)
+ print(f"Vault stems: {len(all_stems)}")
+ print(f"Catalog domains: {len(catalog)} ({', '.join(f'{d}:{len(v)}' for d, v in catalog.items())})")
+ print(f"Files needing links: {len(entries)}")
+ if not entries:
+ print("Nothing to do.")
+ return
+
+ # 4. Output dir
+ output_dir = vault_dir / '.graph' / 'enrich' / 'swarm-links'
+ output_dir.mkdir(parents=True, exist_ok=True)
+
+ # 5. Group entries by domain, prepare batches
+ domain_entries = {}
+ for e in entries:
+ d = e.get('domain', '') or 'general'
+ domain_entries.setdefault(d, []).append(e)
+
+ api_tasks = []
+ batch_num = 0
+ for domain, d_entries in domain_entries.items():
+ # Build catalog text for this domain + cross-domain top stems
+ domain_catalog = catalog.get(domain, [])
+ # Add top-30 stems from other domains for cross-domain links
+ cross_entries = []
+ for other_d, other_entries in catalog.items():
+ if other_d != domain:
+ cross_entries.extend(other_entries[:30])
+ combined = domain_catalog + cross_entries
+ cat_text = format_catalog(combined)
+
+ for chunk in chunk_list(d_entries, FILES_PER_SWARM_CALL):
+ batch_num += 1
+ api_tasks.append((f'batch-{batch_num:03d}', chunk, cat_text, domain))
+
+ print(f"API batches: {len(api_tasks)}")
+
+ # 6. Process with ThreadPoolExecutor
+ results = []
+ errors = 0
+ with ThreadPoolExecutor(max_workers=workers) as pool:
+ futures = {
+ pool.submit(_process_swarm_link_batch, bid, files, cat_text, domain,
+ all_stems, model, delay, output_dir, force): bid
+ for bid, files, cat_text, domain in api_tasks
+ }
+ for future in as_completed(futures):
+ bid = futures[future]
+ try:
+ result = future.result()
+ if result:
+ with _results_lock:
+ results.append(result)
+ v = result.get('validation', {})
+ print(f" OK {bid}: {v.get('total_matched', 0)}/{v.get('total_raw', 0)} links matched")
+ else:
+ errors += 1
+ except Exception as e:
+ errors += 1
+ print(f" FAIL {bid}: {e}", file=sys.stderr)
+
+ # 7. Run metadata
+ total_raw = sum(r.get('validation', {}).get('total_raw', 0) for r in results)
+ total_matched = sum(r.get('validation', {}).get('total_matched', 0) for r in results)
+ meta = {
+ 'mode': 'swarm-links',
+ 'total_files': len(entries),
+ 'total_batches': len(api_tasks),
+ 'total_links_raw': total_raw,
+ 'total_links_matched': total_matched,
+ 'match_rate': round(total_matched / total_raw * 100, 1) if total_raw > 0 else 0,
+ 'errors': errors,
+ 'model': model,
+ 'validation': 'strict_set_membership',
+ 'timestamp': datetime.now().isoformat(),
+ }
+ (output_dir / 'run-meta.json').write_text(json.dumps(meta, indent=2) + '\n')
+ print(f"\nDone: {total_matched}/{total_raw} links matched ({meta['match_rate']}%), {errors} errors")
+
+ # 8. Apply (reuse apply_links)
+ if apply:
+ applied = apply_links(vault_dir, output_dir, stem_to_path)
+ print(f"Applied links to {applied} files")
+
+
+# ─── CLI ──────────────────────────────────────────────────
+
+def parse_args(args: list[str]) -> dict:
+ """Parse CLI arguments."""
+ if len(args) < 2:
+ print("Usage: enrich.py {tags|swarm-links} <vault-dir> [options]", file=sys.stderr)
+ sys.exit(1)
+
+ cmd = args[0]
+ vault_dir = Path(args[1])
+
+ # Defaults vary by subcommand
+ budgets = {'tags': 50000, 'swarm-links': 100000}
+ delays = {'swarm-links': 0.3}
+ worker_counts = {'swarm-links': 5}
+ models = {'swarm-links': 'google/gemini-2.0-flash-001'}
+
+ opts = {
+ 'cmd': cmd,
+ 'vault_dir': vault_dir,
+ 'apply': False,
+ 'budget': budgets.get(cmd, 70000),
+ 'model': models.get(cmd, DEFAULT_MODEL),
+ 'force': False,
+ 'delay': delays.get(cmd, 0.5),
+ 'workers': worker_counts.get(cmd, 3),
+ }
+
+ i = 2
+ while i < len(args):
+ a = args[i]
+ if a == '--apply':
+ opts['apply'] = True
+ elif a == '--force':
+ opts['force'] = True
+ elif a == '--budget' and i + 1 < len(args):
+ i += 1
+ opts['budget'] = int(args[i])
+ elif a == '--model' and i + 1 < len(args):
+ i += 1
+ opts['model'] = args[i]
+ elif a == '--delay' and i + 1 < len(args):
+ i += 1
+ opts['delay'] = float(args[i])
+ elif a == '--workers' and i + 1 < len(args):
+ i += 1
+ opts['workers'] = int(args[i])
+ i += 1
+
+ return opts
+
+
+def main():
+ opts = parse_args(sys.argv[1:])
+
+ if not opts['vault_dir'].is_dir():
+ print(f"Not a directory: {opts['vault_dir']}", file=sys.stderr)
+ sys.exit(1)
+
+ if opts['cmd'] == 'tags':
+ cmd_tags(opts['vault_dir'], opts['apply'], opts['budget'],
+ opts['model'], opts['force'], opts['delay'], opts['workers'])
+ elif opts['cmd'] == 'swarm-links':
+ cmd_swarm_links(opts['vault_dir'], opts['apply'], opts['budget'],
+ opts['model'], opts['force'], opts['delay'], opts['workers'])
+ else:
+ print(f"Unknown command: {opts['cmd']}. Use 'tags' or 'swarm-links'.", file=sys.stderr)
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/generate_schema.py b/vault/.claude/skills/autograph/scripts/generate_schema.py
new file mode 100644
index 0000000..4ce526d
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/generate_schema.py
@@ -0,0 +1,185 @@
+#!/usr/bin/env python3
+"""
+autograph generate_schema — turn discovery JSON into a draft schema.json.
+
+Usage: python3 generate_schema.py <discovery.json> [output.json]
+ cat discovery.json | python3 generate_schema.py -
+"""
+
+import json, sys
+from pathlib import Path
+
+_CANONICAL_ALIASES = {
+ 'blog-post': 'note', 'blog_post': 'note', 'blogpost': 'note',
+ 'article': 'note', 'post': 'note', 'page': 'note', 'idea': 'note',
+ 'person': 'contact', 'people': 'contact',
+ 'task': 'project', 'initiative': 'project',
+ 'ref': 'reference', 'resource': 'reference',
+}
+_STATUS_ORDER = {
+ 'active': 0, 'prospect': 1, 'negotiation': 2, 'won': 3, 'standby': 4,
+ 'pending': 5, 'lost': 6, 'inactive': 7, 'done': 8, 'draft': 9,
+ 'paused': 10, 'cancelled': 11, 'archived': 12,
+}
+_SYSTEM_FIELDS = {
+ 'type', 'status', 'tags', 'description', 'title', 'date',
+ 'last_accessed', 'tier', 'relevance', 'domain', 'priority',
+ 'created', 'updated', 'aliases',
+}
+_DOMAIN_KW = [
+ (['project', 'work', 'client'], 'work'),
+ (['note', 'zettel', 'knowledge', 'wiki'], 'knowledge'),
+ (['personal', 'journal', 'diary', 'daily'], 'personal'),
+ (['ai', 'prompt', 'tool'], 'tech'),
+ (['clip', 'bookmark', 'save', 'read'], 'reference'),
+ (['import', 'onenote', 'apple', 'notion'], 'import'),
+]
+
+def _edit_dist(a: str, b: str) -> int:
+ if len(a) < len(b): return _edit_dist(b, a)
+ prev = list(range(len(b) + 1))
+ for i, ca in enumerate(a):
+ curr = [i + 1]
+ for j, cb in enumerate(b):
+ curr.append(min(prev[j] + (ca != cb), prev[j + 1] + 1, curr[j] + 1))
+ prev = curr
+ return prev[-1]
+
+def _similar(a: str, b: str) -> bool:
+ mx = max(len(a), len(b))
+ return a != b and mx > 0 and _edit_dist(a, b) / mx <= 0.3
+
+def _get_vals(enums: dict, field: str) -> dict:
+ return enums.get(field, {}).get('values', {})
+
+def _find_aliases(vals: dict) -> dict:
+ """Map known aliases and typos to canonical forms (high-frequency first)."""
+ by_count = sorted(vals, key=lambda v: -vals[v])
+ canonical, aliases = set(), {}
+ for v in by_count:
+ if v in _CANONICAL_ALIASES:
+ target = _CANONICAL_ALIASES[v]
+ if target in vals or target in canonical:
+ aliases[v] = target; continue
+ matched = False
+ for c in canonical:
+ if _similar(v, c):
+ aliases[v] = c; matched = True; break
+ if not matched:
+ canonical.add(v)
+ return aliases
+
+def build_node_types(enums: dict, exclude: set = None) -> dict:
+ vals = _get_vals(enums, 'type')
+ if not vals:
+ return {'note': {'description': 'General note', 'required': ['description', 'tags'],
+ 'status': ['active', 'draft', 'archived']}}
+ exclude = exclude or set()
+ statuses = sorted(_get_vals(enums, 'status').keys(),
+ key=lambda s: _STATUS_ORDER.get(s, 99)) or ['active', 'draft', 'archived']
+ return {v: {'description': v.replace('-', ' ').replace('_', ' ').title(),
+ 'required': ['description', 'tags'], 'status': list(statuses)}
+ for v in sorted(vals, key=lambda x: -vals[x]) if v not in exclude}
+
+def build_domain_inference(folders: dict) -> dict:
+ result = {}
+ for folder in folders:
+ if folder == 'root': continue
+ key, low = folder.rstrip('/') + '/', folder.lower()
+ matched = False
+ for keywords, domain in _DOMAIN_KW:
+ if any(w in low for w in keywords):
+ result[key] = domain; matched = True; break
+ if not matched:
+ result[key] = low.replace(' ', '_')
+ return result
+
+def build_field_fixes(enums: dict) -> dict:
+ fixes = {}
+ for field, info in enums.items():
+ if field == 'type': continue
+ vals = info.get('values', {})
+ popular = {v for v, c in vals.items() if c >= 10}
+ rare = [v for v, c in vals.items() if c < 5]
+ if not popular or not rare: continue
+ fm = {}
+ for rv in rare:
+ for pv in popular:
+ if _similar(rv, pv): fm[rv] = pv; break
+ if fm: fixes[field] = fm
+ return fixes
+
+def build_status_order(enums: dict) -> dict:
+ vals, order, idx = list(_get_vals(enums, 'status').keys()), {}, 0
+ for s in sorted(_STATUS_ORDER, key=_STATUS_ORDER.get):
+ if s in vals: order[s] = idx; idx += 1
+ for s in vals:
+ if s not in order: order[s] = idx; idx += 1
+ order['_comment'] = 'sort order for MOC generation (lower = higher priority)'
+ return order
+
+def build_path_type_hints(folders: dict, node_types: dict) -> dict:
+ hints = {'_comment': 'folder substring -> type name (used when card has no type field)'}
+ valid = set(node_types.keys())
+ for folder in folders:
+ low = folder.lower().rstrip('/')
+ for t in valid:
+ if t in low: hints[low + '/'] = t; break
+ for kw, tn in [('lead', 'lead'), ('contact', 'contact'), ('people', 'contact')]:
+ if kw in low and tn in valid: hints[low + '/'] = tn
+ return {k: v for k, v in hints.items() if v is not None}
+
+def build_schema(discovery: dict) -> dict:
+ enums = discovery.get('enums', {})
+ folders = discovery.get('folder_structure', {})
+ type_aliases = _find_aliases(_get_vals(enums, 'type'))
+ node_types = build_node_types(enums, exclude=set(type_aliases.keys()))
+ richness = [f for f in enums if f.lower() not in _SYSTEM_FIELDS]
+ if not richness:
+ richness = ['telegram', 'email', 'company', 'role', 'source',
+ 'deal_status', 'responsible']
+ return {
+ '_comment': 'Draft schema generated from discover.py output — review and adjust',
+ 'node_types': node_types,
+ 'type_aliases': type_aliases,
+ 'field_fixes': build_field_fixes(enums),
+ 'region_fixes': {},
+ 'domain_inference': build_domain_inference(folders),
+ 'path_type_hints': build_path_type_hints(folders, node_types),
+ 'status_order': build_status_order(enums),
+ 'status_defaults': {
+ '_comment': 'default status for new cards by type (when status is missing)',
+ 'default': 'active',
+ },
+ 'richness_fields': {
+ '_comment': 'frontmatter fields that indicate content richness (used by dedup)',
+ 'bonus_fields': richness,
+ },
+ 'entity_extraction': {
+ '_comment': 'settings for daily.py entity extraction',
+ 'noise_words': ['TODO', 'FIX', 'NEW', 'UPDATED', 'Phase', 'Score',
+ 'Vault', 'Health', 'Pricing'],
+ },
+ 'decay': {'rate': 0.015, 'floor': 0.1,
+ 'tiers': {'active': 7, 'warm': 21, 'cold': 60}},
+ 'ignore_tags': [],
+ }
+
+def main():
+ args = sys.argv[1:]
+ if not args:
+ print("Usage: generate_schema.py <discovery.json> [output.json]", file=sys.stderr)
+ sys.exit(1)
+ src = args[0]
+ data = json.load(sys.stdin) if src == '-' else json.loads(Path(src).read_text())
+ schema = build_schema(data)
+ output = json.dumps(schema, ensure_ascii=False, indent=2) + '\n'
+ if len(args) >= 2:
+ out_path = Path(args[1])
+ out_path.write_text(output)
+ print(f"Schema written to {out_path}", file=sys.stderr)
+ else:
+ sys.stdout.write(output)
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/graph.py b/vault/.claude/skills/autograph/scripts/graph.py
new file mode 100755
index 0000000..948e419
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/graph.py
@@ -0,0 +1,412 @@
+#!/usr/bin/env python3
+"""
+autograph graph — vault graph analysis, link repair, backlinks, orphans.
+
+Commands:
+ graph.py health <vault-dir> [schema.json] — health score + report
+ graph.py fix <vault-dir> [schema.json] [--apply] — fix broken links
+ graph.py backlinks <vault-dir> <target> — incoming links
+ graph.py orphans <vault-dir> — files with no incoming links
+
+All domain/type logic from schema.json. No hardcoded values.
+"""
+
+import json
+import re
+import sys
+from pathlib import Path
+from datetime import datetime
+from collections import defaultdict
+
+from common import (
+ load_schema, parse_frontmatter, walk_vault, rel_path,
+ extract_wikilinks, infer_domain, get_domain_map, IGNORE_DIRS,
+ build_link_index, normalize_link_target, resolve_link_target, is_hub_path
+)
+
+EMBED_EXTS = {'.jpg', '.jpeg', '.png', '.gif', '.svg', '.pdf', '.mp3', '.mp4', '.webp'}
+
+
+def build_graph(vault_dir: Path, schema: dict) -> dict:
+ """Scan vault, build full graph structure."""
+ vault_dir = Path(vault_dir)
+ files = walk_vault(vault_dir)
+
+ link_index = build_link_index(vault_dir, files)
+
+ nodes = {}
+ all_links = [] # (source, raw_target, resolved_target)
+ broken_links = [] # (source, raw_target)
+
+ for md in files:
+ rp = rel_path(md, vault_dir)
+ rp_noext = rp.replace('.md', '')
+ try:
+ content = md.read_text(errors='replace')
+ except Exception:
+ continue
+
+ fm, body, _ = parse_frontmatter(content)
+ if fm is None:
+ fm = {}
+
+ domain = infer_domain(rp, schema)
+ has_desc = bool(fm.get('description', ''))
+ card_type = fm.get('type', 'unknown')
+
+ outgoing = []
+ links = extract_wikilinks(body if body else content)
+ for target, display in links:
+ # Skip embeds
+ if any(target.lower().endswith(ext) for ext in EMBED_EXTS):
+ continue
+ target_clean = normalize_link_target(target)
+ resolved, _ = resolve_link_target(target_clean, link_index)
+ if resolved:
+ outgoing.append(resolved)
+ all_links.append((rp_noext, target_clean, resolved))
+ else:
+ broken_links.append((rp_noext, target_clean))
+
+ nodes[rp_noext] = {
+ 'domain': domain,
+ 'type': card_type,
+ 'has_description': has_desc,
+ 'outgoing': outgoing,
+ 'incoming': [], # filled below
+ 'link_count': len(outgoing),
+ }
+
+ # Build incoming links
+ for src, raw, resolved in all_links:
+ if resolved in nodes:
+ nodes[resolved]['incoming'].append(src)
+
+ # Compute stats
+ total = len(nodes)
+ total_links = len(all_links)
+ avg_links = total_links / max(total, 1)
+
+ orphans = [p for p, n in nodes.items() if not n['incoming'] and not is_hub_path(p)]
+ dead_ends = [p for p, n in nodes.items() if not n['outgoing'] and n['incoming']]
+ desc_count = sum(1 for n in nodes.values() if n['has_description'])
+ desc_ratio = desc_count / max(total, 1)
+
+ orphan_ratio = len(orphans) / max(total, 1)
+ broken_ratio = len(broken_links) / max(total, 1)
+
+ health = 100.0
+ health -= orphan_ratio * 30
+ health -= broken_ratio * 30
+ health -= max(0, (3 - avg_links) * 15)
+ health -= (1 - desc_ratio) * 10
+ health = max(0, round(health, 1))
+
+ # Domain stats + non-standard domain detection
+ valid_domains = set(get_domain_map(schema).values()) if schema else set()
+ domain_stats = defaultdict(lambda: {'files': 0, 'links': 0, 'orphans': 0})
+ nonstandard_domains = defaultdict(list) # domain -> [file_paths]
+ for path, node in nodes.items():
+ d = node['domain']
+ domain_stats[d]['files'] += 1
+ domain_stats[d]['links'] += node['link_count']
+ if valid_domains and d not in valid_domains:
+ nonstandard_domains[d].append(path)
+ for o in orphans:
+ if o in nodes:
+ domain_stats[nodes[o]['domain']]['orphans'] += 1
+
+ nonstandard_count = sum(len(v) for v in nonstandard_domains.values())
+ nonstandard_ratio = nonstandard_count / max(total, 1)
+ health -= nonstandard_ratio * 5 # small penalty for domain inconsistency
+ health = max(0, round(health, 1))
+
+ return {
+ 'generated': datetime.now().isoformat(),
+ 'stats': {
+ 'total_files': total,
+ 'total_links': total_links,
+ 'avg_links': round(avg_links, 2),
+ 'orphans': len(orphans),
+ 'dead_ends': len(dead_ends),
+ 'broken_links': len(broken_links),
+ 'desc_coverage': round(desc_ratio * 100, 1),
+ 'nonstandard_domains': nonstandard_count,
+ 'health_score': health,
+ },
+ 'domains': dict(domain_stats),
+ 'nonstandard_domain_list': {k: v[:10] for k, v in nonstandard_domains.items()},
+ 'orphan_list': sorted(orphans),
+ 'dead_end_list': sorted(dead_ends),
+ 'broken_link_list': [{'source': s, 'target': t} for s, t in broken_links],
+ 'nodes': {k: {'domain': v['domain'], 'type': v['type'], 'has_description': v['has_description'],
+ 'outgoing': v['outgoing'], 'incoming': v['incoming']}
+ for k, v in nodes.items()},
+ }
+
+
+def resolve_link(target: str, path_index: dict) -> str | None:
+ """Backward-compatible wrapper around deterministic resolver."""
+ if 'exact' in path_index and 'unique_stem' in path_index:
+ return resolve_link_target(target, path_index)[0]
+
+ target = normalize_link_target(target)
+ if not target:
+ return None
+ if target in path_index:
+ return path_index[target]
+ stem = target.split('/')[-1]
+ if stem in path_index:
+ return path_index[stem]
+ return None
+
+
+def generate_report(stats: dict, domains: dict) -> str:
+ """Generate markdown health report."""
+ s = stats
+ lines = [
+ f"# Vault Health Report",
+ f"",
+ f"Generated: {datetime.now().strftime('%Y-%m-%d %H:%M')}",
+ f"",
+ f"| Metric | Value |",
+ f"|--------|-------|",
+ f"| Health Score | **{s['health_score']}/100** |",
+ f"| Total files | {s['total_files']} |",
+ f"| Total links | {s['total_links']} |",
+ f"| Avg links/file | {s['avg_links']} |",
+ f"| Orphans | {s['orphans']} |",
+ f"| Dead-ends | {s['dead_ends']} |",
+ f"| Broken links | {s['broken_links']} |",
+ f"| Desc coverage | {s['desc_coverage']}% |",
+ f"",
+ f"## Domains",
+ f"",
+ ]
+ for domain, ds in sorted(domains.items()):
+ lines.append(f"- **{domain}**: {ds['files']} files, {ds['links']} links, {ds['orphans']} orphans")
+ return '\n'.join(lines)
+
+
+def update_history(vault_dir: Path, stats: dict):
+ """Append to health-history.json (max 90 entries)."""
+ hist_path = vault_dir / '.graph' / 'health-history.json'
+ history = []
+ if hist_path.exists():
+ try:
+ history = json.loads(hist_path.read_text())
+ except Exception:
+ history = []
+ history.append({
+ 'date': datetime.now().strftime('%Y-%m-%d'),
+ **stats
+ })
+ history = history[-90:] # keep last 90
+ hist_path.write_text(json.dumps(history, indent=2))
+
+
+# ─── FIX BROKEN LINKS ─────────────────────────────────────
+def fix_broken_links(vault_dir: Path, graph: dict, apply: bool = False) -> list:
+ """Suggest and optionally apply fixes for broken links."""
+ link_index = build_link_index(vault_dir)
+
+ fixes = []
+ for bl in graph['broken_link_list']:
+ src = bl['source']
+ target = bl['target']
+ resolved, strategy = resolve_link_target(target, link_index)
+ if resolved and strategy in ('unique_suffix', 'unique_stem'):
+ fixes.append({
+ 'source': src,
+ 'old': target,
+ 'new': resolved,
+ 'strategy': strategy
+ })
+
+ if apply:
+ applied = 0
+ for fix in fixes:
+ src_path = vault_dir / (fix['source'] + '.md')
+ if not src_path.exists():
+ continue
+ content = src_path.read_text(errors='replace')
+ pattern = re.compile(
+ r'\[\[' + re.escape(normalize_link_target(fix['old'])) + r'(?P<anchor>#[^\]|]+)?(?P<alias>\|[^\]]+)?\]\]'
+ )
+ if pattern.search(content):
+ content = pattern.sub(
+ lambda m: f"[[{fix['new']}{m.group('anchor') or ''}{m.group('alias') or ''}]]",
+ content
+ )
+ src_path.write_text(content)
+ applied += 1
+ return fixes, applied
+ return fixes, 0
+
+
+# ─── BACKLINKS ─────────────────────────────────────────────
+def find_text_mentions(vault_dir: Path, target: str, wikilink_sources: set) -> list:
+ """Find plain-text mentions of target stem (not wikilinks) in vault files."""
+ stem = target.replace('.md', '').split('/')[-1]
+ # Build search variants: stem as-is and with spaces instead of hyphens
+ variants = {stem.lower()}
+ if '-' in stem:
+ variants.add(stem.replace('-', ' ').lower())
+ # Also add concatenated form: "foobar" from "foo-bar"
+ variants.add(stem.replace('-', '').lower())
+
+ mentions = []
+ wikilink_pat = re.compile(r'\[\[[^\]]*\]\]')
+
+ for md in walk_vault(vault_dir):
+ rp = rel_path(md, vault_dir).replace('.md', '')
+ if rp in wikilink_sources or rp == target.replace('.md', ''):
+ continue # skip files already found via wikilinks
+ try:
+ content = md.read_text(errors='replace').lower()
+ except Exception:
+ continue
+ # Strip wikilinks so we only find plain-text mentions
+ content_no_wl = wikilink_pat.sub('', content)
+ for v in variants:
+ if v in content_no_wl:
+ mentions.append(rp)
+ break
+
+ return sorted(set(mentions))
+
+
+def find_backlinks(graph: dict, target: str, vault_dir: Path = None) -> tuple[list, list]:
+ """Find all files linking to target.
+ Returns: (wikilink_backlinks, text_mentions) if vault_dir given,
+ else (wikilink_backlinks, []) for backward compat."""
+ # Normalize target
+ target_clean = target.replace('.md', '')
+ results = []
+
+ if target_clean in graph['nodes']:
+ for src in graph['nodes'][target_clean].get('incoming', []):
+ results.append(src)
+ else:
+ # Fuzzy: check if target is a stem
+ stem = target_clean.split('/')[-1]
+ for path, node in graph['nodes'].items():
+ if path.endswith(stem):
+ for src in node.get('incoming', []):
+ results.append(src)
+
+ wikilinks = sorted(set(results))
+
+ # Text mentions (optional, needs vault_dir)
+ text_mentions = []
+ if vault_dir:
+ text_mentions = find_text_mentions(vault_dir, target_clean, set(wikilinks))
+
+ return wikilinks, text_mentions
+
+
+# ─── CLI ───────────────────────────────────────────────────
+def find_schema(args: list) -> Path | None:
+ """Find schema.json in args or default location."""
+ for a in args:
+ if a.endswith('.json') and Path(a).exists():
+ return Path(a)
+ default = Path(__file__).parent.parent / 'schema.json'
+ if default.exists():
+ return default
+ return None
+
+
+def main():
+ args = sys.argv[1:]
+ if not args or args[0] in ('-h', '--help'):
+ print(__doc__)
+ sys.exit(0)
+
+ cmd = args[0]
+ vault_dir = Path(args[1]) if len(args) > 1 else None
+
+ if not vault_dir or not vault_dir.is_dir():
+ print(f"Error: vault directory required", file=sys.stderr)
+ sys.exit(1)
+
+ schema_path = find_schema(args)
+ schema = load_schema(schema_path) if schema_path else {}
+
+ if cmd == 'health':
+ graph = build_graph(vault_dir, schema)
+ stats = graph['stats']
+
+ # Save outputs
+ out_dir = vault_dir / '.graph'
+ out_dir.mkdir(exist_ok=True)
+ (out_dir / 'vault-graph.json').write_text(
+ json.dumps(graph, indent=2, ensure_ascii=False, default=str))
+ (out_dir / 'report.md').write_text(
+ generate_report(stats, graph['domains']))
+ update_history(vault_dir, stats)
+
+ print(f"\n{'='*50}")
+ print(f"Health Score: {stats['health_score']}/100")
+ print(f"Total files: {stats['total_files']}")
+ print(f"Total links: {stats['total_links']}")
+ print(f"Avg links/file: {stats['avg_links']}")
+ print(f"Orphan files: {stats['orphans']}")
+ print(f"Dead-ends: {stats['dead_ends']}")
+ print(f"Broken links: {stats['broken_links']}")
+ print(f"Desc coverage: {stats['desc_coverage']}%")
+ ns_count = stats.get('nonstandard_domains', 0)
+ if ns_count > 0:
+ print(f"Bad domains: {ns_count}")
+ print(f"{'='*50}")
+ for d, ds in sorted(graph['domains'].items()):
+ print(f" {d}: {ds['files']} files, {ds['links']} links, {ds['orphans']} orphans")
+ ns_list = graph.get('nonstandard_domain_list', {})
+ if ns_list:
+ print(f"\n Non-standard domains ({ns_count} files):")
+ for domain, files in sorted(ns_list.items()):
+ print(f" '{domain}' ({len(files)} files): {', '.join(files[:5])}")
+
+ elif cmd == 'fix':
+ graph = build_graph(vault_dir, schema)
+ apply = '--apply' in args
+ fixes, applied = fix_broken_links(vault_dir, graph, apply=apply)
+ mode = 'APPLIED' if apply else 'DRY RUN'
+ print(f"\n Broken links: {len(graph['broken_link_list'])}")
+ print(f" Fixable: {len(fixes)}")
+ if apply:
+ print(f" Applied: {applied}")
+ for f in fixes[:20]:
+ print(f" {f['source']}: {f['old']} → {f['new']}")
+
+ elif cmd == 'backlinks':
+ target = args[2] if len(args) > 2 else None
+ if not target:
+ print("Usage: graph.py backlinks <vault> <target>", file=sys.stderr)
+ sys.exit(1)
+ graph = build_graph(vault_dir, schema)
+ wikilinks, mentions = find_backlinks(graph, target, vault_dir)
+ print(f"Backlinks to '{target}': {len(wikilinks)} wikilinks, {len(mentions)} text mentions")
+ if wikilinks:
+ print(f"\n Wikilinks ({len(wikilinks)}):")
+ for b in wikilinks:
+ print(f" ← {b}")
+ if mentions:
+ print(f"\n Text mentions ({len(mentions)}):")
+ for m in mentions:
+ print(f" ~ {m}")
+
+ elif cmd == 'orphans':
+ graph = build_graph(vault_dir, schema)
+ orphans = graph['orphan_list']
+ print(f"Orphans: {len(orphans)}")
+ for o in orphans:
+ print(f" {o}")
+
+ else:
+ print(f"Unknown command: {cmd}", file=sys.stderr)
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/link_cleanup.py b/vault/.claude/skills/autograph/scripts/link_cleanup.py
new file mode 100644
index 0000000..2e9f243
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/link_cleanup.py
@@ -0,0 +1,166 @@
+#!/usr/bin/env python3
+"""
+autograph link_cleanup — remove phantom wikilinks from ## Related sections.
+
+No API calls. Checks every wikilink in ## Related against real vault stems.
+Broken links are removed; if all links in a section are broken, the section is deleted.
+
+Usage:
+ link_cleanup.py <vault-dir> # dry run
+ link_cleanup.py <vault-dir> --apply
+"""
+
+import json
+import re
+import sys
+from pathlib import Path
+from datetime import datetime
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from common import walk_vault, rel_path, extract_wikilinks, build_link_index, resolve_link_target
+
+
+def build_stems_and_paths(vault_dir: Path) -> dict:
+ """Build deterministic link indexes shared with graph.py."""
+ return build_link_index(vault_dir)
+
+
+def check_link_target(target: str, valid_targets: dict) -> bool:
+ """Check if a wikilink target resolves to a real file."""
+ resolved, reason = resolve_link_target(target, valid_targets)
+ if reason == 'empty':
+ return True # empty after strip = self-link, not broken
+ return resolved is not None
+
+
+def cleanup_related_section(content: str, valid_targets: dict) -> tuple[str, list[str], list[str]]:
+ """Clean phantom links from ## Related section.
+
+ Returns (new_content, removed_links, kept_links).
+ Only touches ## Related section. Body links are never modified.
+ """
+ # Find ## Related section
+ pattern = re.compile(r'(\n## Related\n)(.*?)(?=\n## |\Z)', re.DOTALL)
+ match = pattern.search(content)
+ if not match:
+ return content, [], []
+
+ header = match.group(1)
+ section_body = match.group(2)
+
+ removed = []
+ kept = []
+ new_lines = []
+
+ for line in section_body.split('\n'):
+ links = extract_wikilinks(line)
+ if not links:
+ # Non-link line (empty, comments, etc.) — keep as-is
+ new_lines.append(line)
+ continue
+
+ # Check each link target in the line
+ line_valid = False
+ for target, _ in links:
+ if check_link_target(target, valid_targets):
+ kept.append(target)
+ line_valid = True
+ else:
+ removed.append(target)
+
+ if line_valid:
+ new_lines.append(line)
+
+ # If all links removed → delete entire section
+ if not kept:
+ new_content = content[:match.start()] + content[match.end():]
+ return new_content.rstrip() + '\n', removed, kept
+
+ # Rebuild section
+ new_section = header + '\n'.join(new_lines)
+ new_content = content[:match.start()] + new_section + content[match.end():]
+ return new_content, removed, kept
+
+
+def run_cleanup(vault_dir: Path, apply: bool = False) -> dict:
+ """Run link cleanup on vault. Returns report dict."""
+ vault_dir = vault_dir.resolve()
+ valid_targets = build_stems_and_paths(vault_dir)
+
+ report = {
+ 'total_files_scanned': 0,
+ 'files_with_related': 0,
+ 'files_modified': 0,
+ 'links_removed': [],
+ 'links_kept': 0,
+ 'timestamp': datetime.now().isoformat(),
+ 'apply': apply,
+ }
+
+ for md in walk_vault(vault_dir):
+ report['total_files_scanned'] += 1
+ content = md.read_text(errors='replace')
+
+ if '## Related' not in content:
+ continue
+
+ report['files_with_related'] += 1
+ new_content, removed, kept = cleanup_related_section(content, valid_targets)
+
+ report['links_kept'] += len(kept)
+
+ if removed:
+ rp = rel_path(md, vault_dir)
+ report['links_removed'].append({
+ 'file': rp,
+ 'removed': removed,
+ 'kept': [k for k in kept],
+ })
+ report['files_modified'] += 1
+
+ if apply:
+ md.write_text(new_content)
+
+ # Print summary
+ print(f"Scanned: {report['total_files_scanned']} files")
+ print(f"With ## Related: {report['files_with_related']}")
+ print(f"Files to {'fix' if not apply else 'fixed'}: {report['files_modified']}")
+ total_removed = sum(len(r['removed']) for r in report['links_removed'])
+ print(f"Links removed: {total_removed}")
+ print(f"Links kept: {report['links_kept']}")
+
+ if report['links_removed'] and not apply:
+ print("\nPhantom links found (dry run):")
+ for item in report['links_removed']:
+ print(f" {item['file']}:")
+ for link in item['removed']:
+ print(f" - [[{link}]] (broken)")
+
+ # Write report
+ report_dir = vault_dir / '.graph'
+ report_dir.mkdir(parents=True, exist_ok=True)
+ report_path = report_dir / 'link-cleanup-report.json'
+ # Convert for JSON serialization
+ report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2) + '\n')
+ print(f"\nReport: {report_path}")
+
+ return report
+
+
+def main():
+ args = sys.argv[1:]
+ if not args or args[0] in ('-h', '--help'):
+ print("Usage: link_cleanup.py <vault-dir> [--apply]", file=sys.stderr)
+ sys.exit(1)
+
+ vault_dir = Path(args[0])
+ if not vault_dir.is_dir():
+ print(f"Not a directory: {vault_dir}", file=sys.stderr)
+ sys.exit(1)
+
+ apply = '--apply' in args
+ run_cleanup(vault_dir, apply)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/moc.py b/vault/.claude/skills/autograph/scripts/moc.py
new file mode 100644
index 0000000..88b5cd8
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/moc.py
@@ -0,0 +1,222 @@
+#!/usr/bin/env python3
+"""
+autograph moc — generate Maps of Content from vault cards.
+All domain/type/status logic from schema.json. No hardcoded values.
+
+Commands:
+ moc.py generate <vault-dir> [schema.json]
+ moc.py generate <vault-dir> [schema.json] --domain <name>
+"""
+
+import sys
+import json
+from pathlib import Path
+from collections import defaultdict
+from datetime import datetime
+
+from common import (
+ load_schema, parse_frontmatter, walk_vault, rel_path,
+ infer_domain, get_node_types, get_status_order
+)
+
+
+def build_moc_data(vault_dir: Path, schema: dict, domain_filter: str = None) -> dict:
+ """Scan vault, group cards by domain → type → industry/category → status."""
+ files = walk_vault(vault_dir)
+ domains = defaultdict(list)
+
+ for md in files:
+ rp = rel_path(md, vault_dir)
+ try:
+ content = md.read_text(errors='replace')
+ except Exception:
+ continue
+
+ fm, body, _ = parse_frontmatter(content)
+ if fm is None:
+ continue
+
+ domain = fm.get('domain') or infer_domain(rp, schema)
+ if domain_filter and domain != domain_filter:
+ continue
+
+ card_type = fm.get('type', 'note')
+ status = fm.get('status', '')
+ name = fm.get('title') or md.stem.replace('-', ' ').title()
+ rp_noext = rp.replace('.md', '')
+
+ card = {
+ 'path': rp_noext,
+ 'name': name,
+ 'type': card_type,
+ 'status': status,
+ 'priority': fm.get('priority', ''),
+ 'industry': fm.get('industry', 'Other'),
+ 'region': fm.get('region', ''),
+ 'deal_status': fm.get('deal_status', ''),
+ 'deal_deadline': fm.get('deal_deadline', ''),
+ 'responsible': fm.get('responsible', ''),
+ 'potential': fm.get('potential', ''),
+ 'source': fm.get('source', ''),
+ 'description': fm.get('description', ''),
+ }
+ domains[domain].append(card)
+
+ return dict(domains)
+
+
+def format_card_line(card: dict) -> str:
+ """Format a single MOC line with context."""
+ path = card['path']
+ name = card['name']
+ parts = []
+
+ if card['status']:
+ parts.append(card['status'])
+ if card['priority']:
+ parts.append(f"p:{card['priority']}")
+ if card['deal_status']:
+ parts.append(card['deal_status'])
+ if card['deal_deadline']:
+ parts.append(f"⏰ {card['deal_deadline']}")
+ if card['region']:
+ parts.append(card['region'].upper())
+ if card['responsible']:
+ parts.append(f"→ {card['responsible']}")
+ if card['potential']:
+ parts.append(f"🔥{card['potential']}")
+
+ context = ', '.join(parts)
+ return f"- [[{path}|{name}]]{' — ' + context if context else ''}"
+
+
+def generate_moc(domain: str, cards: list, schema: dict) -> str:
+ """Generate MOC markdown for a domain."""
+ status_order = get_status_order(schema)
+ lines = [
+ f"---",
+ f'description: "MOC — {domain.title()} domain index ({len(cards)} cards)"',
+ f"type: moc",
+ f"---",
+ f"",
+ f"# MOC — {domain.title()}",
+ f"",
+ f"_Auto-generated by autograph. {datetime.now().strftime('%Y-%m-%d %H:%M')}_",
+ f"_Cards: {len(cards)}_",
+ f"",
+ ]
+
+ # Group by type
+ by_type = defaultdict(list)
+ for c in cards:
+ by_type[c['type']].append(c)
+
+ for card_type in sorted(by_type.keys()):
+ type_cards = by_type[card_type]
+ lines.append(f"## {card_type.title()} ({len(type_cards)})")
+ lines.append("")
+
+ # Sub-group by industry
+ by_industry = defaultdict(list)
+ for c in type_cards:
+ by_industry[c['industry']].append(c)
+
+ for industry in sorted(by_industry.keys()):
+ ind_cards = by_industry[industry]
+ if len(by_industry) > 1:
+ lines.append(f"### {industry}")
+ lines.append("")
+
+ # Sort by status priority
+ ind_cards.sort(key=lambda c: status_order.get(c['status'], 99))
+
+ for c in ind_cards:
+ lines.append(format_card_line(c))
+
+ lines.append("")
+
+ # Stats
+ type_counts = defaultdict(int)
+ status_counts = defaultdict(int)
+ for c in cards:
+ type_counts[c['type']] += 1
+ if c['status']:
+ status_counts[c['status']] += 1
+
+ lines.append("## Stats")
+ lines.append("")
+ lines.append("| Type | Count |")
+ lines.append("|------|-------|")
+ for t, count in sorted(type_counts.items(), key=lambda x: -x[1]):
+ lines.append(f"| {t} | {count} |")
+ lines.append("")
+ lines.append("| Status | Count |")
+ lines.append("|--------|-------|")
+ for s, count in sorted(status_counts.items(), key=lambda x: -x[1]):
+ lines.append(f"| {s} | {count} |")
+
+ return '\n'.join(lines)
+
+
+def main():
+ args = sys.argv[1:]
+ if not args or args[0] in ('-h', '--help'):
+ print(__doc__)
+ sys.exit(0)
+
+ cmd = args[0]
+ vault_dir = Path(args[1]) if len(args) > 1 else None
+
+ if not vault_dir or not vault_dir.is_dir():
+ print("Error: vault directory required", file=sys.stderr)
+ sys.exit(1)
+
+ schema_path = None
+ for a in args:
+ if a.endswith('.json') and Path(a).exists():
+ schema_path = Path(a)
+ break
+ schema = load_schema(schema_path)
+
+ domain_filter = None
+ if '--domain' in args:
+ idx = args.index('--domain')
+ if idx + 1 < len(args):
+ domain_filter = args[idx + 1]
+
+ if cmd == 'generate':
+ data = build_moc_data(vault_dir, schema, domain_filter)
+
+ moc_dir = vault_dir / 'MOC'
+ moc_dir.mkdir(exist_ok=True)
+ total_links = 0
+
+ if domain_filter:
+ cards = data.get(domain_filter, [])
+ moc_text = generate_moc(domain_filter, cards, schema)
+ outpath = moc_dir / f'MOC-{domain_filter}.md'
+ outpath.write_text(moc_text)
+ link_count = moc_text.count('[[')
+ total_links += link_count
+ print(f"✓ Generated: {outpath.relative_to(vault_dir)}")
+ print(f" {link_count} wikilinks")
+ else:
+ for domain, cards in sorted(data.items()):
+ if not cards:
+ continue
+ moc_text = generate_moc(domain, cards, schema)
+ outpath = moc_dir / f'MOC-{domain}.md'
+ outpath.write_text(moc_text)
+ link_count = moc_text.count('[[')
+ total_links += link_count
+ print(f"✓ Generated: {outpath.relative_to(vault_dir)}")
+ print(f" {link_count} wikilinks")
+
+ print(f"\nTotal: {total_links} wikilinks across {len(data)} domains")
+ else:
+ print(f"Unknown command: {cmd}", file=sys.stderr)
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/orchestrate.py b/vault/.claude/skills/autograph/scripts/orchestrate.py
new file mode 100644
index 0000000..7ba3a7d
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/orchestrate.py
@@ -0,0 +1,393 @@
+#!/usr/bin/env python3
+"""
+autograph orchestrate — multi-agent workflow orchestration.
+
+Phase 0 commands (script sequencing, no LLM):
+ orchestrate.py health <vault-dir>
+ orchestrate.py bootstrap <vault-dir>
+
+Data prep commands (builds input for Claude Code agent judgment):
+ orchestrate.py dedup-prepare <vault-dir>
+ orchestrate.py link-prepare <vault-dir>
+ orchestrate.py graph-prepare <vault-dir>
+
+All judgment calls are made by the Claude Code agent directly —
+no API keys, no external LLM calls. The prep commands build
+structured JSON that the agent reads, reviews, and acts on.
+"""
+
+import argparse
+import json
+import sys
+from datetime import datetime
+from pathlib import Path
+import subprocess
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from common import parse_frontmatter, load_schema, extract_wikilinks
+from enrich import scan_vault_for_links, format_catalog, chunk_list
+
+SCRIPTS_DIR = Path(__file__).resolve().parent
+
+
+# ─── SCRIPT RUNNER ───────────────────────────────────────
+
+def run_script(name: str, *args) -> tuple[int, str]:
+ script = str(SCRIPTS_DIR / name)
+ cmd = ["python3", script] + list(args)
+ print(f" > {name} {' '.join(args)}")
+ result = subprocess.run(cmd, capture_output=True, text=True)
+ output = result.stdout + result.stderr
+ if result.returncode != 0:
+ print(f" FAIL (exit {result.returncode})")
+ for line in output.strip().split('\n')[-5:]:
+ print(f" {line}")
+ return result.returncode, output
+
+
+def read_json(path: Path) -> dict:
+ try:
+ return json.loads(path.read_text())
+ except (FileNotFoundError, json.JSONDecodeError):
+ return {}
+
+
+# ═══════════════════════════════════════════════════════════
+# PHASE 0: HEALTH
+# ═══════════════════════════════════════════════════════════
+
+def cmd_health(vault_dir: Path):
+ vd = str(vault_dir)
+ print(f"\n{'='*55}")
+ print(f" AUTOGRAPH HEALTH — {vault_dir.name}")
+ print(f"{'='*55}")
+
+ print("\n[1/5] Health check...")
+ run_script('graph.py', 'health', vd)
+ graph = read_json(vault_dir / '.graph' / 'vault-graph.json')
+ score = graph.get('stats', {}).get('health_score', 0)
+ broken = graph.get('stats', {}).get('broken_links', 0)
+ orphans = graph.get('stats', {}).get('orphans', 0)
+ print(f" Score: {score}/100 broken: {broken} orphans: {orphans}")
+
+ if broken > 0:
+ print(f"\n[2/5] Fixing {broken} broken links...")
+ run_script('graph.py', 'fix', vd, '--apply')
+ else:
+ print("\n[2/5] No broken links.")
+
+ print("\n[3/5] Link cleanup...")
+ run_script('link_cleanup.py', vd, '--apply')
+
+ print("\n[4/5] MOC generation + decay...")
+ run_script('moc.py', 'generate', vd)
+ run_script('engine.py', 'decay', vd)
+
+ print("\n[5/5] Final health check...")
+ run_script('graph.py', 'health', vd)
+ graph2 = read_json(vault_dir / '.graph' / 'vault-graph.json')
+ final = graph2.get('stats', {}).get('health_score', 0)
+ delta = final - score
+
+ print(f"\n Result: {score} -> {final} ({'+' if delta >= 0 else ''}{delta})")
+ print(f" Status: {'HEALTHY' if final >= 90 else 'NEEDS ATTENTION'}")
+
+
+# ═══════════════════════════════════════════════════════════
+# PHASE 0: BOOTSTRAP
+# ═══════════════════════════════════════════════════════════
+
+def cmd_bootstrap(vault_dir: Path):
+ vd = str(vault_dir)
+ print(f"\n{'='*55}")
+ print(f" AUTOGRAPH BOOTSTRAP — {vault_dir.name}")
+ print(f"{'='*55}")
+
+ steps = [
+ ("Enforce", 'enforce.py', [vd, '--apply']),
+ ("Link cleanup", 'link_cleanup.py', [vd, '--apply']),
+ ("Tag enrichment", 'enrich.py', ['tags', vd, '--apply']),
+ ("Dedup (scan)", 'dedup.py', [vd, '--manifest',
+ str(vault_dir / '.graph' / 'dedup-manifest.json')]),
+ ]
+
+ for i, (label, script, args) in enumerate(steps, 1):
+ print(f"\n[{i}/7] {label}...")
+ rc, _ = run_script(script, *args)
+ if rc != 0 and 'Dedup' not in label:
+ print(f" ABORT: {label} failed")
+ return
+
+ manifest = read_json(vault_dir / '.graph' / 'dedup-manifest.json')
+ clusters = manifest.get('clusters', [])
+ if clusters:
+ print(f"\n[5/7] Preparing dedup review ({len(clusters)} clusters)...")
+ cmd_dedup_prepare(vault_dir)
+ print(f" >> Agent: review .graph/dedup-review-input.json and approve/reject clusters")
+ else:
+ print(f"\n[5/7] No duplicates.")
+
+ print(f"\n[6/7] Link enrichment (swarm-links)...")
+ run_script('enrich.py', 'swarm-links', vd, '--apply')
+
+ print(f"\n[7/7] MOC + final health...")
+ run_script('moc.py', 'generate', vd)
+ run_script('graph.py', 'health', vd)
+ graph = read_json(vault_dir / '.graph' / 'vault-graph.json')
+ score = graph.get('stats', {}).get('health_score', 0)
+ print(f"\n Final health: {score}/100")
+
+
+# ═══════════════════════════════════════════════════════════
+# PHASE 1: DEDUP PREPARE (data for agent judgment)
+# ═══════════════════════════════════════════════════════════
+
+def cmd_dedup_prepare(vault_dir: Path):
+ """Build enriched dedup manifest for agent review.
+
+ Reads dedup manifest, adds content previews for each file,
+ writes .graph/dedup-review-input.json.
+ The agent reads this file, reviews clusters, and writes
+ decisions to the manifest's 'approved' field.
+ """
+ manifest_path = vault_dir / '.graph' / 'dedup-manifest.json'
+
+ if not manifest_path.exists():
+ print(" Generating dedup manifest...")
+ run_script('dedup.py', str(vault_dir), '--manifest', str(manifest_path))
+
+ manifest = read_json(manifest_path)
+ clusters = manifest.get('clusters', [])
+ if not clusters:
+ print(" No dedup clusters.")
+ return
+
+ enriched_clusters = []
+ for cluster in clusters:
+ previews = {}
+ all_paths = [cluster.get('canonical', '')] + list(cluster.get('extras', []))
+ for path in all_paths:
+ if not path:
+ continue
+ full = vault_dir / path
+ if not full.exists():
+ continue
+ content = full.read_text(errors='replace')
+ fm, body, _ = parse_frontmatter(content)
+ previews[path] = {
+ 'frontmatter': {k: v for k, v in (fm or {}).items()
+ if k in ('description', 'tags', 'status', 'type',
+ 'domain', 'source', 'potential', 'priority')},
+ 'body_preview': (body or '')[:500].strip(),
+ }
+ enriched_clusters.append({
+ 'id': cluster.get('id', ''),
+ 'action': cluster.get('action', ''),
+ 'risk': cluster.get('risk', ''),
+ 'reason': cluster.get('reason', ''),
+ 'canonical': cluster.get('canonical', ''),
+ 'extras': cluster.get('extras', []),
+ 'records': [{'stem': r.get('stem'), 'kind': r.get('kind'),
+ 'richness': r.get('richness'), 'title': r.get('title'),
+ 'status': r.get('status')} for r in cluster.get('records', [])],
+ 'previews': previews,
+ })
+
+ output = {
+ 'generated_at': datetime.now().isoformat(),
+ 'vault': str(vault_dir),
+ 'cluster_count': len(enriched_clusters),
+ 'clusters': enriched_clusters,
+ 'instructions': (
+ 'Review each cluster. For each, decide: merge_duplicate (safe) or manual_hold (ambiguous). '
+ 'Set "approved": true on safe merges. Then run: '
+ 'python3 dedup.py <vault> --apply-manifest <this-file>'
+ ),
+ }
+
+ out_path = vault_dir / '.graph' / 'dedup-review-input.json'
+ out_path.write_text(json.dumps(output, indent=2, ensure_ascii=False) + '\n')
+ print(f" {len(enriched_clusters)} clusters prepared: {out_path}")
+
+
+# ═══════════════════════════════════════════════════════════
+# PHASE 2: LINK PREPARE (data for agent judgment)
+# ═══════════════════════════════════════════════════════════
+
+def cmd_link_prepare(vault_dir: Path, force: bool = False):
+ """Build per-domain catalogs and file lists for agent link suggestions.
+
+ Writes .graph/link-review-input.json with domain catalogs and
+ files needing links. The agent reads this, suggests links per
+ domain, and writes results to .graph/enrich/specialists/.
+ """
+ vault_dir = vault_dir.resolve()
+ all_stems, stem_to_path, catalog, entries = scan_vault_for_links(vault_dir, force)
+
+ print(f" Stems: {len(all_stems)} Domains: {len(catalog)} Needing links: {len(entries)}")
+ if not entries:
+ print(" Nothing to do.")
+ return
+
+ domain_data = {}
+ for e in entries:
+ d = e.get('domain', '') or 'general'
+ domain_data.setdefault(d, {'files': [], 'catalog': ''})
+ domain_data[d]['files'].append({
+ 'path': e['path'],
+ 'type': e.get('type', ''),
+ 'tags': e.get('tags', [])[:10],
+ 'existing_links': e.get('existing_links', []),
+ 'summary': e.get('summary', '')[:400],
+ })
+
+ for domain in domain_data:
+ domain_catalog = catalog.get(domain, [])
+ cross = []
+ for other_d, other_e in catalog.items():
+ if other_d != domain:
+ cross.extend(other_e[:30])
+ domain_data[domain]['catalog'] = format_catalog(domain_catalog + cross)
+
+ output = {
+ 'generated_at': datetime.now().isoformat(),
+ 'vault': str(vault_dir),
+ 'all_stems': sorted(all_stems),
+ 'stem_to_path': stem_to_path,
+ 'domains': {d: {'file_count': len(v['files']), 'catalog_lines': v['catalog'].count('\n') + 1}
+ for d, v in domain_data.items()},
+ 'domain_data': domain_data,
+ 'instructions': (
+ 'For each domain, review files and suggest 3-8 links from the catalog. '
+ 'ONLY use stems from all_stems. No self-links, no existing links. '
+ 'Write results as batch-NNN-results.json to .graph/enrich/specialists/. '
+ 'Then run: python3 enrich.py swarm-links <vault> --apply '
+ 'to apply from that directory.'
+ ),
+ }
+
+ out_path = vault_dir / '.graph' / 'link-review-input.json'
+ out_path.write_text(json.dumps(output, indent=2, ensure_ascii=False) + '\n')
+ print(f" {len(domain_data)} domains, {len(entries)} files prepared: {out_path}")
+
+
+# ═══════════════════════════════════════════════════════════
+# PHASE 3: GRAPH PREPARE (data for agent analysis)
+# ═══════════════════════════════════════════════════════════
+
+def cmd_graph_prepare(vault_dir: Path):
+ """Build graph analysis input for agent semantic review.
+
+ Runs graph.py health, then builds a concise summary of the graph
+ for the agent to analyze: domain distributions, orphans, broken
+ links, sample nodes.
+ """
+ vault_dir = vault_dir.resolve()
+
+ print(" Running health check...")
+ run_script('graph.py', 'health', str(vault_dir))
+
+ graph_path = vault_dir / '.graph' / 'vault-graph.json'
+ if not graph_path.exists():
+ print(" ERROR: No vault-graph.json")
+ return
+
+ graph = json.loads(graph_path.read_text())
+ nodes = graph.get('nodes', {})
+ stats = graph.get('stats', {})
+
+ domain_counts: dict[str, int] = {}
+ type_counts: dict[str, int] = {}
+ sample_nodes = []
+
+ for path, node in list(nodes.items()):
+ d = node.get('domain', 'unknown')
+ t = node.get('type', 'unknown')
+ domain_counts[d] = domain_counts.get(d, 0) + 1
+ type_counts[t] = type_counts.get(t, 0) + 1
+
+ if len(sample_nodes) < 200:
+ sample_nodes.append({
+ 'path': path,
+ 'type': t,
+ 'domain': d,
+ 'has_desc': node.get('has_description', False),
+ 'outgoing': len(node.get('outgoing', [])),
+ 'incoming': len(node.get('incoming', [])),
+ })
+
+ output = {
+ 'generated_at': datetime.now().isoformat(),
+ 'vault': str(vault_dir),
+ 'stats': stats,
+ 'domain_distribution': domain_counts,
+ 'type_distribution': type_counts,
+ 'orphans': graph.get('orphan_list', [])[:50],
+ 'broken_links': graph.get('broken_link_list', [])[:30],
+ 'dead_ends': graph.get('dead_end_list', [])[:30],
+ 'sample_nodes': sample_nodes,
+ 'instructions': (
+ 'Analyze for: contradictions (active->archived links), '
+ 'missing obvious links, stale hubs, isolated clusters, '
+ 'decay anomalies. Write findings to .graph/graph-intelligence.json.'
+ ),
+ }
+
+ out_path = vault_dir / '.graph' / 'graph-analysis-input.json'
+ out_path.write_text(json.dumps(output, indent=2, ensure_ascii=False) + '\n')
+ print(f" {len(nodes)} nodes, {len(sample_nodes)} sampled: {out_path}")
+
+
+# ═══════════════════════════════════════════════════════════
+# CLI
+# ═══════════════════════════════════════════════════════════
+
+def main(argv: list[str] | None = None):
+ parser = argparse.ArgumentParser(
+ description='autograph orchestrate — multi-agent workflows (no API keys)',
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ epilog="""\
+Phase 0 (script sequencing):
+ health run full health workflow
+ bootstrap run full bootstrap workflow
+
+Data prep (for Claude Code agent judgment):
+ dedup-prepare enrich manifest for agent review
+ link-prepare build domain catalogs for agent link suggestions
+ graph-prepare build graph summary for agent analysis""")
+
+ sub = parser.add_subparsers(dest='command')
+
+ sub.add_parser('health').add_argument('vault_dir')
+ sub.add_parser('bootstrap').add_argument('vault_dir')
+
+ sub.add_parser('dedup-prepare').add_argument('vault_dir')
+
+ lp = sub.add_parser('link-prepare')
+ lp.add_argument('vault_dir')
+ lp.add_argument('--force', action='store_true')
+
+ sub.add_parser('graph-prepare').add_argument('vault_dir')
+
+ ns = parser.parse_args(argv)
+ if not ns.command:
+ parser.print_help()
+ sys.exit(1)
+
+ vault = Path(ns.vault_dir).resolve()
+ if not vault.is_dir():
+ print(f"ERROR: {vault} is not a directory", file=sys.stderr)
+ sys.exit(1)
+
+ cmds = {
+ 'health': lambda: cmd_health(vault),
+ 'bootstrap': lambda: cmd_bootstrap(vault),
+ 'dedup-prepare': lambda: cmd_dedup_prepare(vault),
+ 'link-prepare': lambda: cmd_link_prepare(vault, getattr(ns, 'force', False)),
+ 'graph-prepare': lambda: cmd_graph_prepare(vault),
+ }
+ cmds[ns.command]()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/research.py b/vault/.claude/skills/autograph/scripts/research.py
new file mode 100644
index 0000000..f5673e9
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/research.py
@@ -0,0 +1,303 @@
+#!/usr/bin/env python3
+"""
+autograph research — helper for /autograph:research slash command.
+
+Two subcommands:
+
+ plan <vault> → gate + small manifests for 3–5 exploration agents
+ reduce <vault> <observations> <answers> → schema draft from agents + user Q&A
+
+The orchestration (spawning agents, asking users) lives in commands/research.md.
+This script only does deterministic work: file walks, bin-packing, schema synthesis.
+"""
+
+import json
+import sys
+from pathlib import Path
+from collections import Counter
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from common import walk_vault, rel_path, parse_frontmatter # noqa: E402
+
+MAX_AGENTS = 5
+MAX_FILES_PER_AGENT = 20
+SAMPLE_TITLES = 5
+
+# ─── GATE ─────────────────────────────────────────────────
+
+def gate(vault_dir: Path) -> dict:
+ """Classify vault as empty / chaos / structured."""
+ files = walk_vault(vault_dir)
+ total = len(files)
+ folders = {rel_path(f, vault_dir).split('/')[0] for f in files if '/' in rel_path(f, vault_dir)}
+ has_schema = (vault_dir / 'schema.json').exists()
+
+ fm_hits = 0
+ typed_hits = 0
+ for md in files[:500]:
+ try:
+ fm, _, _ = parse_frontmatter(md.read_text(errors='replace'))
+ except Exception:
+ continue
+ if fm:
+ fm_hits += 1
+ if fm.get('type') or fm.get('domain'):
+ typed_hits += 1
+
+ sampled = min(total, 500)
+ fm_coverage = (fm_hits / sampled) if sampled else 0.0
+ typed_coverage = (typed_hits / sampled) if sampled else 0.0
+
+ if total <= 20:
+ verdict = 'empty'
+ elif has_schema and typed_coverage >= 0.3:
+ verdict = 'structured'
+ elif len(folders) >= 3 and typed_coverage >= 0.3:
+ verdict = 'structured'
+ else:
+ verdict = 'chaos'
+
+ return {
+ 'files_total': total,
+ 'folders_total': len(folders),
+ 'has_schema_json': has_schema,
+ 'frontmatter_coverage': round(fm_coverage, 3),
+ 'typed_coverage': round(typed_coverage, 3),
+ 'verdict': verdict,
+ }
+
+
+# ─── PLAN ─────────────────────────────────────────────────
+
+def _bin_pack(files: list[Path], vault_dir: Path, max_agents: int, max_per_agent: int) -> list[list[str]]:
+ """Group files by top folder, then spread across up to max_agents buckets."""
+ by_folder: dict[str, list[str]] = {}
+ for f in files:
+ rp = rel_path(f, vault_dir)
+ top = rp.split('/')[0] if '/' in rp else '_root'
+ by_folder.setdefault(top, []).append(rp)
+
+ # Sort folders largest-first so each agent gets a balanced mix.
+ folders = sorted(by_folder, key=lambda f: -len(by_folder[f]))
+ buckets: list[list[str]] = [[] for _ in range(max_agents)]
+ i = 0
+ for folder in folders:
+ for path in by_folder[folder]:
+ # Round-robin into buckets that still have room.
+ for _ in range(max_agents):
+ if len(buckets[i % max_agents]) < max_per_agent:
+ buckets[i % max_agents].append(path)
+ i += 1
+ break
+ i += 1
+ else:
+ break # all buckets full
+
+ return [b for b in buckets if b]
+
+
+def _sample_titles(files: list[Path], n: int) -> list[str]:
+ titles = []
+ for md in files[: n * 3]:
+ titles.append(md.stem)
+ if len(titles) >= n:
+ break
+ return titles
+
+
+def cmd_plan(vault_dir: Path) -> dict:
+ vault_dir = vault_dir.resolve()
+ g = gate(vault_dir)
+ files = walk_vault(vault_dir)
+
+ manifests = []
+ if g['verdict'] == 'chaos':
+ buckets = _bin_pack(files, vault_dir, MAX_AGENTS, MAX_FILES_PER_AGENT)
+ for idx, bucket in enumerate(buckets, 1):
+ manifests.append({
+ 'agent_id': f'explorer-{idx:02d}',
+ 'file_count': len(bucket),
+ 'files': bucket,
+ })
+
+ folder_counts = Counter()
+ for f in files:
+ rp = rel_path(f, vault_dir)
+ folder_counts[rp.split('/')[0] if '/' in rp else '_root'] += 1
+
+ return {
+ 'vault_dir': str(vault_dir),
+ 'gate': g,
+ 'top_folders': folder_counts.most_common(10),
+ 'sample_titles': _sample_titles(files, SAMPLE_TITLES),
+ 'manifests': manifests,
+ 'agent_prompt_template': (
+ "You are a vault exploration agent. Read the files listed below in full "
+ "and return ONE JSON object (no prose) with fields: observed_themes "
+ "(5-10 short tags), entity_types (list of kinds you see: person, project, "
+ "note, meeting, etc.), frontmatter_fields (fields actually used in "
+ "frontmatter), language (ru/en/mixed), folder_semantics (object mapping "
+ "folder -> one-line purpose), sample_titles (5 titles). Focus on patterns, "
+ "not per-file classification. Keep under 400 words total.\n\nFILES:\n{files}"
+ ),
+ }
+
+
+# ─── REDUCE ───────────────────────────────────────────────
+
+DEFAULT_STATUS_SETS = {
+ 'minimal': ['active', 'done'],
+ 'standard': ['active', 'draft', 'done', 'archived'],
+ 'full': ['active', 'draft', 'in-progress', 'blocked', 'done', 'archived'],
+}
+
+DEFAULT_DECAY_PROFILES = {
+ 'fast': {'rate': 0.025, 'tiers': {'active': 5, 'warm': 14, 'cold': 30}},
+ 'balanced': {'rate': 0.015, 'tiers': {'active': 7, 'warm': 21, 'cold': 60}},
+ 'slow': {'rate': 0.008, 'tiers': {'active': 14, 'warm': 45, 'cold': 120}},
+}
+
+
+def _merge_observations(observations: list[dict]) -> dict:
+ themes = Counter()
+ entity_types = Counter()
+ fm_fields = Counter()
+ languages = Counter()
+ folder_semantics: dict[str, list[str]] = {}
+
+ for obs in observations:
+ for t in obs.get('observed_themes', []) or []:
+ themes[str(t).lower()] += 1
+ for et in obs.get('entity_types', []) or []:
+ entity_types[str(et).lower()] += 1
+ for f in obs.get('frontmatter_fields', []) or []:
+ fm_fields[str(f).lower()] += 1
+ lang = obs.get('language')
+ if lang:
+ languages[str(lang).lower()] += 1
+ for folder, purpose in (obs.get('folder_semantics') or {}).items():
+ folder_semantics.setdefault(folder, []).append(str(purpose))
+
+ return {
+ 'themes': themes,
+ 'entity_types': entity_types,
+ 'frontmatter_fields': fm_fields,
+ 'languages': languages,
+ 'folder_semantics': folder_semantics,
+ }
+
+
+def cmd_reduce(vault_dir: Path, observations_path: Path, answers_path: Path) -> dict:
+ observations = json.loads(observations_path.read_text())
+ if isinstance(observations, dict):
+ observations = [observations]
+ answers = json.loads(answers_path.read_text())
+
+ merged = _merge_observations(observations)
+
+ # User answers drive the taxonomy — observations are hints only.
+ purpose = (answers.get('purpose') or 'mixed').lower()
+ ambition = (answers.get('ambition') or 'standard').lower()
+ declared_domains = answers.get('domains') or []
+ status_choice = (answers.get('status_profile') or 'standard').lower()
+ decay_choice = (answers.get('decay_profile') or 'balanced').lower()
+ domain_map = answers.get('domain_map') or {}
+
+ # node_types: intersect user ambition with observed entity_types.
+ ambition_caps = {'minimal': 5, 'standard': 10, 'full': 15}
+ cap = ambition_caps.get(ambition, 10)
+ candidate_types = [t for t, _ in merged['entity_types'].most_common() if t.isidentifier()]
+ if 'note' not in candidate_types:
+ candidate_types.insert(0, 'note')
+ node_types = {}
+ for t in candidate_types[:cap]:
+ node_types[t] = {'status': DEFAULT_STATUS_SETS.get(status_choice, DEFAULT_STATUS_SETS['standard'])}
+
+ # domain_inference: prefer user-provided map, fall back to declared domains on top folders.
+ domain_inference = dict(domain_map)
+ for folder in merged['folder_semantics']:
+ if folder in domain_inference or folder == '_root':
+ continue
+ if declared_domains:
+ domain_inference[f'{folder}/'] = declared_domains[0]
+
+ decay = DEFAULT_DECAY_PROFILES.get(decay_choice, DEFAULT_DECAY_PROFILES['balanced'])
+
+ schema = {
+ '_generated_by': 'autograph /research',
+ '_purpose': purpose,
+ '_ambition': ambition,
+ 'node_types': node_types,
+ 'type_aliases': {},
+ 'field_fixes': {},
+ 'domain_inference': domain_inference,
+ 'path_type_hints': {},
+ 'status_order': {s: i for i, s in enumerate(
+ DEFAULT_STATUS_SETS.get(status_choice, DEFAULT_STATUS_SETS['standard'])
+ )},
+ 'status_defaults': {'default': 'active', 'type': 'status'},
+ 'richness_fields': {'bonus_fields': sorted(merged['frontmatter_fields'])[:10]},
+ 'entity_extraction': {'noise_words': ['TODO', 'FIX', 'NOTE', 'WIP']},
+ 'decay': {
+ 'rate': decay['rate'],
+ 'floor': 0.1,
+ 'tiers': decay['tiers'],
+ 'domain_rates': {},
+ },
+ 'ignore_tags': [],
+ }
+ return schema
+
+
+# ─── MAIN ─────────────────────────────────────────────────
+
+USAGE = """\
+Usage:
+ research.py plan <vault-dir>
+ research.py reduce <vault-dir> <observations.json> <user-answers.json>
+"""
+
+
+def main():
+ args = sys.argv[1:]
+ if not args or args[0] in ('-h', '--help'):
+ sys.stdout.write(USAGE)
+ sys.exit(0)
+
+ cmd = args[0]
+ if cmd == 'plan':
+ if len(args) < 2:
+ sys.stderr.write(USAGE)
+ sys.exit(1)
+ vault_dir = Path(args[1])
+ if not vault_dir.is_dir():
+ sys.stderr.write(f"Not a directory: {vault_dir}\n")
+ sys.exit(1)
+ result = cmd_plan(vault_dir)
+ sys.stdout.write(json.dumps(result, ensure_ascii=False, indent=2))
+ sys.stdout.write('\n')
+ return
+
+ if cmd == 'reduce':
+ if len(args) < 4:
+ sys.stderr.write(USAGE)
+ sys.exit(1)
+ vault_dir = Path(args[1])
+ observations_path = Path(args[2])
+ answers_path = Path(args[3])
+ for p in (observations_path, answers_path):
+ if not p.exists():
+ sys.stderr.write(f"Missing: {p}\n")
+ sys.exit(1)
+ schema = cmd_reduce(vault_dir, observations_path, answers_path)
+ sys.stdout.write(json.dumps(schema, ensure_ascii=False, indent=2))
+ sys.stdout.write('\n')
+ return
+
+ sys.stderr.write(f"Unknown subcommand: {cmd}\n")
+ sys.stderr.write(USAGE)
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/swarm_prepare.py b/vault/.claude/skills/autograph/scripts/swarm_prepare.py
new file mode 100644
index 0000000..646098e
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/swarm_prepare.py
@@ -0,0 +1,258 @@
+#!/usr/bin/env python3
+"""
+autograph swarm_prepare — split vault into batches for parallel LLM classification.
+
+Map phase: walks vault, estimates tokens per file, greedy bin-packs into batches
+that fit a token budget, writes manifests for Wave 1 agents.
+
+Usage: python3 swarm_prepare.py <vault-dir> [discovery.json] [--budget 50000]
+"""
+
+import json
+import sys
+from pathlib import Path
+from datetime import datetime
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from common import walk_vault, rel_path # noqa: E402
+
+# ─── PROMPT TEMPLATE ──────────────────────────────────────
+
+WAVE1_PROMPT_TEMPLATE = """\
+You are a file classifier. Read each file, output ONE JSONL line per file.
+
+SEED TYPES (prefer these): {seed_types}
+SEED DOMAINS (prefer these): {seed_domains}
+
+Rules:
+- Use seed type if it fits. Only propose new if NONE fit.
+- Output ONLY JSONL lines, no explanations.
+- Empty/unreadable files: type="note", domain="personal", confidence="low"
+
+Format per line:
+{{"path":"<path>","proposed_type":"<type>","proposed_domain":"<domain>","summary":"<120 chars>","seed_match":true,"confidence":"high|medium|low"}}
+
+FILES:
+{file_list}
+"""
+
+# ─── HELPERS ──────────────────────────────────────────────
+
+def estimate_tokens(file_path: Path) -> int:
+ """Estimate token count from file size (~4 bytes per token)."""
+ try:
+ return file_path.stat().st_size // 4
+ except OSError:
+ return 0
+
+
+def top_folder(rel: str) -> str:
+ """Extract top-level folder from relative path, or '_root' for root files."""
+ parts = Path(rel).parts
+ return parts[0] if len(parts) > 1 else '_root'
+
+
+def extract_seed_types(discovery: dict | None) -> list[str]:
+ """Extract seed types from discovery JSON enums."""
+ if not discovery:
+ return ['note']
+ enums = discovery.get('enums', {})
+ type_vals = enums.get('type', {}).get('values', {})
+ if type_vals:
+ return sorted(type_vals, key=lambda t: -type_vals[t])
+ return ['note']
+
+
+def extract_seed_domains(discovery: dict | None) -> list[str]:
+ """Extract seed domains from discovery JSON folder structure."""
+ if not discovery:
+ return ['personal']
+ folders = discovery.get('folder_structure', {})
+ if not folders:
+ return ['personal']
+ # Use folder names as domain hints
+ domains = set()
+ for folder in folders:
+ low = folder.lower().rstrip('/')
+ if low and low != 'root':
+ domains.add(low)
+ return sorted(domains) if domains else ['personal']
+
+
+def augment_seeds_from_schema(discovery: dict | None, seed_types: list[str]) -> list[str]:
+ """Try to augment seed types using generate_schema.build_node_types if available."""
+ try:
+ from generate_schema import build_node_types
+ enums = (discovery or {}).get('enums', {})
+ node_types = build_node_types(enums)
+ for t in node_types:
+ if t not in seed_types:
+ seed_types.append(t)
+ except (ImportError, Exception):
+ pass
+ return seed_types
+
+
+def bin_pack_batches(file_entries: list[dict], budget: int) -> list[list[dict]]:
+ """Greedy bin-packing: group files into batches respecting token budget.
+
+ Groups by top-level folder first (keeps related files together),
+ then packs greedily into budget-sized batches.
+ """
+ # Group by folder
+ by_folder: dict[str, list[dict]] = {}
+ for entry in file_entries:
+ folder = entry['folder']
+ by_folder.setdefault(folder, []).append(entry)
+
+ # Sort folders by total tokens descending (largest folders split first)
+ folder_order = sorted(by_folder, key=lambda f: sum(e['tokens'] for e in by_folder[f]), reverse=True)
+
+ batches: list[list[dict]] = []
+ current_batch: list[dict] = []
+ current_tokens = 0
+
+ for folder in folder_order:
+ for entry in by_folder[folder]:
+ tok = entry['tokens']
+ # Single file exceeds budget → it gets its own batch
+ if tok >= budget:
+ if current_batch:
+ batches.append(current_batch)
+ current_batch = []
+ current_tokens = 0
+ batches.append([entry])
+ continue
+
+ if current_tokens + tok > budget and current_batch:
+ batches.append(current_batch)
+ current_batch = []
+ current_tokens = 0
+
+ current_batch.append(entry)
+ current_tokens += tok
+
+ if current_batch:
+ batches.append(current_batch)
+
+ return batches
+
+
+# ─── MAIN ─────────────────────────────────────────────────
+
+def prepare(vault_dir: Path, discovery_path: Path | None = None, budget: int = 50000) -> dict:
+ """Main prepare logic. Returns swarm-meta dict."""
+ vault_dir = vault_dir.resolve()
+
+ # Load discovery JSON if provided
+ discovery = None
+ if discovery_path and discovery_path.exists():
+ discovery = json.loads(discovery_path.read_text())
+
+ # Walk vault and build file entries
+ md_files = walk_vault(vault_dir)
+ file_entries = []
+ for md in md_files:
+ rp = rel_path(md, vault_dir)
+ tokens = estimate_tokens(md)
+ file_entries.append({
+ 'path': rp,
+ 'folder': top_folder(rp),
+ 'tokens': tokens,
+ })
+
+ if not file_entries:
+ print("No files found in vault.", file=sys.stderr)
+ return {'total_batches': 0, 'total_files': 0}
+
+ # Extract seeds
+ seed_types = extract_seed_types(discovery)
+ seed_types = augment_seeds_from_schema(discovery, seed_types)
+ seed_domains = extract_seed_domains(discovery)
+
+ # Bin-pack into batches
+ batches = bin_pack_batches(file_entries, budget)
+
+ # Create output directories
+ swarm_dir = vault_dir / '.graph' / 'swarm'
+ manifests_dir = swarm_dir / 'manifests'
+ classifications_dir = swarm_dir / 'classifications'
+ manifests_dir.mkdir(parents=True, exist_ok=True)
+ classifications_dir.mkdir(parents=True, exist_ok=True)
+
+ # Write manifests
+ total_batches = len(batches)
+ for i, batch in enumerate(batches):
+ batch_id = f"batch-{i+1:03d}"
+ manifest = {
+ 'batch_id': batch_id,
+ 'vault_dir': str(vault_dir),
+ 'total_batches': total_batches,
+ 'files': [e['path'] for e in batch],
+ 'file_count': len(batch),
+ 'estimated_tokens': sum(e['tokens'] for e in batch),
+ 'seed_types': seed_types,
+ 'seed_domains': seed_domains,
+ }
+ manifest_path = manifests_dir / f'{batch_id}.json'
+ manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + '\n')
+
+ # Write swarm-meta.json
+ total_tokens = sum(e['tokens'] for e in file_entries)
+ meta = {
+ 'vault_dir': str(vault_dir),
+ 'total_files': len(file_entries),
+ 'total_tokens': total_tokens,
+ 'total_batches': total_batches,
+ 'budget_per_batch': budget,
+ 'seed_types': seed_types,
+ 'seed_domains': seed_domains,
+ 'discovery_path': str(discovery_path) if discovery_path else None,
+ 'created_at': datetime.now().isoformat(),
+ 'wave1_prompt_template': WAVE1_PROMPT_TEMPLATE,
+ }
+ meta_path = swarm_dir / 'swarm-meta.json'
+ meta_path.write_text(json.dumps(meta, ensure_ascii=False, indent=2) + '\n')
+
+ # Print summary
+ print(f"Swarm prepared: {total_batches} batches, {len(file_entries)} files, ~{total_tokens} tokens")
+ print(f"Budget per batch: {budget} tokens")
+ print(f"Seed types: {seed_types}")
+ print(f"Seed domains: {seed_domains}")
+ print(f"Manifests: {manifests_dir}")
+ for i, batch in enumerate(batches):
+ batch_tokens = sum(e['tokens'] for e in batch)
+ print(f" batch-{i+1:03d}: {len(batch)} files, ~{batch_tokens} tokens")
+
+ return meta
+
+
+def main():
+ args = sys.argv[1:]
+ if not args or args[0] in ('-h', '--help'):
+ print("Usage: swarm_prepare.py <vault-dir> [discovery.json] [--budget N]",
+ file=sys.stderr)
+ sys.exit(1)
+
+ vault_dir = Path(args[0])
+ if not vault_dir.is_dir():
+ print(f"Not a directory: {vault_dir}", file=sys.stderr)
+ sys.exit(1)
+
+ discovery_path = None
+ budget = 50000
+
+ i = 1
+ while i < len(args):
+ if args[i] == '--budget' and i + 1 < len(args):
+ budget = int(args[i + 1])
+ i += 2
+ else:
+ discovery_path = Path(args[i])
+ i += 1
+
+ prepare(vault_dir, discovery_path, budget)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/scripts/swarm_reduce.py b/vault/.claude/skills/autograph/scripts/swarm_reduce.py
new file mode 100644
index 0000000..a97db9f
--- /dev/null
+++ b/vault/.claude/skills/autograph/scripts/swarm_reduce.py
@@ -0,0 +1,305 @@
+#!/usr/bin/env python3
+"""
+autograph swarm_reduce — consolidate Wave 1 classifications and validate Wave 2 schema.
+
+Two commands:
+ prepare — read JSONL classifications, count frequencies, output consolidation.json
+ finalize — validate Wave 2 agent output, write schema.json
+
+Usage:
+ python3 swarm_reduce.py prepare <vault-dir> [discovery.json] [draft-schema.json]
+ python3 swarm_reduce.py finalize <wave2-output.json> [output-schema.json]
+"""
+
+import json
+import sys
+from pathlib import Path
+from collections import Counter
+from datetime import datetime
+
+# ─── PROMPT TEMPLATE ──────────────────────────────────────
+
+WAVE2_PROMPT_TEMPLATE = """\
+You are a schema architect. Produce a minimal schema.json.
+
+Rules:
+- Target 5-10 node_types. Fewer is better. Merge similar types.
+- Types with <5 files: merge into broader type or drop.
+- Novel types (not seed) need >10 files to survive.
+- All 11 schema sections must be present:
+ node_types, type_aliases, field_fixes, domain_inference,
+ path_type_hints, status_order, status_defaults, richness_fields,
+ entity_extraction, decay, ignore_tags
+- Output ONLY valid JSON, no explanations.
+
+Inputs provided: type frequencies, domain frequencies, all classifications,
+draft schema as starting point, discovery metadata.
+"""
+
+REQUIRED_SCHEMA_SECTIONS = frozenset({
+ 'node_types', 'type_aliases', 'field_fixes', 'domain_inference',
+ 'path_type_hints', 'status_order', 'status_defaults', 'richness_fields',
+ 'entity_extraction', 'decay', 'ignore_tags',
+})
+
+# ─── PREPARE ──────────────────────────────────────────────
+
+def read_classifications(swarm_dir: Path) -> list[dict]:
+ """Read all JSONL files from classifications/ directory."""
+ classifications_dir = swarm_dir / 'classifications'
+ if not classifications_dir.exists():
+ return []
+
+ results = []
+ malformed = 0
+ for jsonl_file in sorted(classifications_dir.glob('*.jsonl')):
+ for line_num, line in enumerate(jsonl_file.read_text().splitlines(), 1):
+ line = line.strip()
+ if not line:
+ continue
+ try:
+ entry = json.loads(line)
+ if 'path' in entry:
+ results.append(entry)
+ else:
+ malformed += 1
+ print(f"WARN: missing 'path' in {jsonl_file.name}:{line_num}",
+ file=sys.stderr)
+ except json.JSONDecodeError:
+ malformed += 1
+ print(f"WARN: malformed JSON in {jsonl_file.name}:{line_num}",
+ file=sys.stderr)
+
+ if malformed:
+ print(f"Skipped {malformed} malformed lines.", file=sys.stderr)
+ return results
+
+
+def count_frequencies(classifications: list[dict], seed_types: list[str] | None = None
+ ) -> dict:
+ """Count type and domain frequencies from classifications."""
+ type_freq: Counter = Counter()
+ domain_freq: Counter = Counter()
+ confidence_dist: Counter = Counter()
+ novel_types: Counter = Counter()
+
+ seed_set = set(seed_types or [])
+
+ for entry in classifications:
+ ptype = entry.get('proposed_type', 'note')
+ pdomain = entry.get('proposed_domain', 'personal')
+ conf = entry.get('confidence', 'medium')
+
+ type_freq[ptype] += 1
+ domain_freq[pdomain] += 1
+ confidence_dist[conf] += 1
+
+ if ptype not in seed_set:
+ novel_types[ptype] += 1
+
+ return {
+ 'type_frequency': dict(type_freq.most_common()),
+ 'domain_frequency': dict(domain_freq.most_common()),
+ 'confidence_distribution': dict(confidence_dist.most_common()),
+ 'novel_types': dict(novel_types.most_common()),
+ 'total_classified': len(classifications),
+ }
+
+
+def prepare_consolidation(vault_dir: Path, discovery_path: Path | None = None,
+ draft_schema_path: Path | None = None) -> dict:
+ """Read Wave 1 outputs, produce consolidation.json for Wave 2."""
+ vault_dir = vault_dir.resolve()
+ swarm_dir = vault_dir / '.graph' / 'swarm'
+
+ # Read swarm meta for seed types
+ meta_path = swarm_dir / 'swarm-meta.json'
+ meta = {}
+ if meta_path.exists():
+ meta = json.loads(meta_path.read_text())
+ seed_types = meta.get('seed_types', ['note'])
+
+ # Read all classifications
+ classifications = read_classifications(swarm_dir)
+ if not classifications:
+ print("No classifications found. Run Wave 1 agents first.", file=sys.stderr)
+ return {}
+
+ # Count frequencies
+ frequencies = count_frequencies(classifications, seed_types)
+
+ # Load optional inputs
+ discovery = None
+ if discovery_path and discovery_path.exists():
+ discovery = json.loads(discovery_path.read_text())
+
+ draft_schema = None
+ if draft_schema_path and draft_schema_path.exists():
+ draft_schema = json.loads(draft_schema_path.read_text())
+
+ # Build consolidation
+ consolidation = {
+ 'created_at': datetime.now().isoformat(),
+ 'total_classified': frequencies['total_classified'],
+ 'type_frequency': frequencies['type_frequency'],
+ 'domain_frequency': frequencies['domain_frequency'],
+ 'confidence_distribution': frequencies['confidence_distribution'],
+ 'novel_types': frequencies['novel_types'],
+ 'seed_types': seed_types,
+ 'classifications': classifications,
+ 'discovery_meta': discovery.get('meta', {}) if discovery else {},
+ 'draft_schema': draft_schema,
+ 'wave2_prompt_template': WAVE2_PROMPT_TEMPLATE,
+ }
+
+ # Write consolidation.json
+ consolidation_path = swarm_dir / 'consolidation.json'
+ consolidation_path.write_text(
+ json.dumps(consolidation, ensure_ascii=False, indent=2) + '\n'
+ )
+
+ # Print summary
+ print(f"Consolidation: {frequencies['total_classified']} classifications")
+ print(f"Types: {frequencies['type_frequency']}")
+ print(f"Domains: {frequencies['domain_frequency']}")
+ print(f"Novel types: {frequencies['novel_types']}")
+ print(f"Output: {consolidation_path}")
+
+ return consolidation
+
+
+# ─── FINALIZE ─────────────────────────────────────────────
+
+def validate_schema(schema: dict) -> list[str]:
+ """Validate schema from Wave 2 agent. Returns list of errors (empty = valid)."""
+ errors = []
+
+ # Check required sections
+ # Note: region_fixes is optional (not in REQUIRED_SCHEMA_SECTIONS)
+ missing = REQUIRED_SCHEMA_SECTIONS - set(schema.keys())
+ if missing:
+ errors.append(f"Missing sections: {sorted(missing)}")
+
+ # Validate node_types
+ node_types = schema.get('node_types', {})
+ nt_clean = {k: v for k, v in node_types.items() if k != '_comment'}
+
+ if not nt_clean:
+ errors.append("node_types is empty")
+ elif len(nt_clean) > 15:
+ errors.append(f"Too many node_types: {len(nt_clean)} (max 15)")
+ elif len(nt_clean) < 3:
+ errors.append(f"Too few node_types: {len(nt_clean)} (min 3)")
+
+ for type_name, type_def in nt_clean.items():
+ if not isinstance(type_def, dict):
+ errors.append(f"node_type '{type_name}' is not a dict")
+ continue
+ for field in ('description', 'required', 'status'):
+ if field not in type_def:
+ errors.append(f"node_type '{type_name}' missing '{field}'")
+
+ # Validate type_aliases targets exist
+ aliases = schema.get('type_aliases', {})
+ for alias, target in aliases.items():
+ if alias == '_comment':
+ continue
+ if target not in nt_clean:
+ errors.append(f"type_alias '{alias}' → '{target}' but '{target}' not in node_types")
+
+ # Validate status_order covers all statuses used in node_types
+ status_order = schema.get('status_order', {})
+ so_clean = {k: v for k, v in status_order.items() if k != '_comment'}
+ all_statuses = set()
+ for td in nt_clean.values():
+ if isinstance(td, dict):
+ all_statuses.update(td.get('status', []))
+ missing_statuses = all_statuses - set(so_clean.keys())
+ if missing_statuses:
+ errors.append(f"status_order missing statuses: {sorted(missing_statuses)}")
+
+ # Validate decay config
+ decay = schema.get('decay', {})
+ if 'rate' not in decay or 'floor' not in decay:
+ errors.append("decay missing 'rate' or 'floor'")
+
+ return errors
+
+
+def finalize_schema(wave2_path: Path, output_path: Path | None = None) -> bool:
+ """Validate and write final schema.json. Returns True on success."""
+ if not wave2_path.exists():
+ print(f"File not found: {wave2_path}", file=sys.stderr)
+ return False
+
+ try:
+ schema = json.loads(wave2_path.read_text())
+ except json.JSONDecodeError as e:
+ print(f"Invalid JSON: {e}", file=sys.stderr)
+ return False
+
+ errors = validate_schema(schema)
+
+ if errors:
+ print("Schema validation FAILED:", file=sys.stderr)
+ for err in errors:
+ print(f" - {err}", file=sys.stderr)
+
+ # Save invalid draft for debugging
+ invalid_path = wave2_path.parent / 'schema-draft-invalid.json'
+ invalid_path.write_text(json.dumps(schema, ensure_ascii=False, indent=2) + '\n')
+ print(f"Raw draft saved to: {invalid_path}", file=sys.stderr)
+ return False
+
+ # Clean up _comment fields from output
+ output = json.dumps(schema, ensure_ascii=False, indent=2) + '\n'
+
+ if output_path:
+ output_path.write_text(output)
+ print(f"Schema written to {output_path}")
+ else:
+ sys.stdout.write(output)
+
+ node_count = len({k for k in schema.get('node_types', {}) if k != '_comment'})
+ print(f"Valid schema: {node_count} node types", file=sys.stderr)
+ return True
+
+
+# ─── MAIN ─────────────────────────────────────────────────
+
+def main():
+ args = sys.argv[1:]
+ if len(args) < 2 or args[0] in ('-h', '--help'):
+ print("Usage:", file=sys.stderr)
+ print(" swarm_reduce.py prepare <vault-dir> [discovery.json] [draft-schema.json]",
+ file=sys.stderr)
+ print(" swarm_reduce.py finalize <wave2-output.json> [output-schema.json]",
+ file=sys.stderr)
+ sys.exit(1)
+
+ command = args[0]
+
+ if command == 'prepare':
+ vault_dir = Path(args[1])
+ if not vault_dir.is_dir():
+ print(f"Not a directory: {vault_dir}", file=sys.stderr)
+ sys.exit(1)
+ discovery_path = Path(args[2]) if len(args) > 2 else None
+ draft_path = Path(args[3]) if len(args) > 3 else None
+ result = prepare_consolidation(vault_dir, discovery_path, draft_path)
+ if not result:
+ sys.exit(1)
+
+ elif command == 'finalize':
+ wave2_path = Path(args[1])
+ output_path = Path(args[2]) if len(args) > 2 else None
+ ok = finalize_schema(wave2_path, output_path)
+ sys.exit(0 if ok else 1)
+
+ else:
+ print(f"Unknown command: {command}", file=sys.stderr)
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/vault/.claude/skills/autograph/tests/test_autograph.py b/vault/.claude/skills/autograph/tests/test_autograph.py
new file mode 100644
index 0000000..b014916
--- /dev/null
+++ b/vault/.claude/skills/autograph/tests/test_autograph.py
@@ -0,0 +1,1512 @@
+#!/usr/bin/env python3
+"""
+autograph self-contained tests — uses temp fixtures, no real vault needed.
+Runs ~40 tests covering common.py functions + all script CLIs + edge cases.
+
+Usage: python3 test_autograph.py
+ (no arguments required, works from any directory)
+"""
+
+import sys
+import os
+import json
+import shutil
+import tempfile
+import subprocess
+from pathlib import Path
+from datetime import date, timedelta
+
+# ─── SETUP ────────────────────────────────────────────────
+SCRIPTS_DIR = Path(__file__).resolve().parent.parent / 'scripts'
+sys.path.insert(0, str(SCRIPTS_DIR))
+
+PASS = 0
+FAIL = 0
+
+
+def test(name: str, condition: bool, detail: str = ""):
+ global PASS, FAIL
+ if condition:
+ PASS += 1
+ print(f" PASS {name}")
+ else:
+ FAIL += 1
+ print(f" FAIL {name}: {detail}")
+
+
+def run(cmd: list, cwd: str = None) -> tuple:
+ r = subprocess.run(cmd, capture_output=True, text=True, cwd=cwd, timeout=60)
+ return r.returncode, r.stdout, r.stderr
+
+
+# ─── FIXTURES ─────────────────────────────────────────────
+
+SCHEMA = {
+ "node_types": {
+ "note": {
+ "description": "Knowledge note",
+ "required": ["description", "tags"],
+ "status": ["active", "draft", "archived"]
+ },
+ "contact": {
+ "description": "Person",
+ "required": ["description", "tags", "status"],
+ "status": ["active", "inactive"]
+ },
+ "project": {
+ "description": "Project with deliverables",
+ "required": ["description", "tags", "status"],
+ "status": ["active", "done", "paused", "cancelled"]
+ },
+ "lead": {
+ "description": "Sales lead",
+ "required": ["description", "status"],
+ "status": ["prospect", "negotiation", "won", "lost"]
+ }
+ },
+ "type_aliases": {
+ "crm": "contact",
+ "person": "contact",
+ "idea": "note"
+ },
+ "field_fixes": {
+ "status": {
+ "actve": "active",
+ "inactiv": "inactive"
+ },
+ "priority": {
+ "hi": "high",
+ "lo": "low"
+ }
+ },
+ "region_fixes": {
+ "KZ ": "KZ",
+ "kz": "KZ"
+ },
+ "domain_inference": {
+ "projects/": "work",
+ "personal/": "personal",
+ "knowledge/": "knowledge",
+ "contacts/": "crm"
+ },
+ "path_type_hints": {
+ "_comment": "folder substring -> type name",
+ "leads/": "lead",
+ "contacts/": "contact",
+ "people/": "contact"
+ },
+ "status_order": {
+ "_comment": "sort for MOC",
+ "active": 0,
+ "prospect": 1,
+ "done": 8,
+ "draft": 9
+ },
+ "status_defaults": {
+ "_comment": "defaults",
+ "default": "active"
+ },
+ "richness_fields": {
+ "_comment": "for dedup",
+ "bonus_fields": ["telegram", "email", "company"]
+ },
+ "entity_extraction": {
+ "_comment": "for daily.py",
+ "noise_words": ["TODO", "FIX", "Score"]
+ },
+ "decay": {
+ "rate": 0.015,
+ "floor": 0.1,
+ "tiers": {
+ "active": 7,
+ "warm": 21,
+ "cold": 60
+ }
+ },
+ "ignore_tags": ["imported"]
+}
+
+# Markdown fixtures: (relative_path, content)
+VAULT_FILES = {
+ "projects/alpha.md": (
+ "---\n"
+ "type: project\n"
+ "status: active\n"
+ "domain: work\n"
+ "tags: [dev, ai]\n"
+ "description: Alpha project for AI platform\n"
+ "priority: high\n"
+ "tier: active\n"
+ "relevance: 1.0\n"
+ f"last_accessed: {date.today().isoformat()}\n"
+ "---\n"
+ "# Alpha Project\n\n"
+ "Building an AI platform. See [[contacts/bob]] and [[knowledge/ml-basics]].\n\n"
+ "## Roadmap\n- Phase 1\n- Phase 2\n"
+ ),
+ "projects/beta.md": (
+ "---\n"
+ "type: project\n"
+ "status: draft\n"
+ "domain: work\n"
+ "tags: [design]\n"
+ "description: Beta design sprint\n"
+ "tier: warm\n"
+ "relevance: 0.7\n"
+ f"last_accessed: {(date.today() - timedelta(days=10)).isoformat()}\n"
+ "---\n"
+ "# Beta Sprint\n\n"
+ "Design phase. Related to [[projects/alpha|Alpha]].\n"
+ ),
+ "contacts/bob.md": (
+ "---\n"
+ "type: contact\n"
+ "status: active\n"
+ "domain: crm\n"
+ "tags: [partner, dev]\n"
+ "description: Bob Smith, lead developer\n"
+ "telegram: @bobdev\n"
+ "email: [email protected]\n"
+ "company: DevCorp\n"
+ "tier: active\n"
+ "relevance: 1.0\n"
+ f"last_accessed: {date.today().isoformat()}\n"
+ "---\n"
+ "# Bob Smith\n\n"
+ "Key partner. Works on [[projects/alpha]] and [[projects/beta]].\n"
+ ),
+ "contacts/alice.md": (
+ "---\n"
+ "type: contact\n"
+ "status: active\n"
+ "domain: crm\n"
+ "tags: [client]\n"
+ "description: Alice Johnson, marketing lead\n"
+ "tier: warm\n"
+ "relevance: 0.8\n"
+ f"last_accessed: {(date.today() - timedelta(days=15)).isoformat()}\n"
+ "---\n"
+ "# Alice Johnson\n\n"
+ "Client contact. Involved in [[projects/beta]].\n"
+ ),
+ "knowledge/ml-basics.md": (
+ "---\n"
+ "type: note\n"
+ "status: active\n"
+ "domain: knowledge\n"
+ "tags: [ai, ml, learning]\n"
+ "description: Machine learning fundamentals\n"
+ "tier: warm\n"
+ "relevance: 0.6\n"
+ f"last_accessed: {(date.today() - timedelta(days=20)).isoformat()}\n"
+ "---\n"
+ "# ML Basics\n\n"
+ "Neural networks, transformers, etc.\n\n"
+ "## Key Concepts\n- Backpropagation\n- Attention\n"
+ ),
+ "knowledge/python-tips.md": (
+ "---\n"
+ "type: note\n"
+ "status: draft\n"
+ "domain: knowledge\n"
+ "tags: [python, dev]\n"
+ "description: Useful Python patterns\n"
+ "tier: cold\n"
+ "relevance: 0.3\n"
+ f"last_accessed: {(date.today() - timedelta(days=45)).isoformat()}\n"
+ "---\n"
+ "# Python Tips\n\n"
+ "Dataclasses, pattern matching, etc.\n"
+ ),
+ "personal/journal.md": (
+ "---\n"
+ "type: note\n"
+ "status: active\n"
+ "domain: personal\n"
+ "tags: [journal]\n"
+ "description: Daily journal entry\n"
+ "tier: archive\n"
+ "relevance: 0.1\n"
+ f"last_accessed: {(date.today() - timedelta(days=100)).isoformat()}\n"
+ "---\n"
+ "# Journal\n\n"
+ "Today I worked on [[projects/alpha]] and talked to [[contacts/alice]].\n"
+ ),
+ "no-frontmatter.md": (
+ "# Just a Note\n\n"
+ "This file has no YAML frontmatter at all.\n"
+ "Links to [[knowledge/ml-basics]].\n"
+ ),
+ "broken-frontmatter.md": (
+ "---\n"
+ "type:\n"
+ "tags: [broken\n"
+ "---\n"
+ "# Broken FM\n\n"
+ "The frontmatter above is malformed.\n"
+ ),
+ # Duplicate slug (same stem as contacts/bob.md)
+ "personal/bob.md": (
+ "---\n"
+ "type: note\n"
+ "status: draft\n"
+ "tags: [personal]\n"
+ "description: Personal notes about Bob\n"
+ "---\n"
+ "# Bob\n\n"
+ "Random notes.\n"
+ ),
+}
+
+DAILY_FILE = {
+ "2026-03-01.md": (
+ "# Memory 2026-03-01\n\n"
+ "Met with @alice_m and @bob_dev today.\n\n"
+ "**Сергей Иванов** presented the new roadmap.\n\n"
+ "Discussed [[projects/alpha]] budget: $5,000.\n\n"
+ "Decided: approved the Q2 plan.\n\n"
+ "Events: launched v2.0\n"
+ )
+}
+
+
+def create_vault(base_dir: Path) -> Path:
+ """Create a temporary vault with test fixtures."""
+ vault = base_dir / "test-vault"
+ vault.mkdir(parents=True, exist_ok=True)
+ for rel, content in VAULT_FILES.items():
+ fp = vault / rel
+ fp.parent.mkdir(parents=True, exist_ok=True)
+ fp.write_text(content)
+ return vault
+
+
+def create_schema(base_dir: Path) -> Path:
+ """Write test schema.json."""
+ sp = base_dir / "schema.json"
+ sp.write_text(json.dumps(SCHEMA, indent=2, ensure_ascii=False))
+ return sp
+
+
+def create_daily_dir(base_dir: Path) -> Path:
+ """Write daily memory files."""
+ mem = base_dir / "memory"
+ mem.mkdir(parents=True, exist_ok=True)
+ for name, content in DAILY_FILE.items():
+ (mem / name).write_text(content)
+ return mem
+
+
+# ─── MAIN ─────────────────────────────────────────────────
+def main():
+ tmp = Path(tempfile.mkdtemp(prefix="autograph_test_"))
+ try:
+ vault_dir = create_vault(tmp)
+ schema_path = create_schema(tmp)
+ daily_dir = create_daily_dir(tmp)
+
+ # Clear schema cache before tests (common.py caches schemas)
+ from common import _schema_cache
+ _schema_cache.clear()
+
+ print(f"\n{'='*60}")
+ print(f" AUTOGRAPH SELF-CONTAINED TESTS")
+ print(f" vault: {vault_dir}")
+ print(f" schema: {schema_path}")
+ print(f" tmp: {tmp}")
+ print(f"{'='*60}\n")
+
+ # ═══════════════════════════════════════════════════════
+ # 1. common.py — pure function tests
+ # ═══════════════════════════════════════════════════════
+ print("--- common.py ---")
+ from common import (
+ load_schema, parse_frontmatter, walk_vault, infer_domain,
+ infer_type, calc_relevance, calc_tier, days_since,
+ extract_wikilinks, IGNORE_DIRS, write_frontmatter, format_field,
+ build_link_index, resolve_link_target, collect_duplicate_groups, is_hub_path
+ )
+
+ # 1.1 schema loading
+ schema = load_schema(schema_path)
+ test("load_schema returns dict", isinstance(schema, dict))
+ test("schema has node_types", 'node_types' in schema)
+ test("schema has decay config", 'decay' in schema)
+ test("schema has domain_inference", 'domain_inference' in schema)
+
+ # 1.2 IGNORE_DIRS
+ test("IGNORE_DIRS is frozenset", isinstance(IGNORE_DIRS, frozenset))
+ test("IGNORE_DIRS contains .obsidian", '.obsidian' in IGNORE_DIRS)
+
+ # 1.3 parse_frontmatter — valid
+ fm, body, lines = parse_frontmatter(
+ "---\ntype: crm\nstatus: active\ntags: [a, b, c]\n---\n# Hello\nBody"
+ )
+ test("parse_fm extracts type", fm.get('type') == 'crm')
+ test("parse_fm extracts tags list", fm.get('tags') == ['a', 'b', 'c'])
+ test("parse_fm body contains text", 'Hello' in body)
+
+ # 1.4 parse_frontmatter — no frontmatter
+ fm2, body2, _ = parse_frontmatter("# Just markdown\nNo frontmatter here")
+ test("parse_fm returns None for no-fm", fm2 is None)
+ test("parse_fm returns full content as body", 'Just markdown' in body2)
+
+ # 1.5 parse_frontmatter — empty value
+ fm3, _, _ = parse_frontmatter("---\ntype:\nstatus: active\n---\nBody")
+ test("parse_fm handles empty value", fm3.get('type') == '')
+
+ # 1.6 extract_wikilinks
+ links = extract_wikilinks("See [[foo/bar|Foo Bar]] and [[baz]] text")
+ test("extract_wikilinks finds 2 links", len(links) == 2)
+ test("extract_wikilinks parses alias", links[0] == ('foo/bar', 'Foo Bar'))
+ test("extract_wikilinks plain link", links[1] == ('baz', 'baz'))
+
+ # 1.7 extract_wikilinks — no links
+ test("extract_wikilinks empty on no links", extract_wikilinks("no links here") == [])
+
+ # 1.8 infer_domain
+ test("infer_domain projects/ -> work",
+ infer_domain("projects/alpha.md", schema) == "work")
+ test("infer_domain personal/ -> personal",
+ infer_domain("personal/journal.md", schema) == "personal")
+ test("infer_domain unknown -> personal (default)",
+ infer_domain("random/file.md", schema) == "personal")
+
+ # 1.9 infer_type
+ test("infer_type note/ folder -> note",
+ infer_type("knowledge/note/something.md", schema) == "note")
+ test("infer_type contacts/ -> contact (via path_type_hints)",
+ infer_type("contacts/someone.md", schema) == "contact")
+
+ # 1.10 calc_relevance
+ test("calc_relevance day 0 = 1.0", calc_relevance(0, schema) == 1.0)
+ test("calc_relevance day 10 = 0.85", calc_relevance(10, schema) == 0.85)
+ test("calc_relevance day 60 = floor 0.1", calc_relevance(60, schema) == 0.1)
+ test("calc_relevance day 100 = floor 0.1", calc_relevance(100, schema) == 0.1)
+
+ # 1.11 calc_tier
+ test("calc_tier day 3 = active", calc_tier(3, schema) == 'active')
+ test("calc_tier day 7 = active (boundary)", calc_tier(7, schema) == 'active')
+ test("calc_tier day 15 = warm", calc_tier(15, schema) == 'warm')
+ test("calc_tier day 21 = warm (boundary)", calc_tier(21, schema) == 'warm')
+ test("calc_tier day 40 = cold", calc_tier(40, schema) == 'cold')
+ test("calc_tier day 90 = archive", calc_tier(90, schema) == 'archive')
+ test("calc_tier core stays core", calc_tier(999, schema, 'core') == 'core')
+
+ # 1.12 days_since
+ today = date.today()
+ test("days_since today = 0", days_since(today.isoformat()) == 0)
+ test("days_since yesterday = 1",
+ days_since((today - timedelta(days=1)).isoformat()) == 1)
+ test("days_since empty = 999", days_since('') == 999)
+ test("days_since garbage = 999", days_since('not-a-date') == 999)
+ test("days_since None = 999", days_since(None) == 999)
+
+ # 1.13 walk_vault
+ files = walk_vault(vault_dir)
+ test("walk_vault finds files", len(files) == len(VAULT_FILES))
+ # Should skip .obsidian, .git etc
+ obs_dir = vault_dir / '.obsidian'
+ obs_dir.mkdir(exist_ok=True)
+ (obs_dir / 'hidden.md').write_text("hidden")
+ files2 = walk_vault(vault_dir)
+ test("walk_vault ignores .obsidian", len(files2) == len(VAULT_FILES))
+
+ # 1.14 write_frontmatter + format_field
+ test("format_field list", format_field('tags', ['a', 'b']) == 'tags: [a, b]')
+ test("format_field string", format_field('type', 'note') == 'type: note')
+ test("format_field int", format_field('count', 42) == 'count: 42')
+ test("format_field float", format_field('relevance', 0.85) == 'relevance: 0.85')
+
+ # 1.15 write_frontmatter multiline roundtrip (no duplication)
+ multiline_fm = "---\ntype: note\ndescription: >-\n Long description that\n spans multiple lines\ntags: [ai, test]\n---\n# Body\n"
+ fm_ml, body_ml, orig_ml = parse_frontmatter(multiline_fm)
+ test("multiline parse: description joined",
+ fm_ml.get('description') == 'Long description that spans multiple lines',
+ f"got: {fm_ml.get('description')}")
+ block_list_fm = "---\ntype: note\ntags:\n - ai\n - test\nstatus: active\n---\n# Body\n"
+ fm_list, _, _ = parse_frontmatter(block_list_fm)
+ test("block list parse: tags extracted",
+ fm_list.get('tags') == ['ai', 'test'],
+ f"got: {fm_list.get('tags')!r}")
+ # Roundtrip: write back same fields → no duplication
+ rebuilt_ml = write_frontmatter(fm_ml, orig_ml)
+ fm_rt, _, _ = parse_frontmatter(f"---\n{rebuilt_ml}\n---\n")
+ test("multiline roundtrip: no duplication",
+ fm_rt.get('description') == fm_ml.get('description'),
+ f"original={fm_ml.get('description')!r}, roundtrip={fm_rt.get('description')!r}")
+ # Second roundtrip — should be stable
+ rebuilt_ml2 = write_frontmatter(fm_rt, rebuilt_ml.split('\n'))
+ fm_rt2, _, _ = parse_frontmatter(f"---\n{rebuilt_ml2}\n---\n")
+ test("multiline double roundtrip: stable",
+ fm_rt2.get('description') == fm_ml.get('description'),
+ f"got: {fm_rt2.get('description')!r}")
+ # Literal block |- continuation lines skipped when key rewritten
+ literal_fm = "---\ntype: note\ndescription: |-\n Line one\n Line two\ntags: [test]\n---\n"
+ fm_lit, _, orig_lit = parse_frontmatter(literal_fm)
+ rebuilt_lit = write_frontmatter(fm_lit, orig_lit)
+ # Key point: continuation lines should NOT appear as extra lines
+ continuation_leaked = 'Line one' in rebuilt_lit and rebuilt_lit.count('Line one') > 1
+ test("literal |- rewrite: no leaked continuation lines",
+ not continuation_leaked,
+ f"got: {rebuilt_lit}")
+ # Untouched multiline key preserved as-is
+ partial_fields = {'tags': ['ai', 'updated']} # only update tags, not description
+ rebuilt_partial = write_frontmatter(partial_fields, orig_ml)
+ test("untouched multiline key preserved",
+ 'spans multiple lines' in rebuilt_partial,
+ f"got: {rebuilt_partial}")
+
+ # 1.16 deterministic link resolver
+ resolver_vault = tmp / 'resolver-vault'
+ (resolver_vault / 'docs/cards').mkdir(parents=True, exist_ok=True)
+ (resolver_vault / 'crm').mkdir(parents=True, exist_ok=True)
+ (resolver_vault / 'misc').mkdir(parents=True, exist_ok=True)
+ (resolver_vault / 'docs/cards/visa.md').write_text("# Visa card\n")
+ (resolver_vault / 'crm/visa.md').write_text("# Visa contact\n")
+ (resolver_vault / 'misc/visa-guide.md').write_text("# Visa guide\n")
+ resolver_index = build_link_index(resolver_vault)
+ resolved_exact, reason_exact = resolve_link_target('docs/cards/visa', resolver_index)
+ test("resolve_link_target exact path",
+ resolved_exact == 'docs/cards/visa' and reason_exact == 'exact',
+ f"got: {(resolved_exact, reason_exact)}")
+ resolved_suffix, reason_suffix = resolve_link_target('cards/visa', resolver_index)
+ test("resolve_link_target unique suffix",
+ resolved_suffix == 'docs/cards/visa' and reason_suffix == 'unique_suffix',
+ f"got: {(resolved_suffix, reason_suffix)}")
+ resolved_ambiguous, reason_ambiguous = resolve_link_target('visa', resolver_index)
+ test("resolve_link_target blocks ambiguous stem",
+ resolved_ambiguous is None and reason_ambiguous == 'ambiguous_stem',
+ f"got: {(resolved_ambiguous, reason_ambiguous)}")
+ resolved_unique_stem, reason_unique_stem = resolve_link_target('visa-guide', resolver_index)
+ test("resolve_link_target unique stem",
+ resolved_unique_stem == 'misc/visa-guide' and reason_unique_stem == 'unique_stem',
+ f"got: {(resolved_unique_stem, reason_unique_stem)}")
+
+ # 1.17 duplicate grouping only merges compatible cards
+ dedup_vault = tmp / 'dedup-vault'
+ (dedup_vault / 'knowledge/notes').mkdir(parents=True, exist_ok=True)
+ (dedup_vault / 'contacts').mkdir(parents=True, exist_ok=True)
+ (dedup_vault / 'personal').mkdir(parents=True, exist_ok=True)
+ (dedup_vault / 'knowledge/foo.md').write_text(
+ "---\ntype: note\ndomain: knowledge\ndescription: Foo\n---\n# Foo\n"
+ )
+ (dedup_vault / 'knowledge/notes/foo.md').write_text(
+ "---\ntype: note\ndomain: knowledge\ndescription: Foo note\n---\n# Foo note\n"
+ )
+ (dedup_vault / 'contacts/bob.md').write_text(
+ "---\ntype: contact\ndomain: crm\ndescription: Bob contact\n---\n# Bob\n"
+ )
+ (dedup_vault / 'personal/bob.md').write_text(
+ "---\ntype: note\ndomain: personal\ndescription: Bob note\n---\n# Bob note\n"
+ )
+ duplicate_groups = collect_duplicate_groups(dedup_vault, schema)
+ test("collect_duplicate_groups keeps same type/domain duplicates",
+ duplicate_groups.get(('foo', 'knowledge', 'note')) == ['knowledge/notes/foo.md', 'knowledge/foo.md'] or
+ duplicate_groups.get(('foo', 'knowledge', 'note')) == ['knowledge/foo.md', 'knowledge/notes/foo.md'],
+ f"got: {duplicate_groups}")
+ test("collect_duplicate_groups skips cross-domain homonyms",
+ all(key[0] != 'bob' for key in duplicate_groups),
+ f"got: {duplicate_groups}")
+ test("is_hub_path detects nested _index", is_hub_path('foo/_index'))
+ test("is_hub_path detects nested MEMORY", is_hub_path('agents/x/MEMORY'))
+
+ # ═══════════════════════════════════════════════════════
+ # 2. Script CLI tests (subprocess against temp vault)
+ # ═══════════════════════════════════════════════════════
+
+ py = sys.executable # use the same python that runs this test
+
+ # --- graph.py ---
+ print("\n--- graph.py ---")
+ _schema_cache.clear()
+ code, out, err = run([py, str(SCRIPTS_DIR / 'graph.py'), 'health',
+ str(vault_dir), str(schema_path)])
+ test("graph health exits 0", code == 0, err[:300])
+ test("graph health shows total files",
+ 'Total files:' in out, out[:200])
+ # Should find our 10 files
+ file_count_str = ''
+ if 'Total files:' in out:
+ file_count_str = out.split('Total files:')[1].split('\n')[0].strip()
+ test("graph health correct file count",
+ file_count_str == str(len(VAULT_FILES)),
+ f"expected {len(VAULT_FILES)}, got '{file_count_str}'")
+ test("graph health shows Health Score", 'Health Score' in out)
+
+ # graph orphans
+ code, out, _ = run([py, str(SCRIPTS_DIR / 'graph.py'), 'orphans',
+ str(vault_dir), str(schema_path)])
+ test("graph orphans exits 0", code == 0)
+ test("graph orphans shows list", 'Orphans:' in out)
+
+ # graph backlinks
+ code, out, _ = run([py, str(SCRIPTS_DIR / 'graph.py'), 'backlinks',
+ str(vault_dir), 'contacts/bob', str(schema_path)])
+ test("graph backlinks exits 0", code == 0)
+ test("graph backlinks finds links", 'Backlinks' in out)
+
+ # --- moc.py ---
+ print("\n--- moc.py ---")
+ _schema_cache.clear()
+ code, out, err = run([py, str(SCRIPTS_DIR / 'moc.py'), 'generate',
+ str(vault_dir), str(schema_path)])
+ test("moc generate exits 0", code == 0, err[:300])
+ test("moc generates wikilinks", 'wikilinks' in out.lower(), out[:200])
+
+ # Check MOC files were created
+ moc_dir = vault_dir / 'MOC'
+ moc_files = list(moc_dir.glob('MOC-*.md')) if moc_dir.exists() else []
+ test("moc creates MOC files", len(moc_files) > 0,
+ f"found {len(moc_files)}")
+
+ # --- engine.py ---
+ print("\n--- engine.py ---")
+ _schema_cache.clear()
+ code, out, err = run([py, str(SCRIPTS_DIR / 'engine.py'), 'stats',
+ str(vault_dir), str(schema_path)])
+ test("engine stats exits 0", code == 0, err[:300])
+ test("engine stats shows total cards", 'total cards:' in out, out[:200])
+ test("engine stats shows decay rate", '0.015' in out)
+ test("engine stats shows tier distribution", 'tier distribution' in out)
+
+ # engine creative
+ _schema_cache.clear()
+ code, out, err = run([py, str(SCRIPTS_DIR / 'engine.py'), 'creative', '2',
+ str(vault_dir), str(schema_path)])
+ test("engine creative exits 0", code == 0, err[:300])
+
+ # --- enforce.py ---
+ print("\n--- enforce.py ---")
+ _schema_cache.clear()
+ code, out, err = run([py, str(SCRIPTS_DIR / 'enforce.py'),
+ str(vault_dir), str(schema_path)])
+ test("enforce exits 0", code == 0, err[:300])
+ test("enforce shows compliance score", 'SCHEMA COMPLIANCE' in out, out[:300])
+
+ # --- discover.py ---
+ print("\n--- discover.py ---")
+ _schema_cache.clear()
+ code, out, err = run([py, str(SCRIPTS_DIR / 'discover.py'), str(vault_dir)])
+ test("discover exits 0", code == 0, err[:300])
+ try:
+ disc = json.loads(out)
+ test("discover outputs valid JSON", True)
+ # MOC files may have been created by moc.py above, so count >= original
+ test("discover finds files >= fixture count",
+ disc['meta']['total_files'] >= len(VAULT_FILES),
+ f"expected >= {len(VAULT_FILES)}, got {disc['meta'].get('total_files')}")
+ except Exception as e:
+ test("discover outputs valid JSON", False, str(e)[:200])
+
+ # --- dedup.py ---
+ print("\n--- dedup.py ---")
+ _schema_cache.clear()
+ code, out, err = run([py, str(SCRIPTS_DIR / 'dedup.py'), str(vault_dir)])
+ test("dedup exits 0", code == 0, err[:300])
+ # We have bob.md in two places, so duplicates should be found
+ test("dedup finds duplicates",
+ 'Duplicate groups' in out or 'No duplicates' in out,
+ out[:200])
+
+ policy_vault = tmp / 'policy-vault'
+ (policy_vault / 'pu').mkdir(parents=True, exist_ok=True)
+ (policy_vault / 'sample/contacts').mkdir(parents=True, exist_ok=True)
+ (policy_vault / 'sample/clients').mkdir(parents=True, exist_ok=True)
+ (policy_vault / 'crm/sample').mkdir(parents=True, exist_ok=True)
+ (policy_vault / 'pu/alex.md').write_text(
+ "---\ntype: power_user\ndomain: sample\nstatus: active\n---\n# Alex\n")
+ (policy_vault / 'sample/contacts/alex.md').write_text(
+ "---\ntype: contact\ndomain: sample\nstatus: active\n---\n# Alex\n")
+ (policy_vault / 'sample/clients/acme.md').write_text(
+ "---\ntype: client\ndomain: sample\nstatus: active\n---\n# Acme\n")
+ (policy_vault / 'sample/contacts/acme.md').write_text(
+ "---\ntype: contact\ndomain: sample\nstatus: active\n---\n# Acme\n")
+ policy_schema = dict(SCHEMA)
+ policy_schema['node_types'] = dict(SCHEMA['node_types'])
+ policy_schema['node_types']['power_user'] = {
+ "description": "Power user", "required": ["description"], "status": ["active"]
+ }
+ policy_schema['node_types']['client'] = {
+ "description": "Client", "required": ["description"], "status": ["active"]
+ }
+ policy_schema['node_types']['crm'] = {
+ "description": "CRM overlay", "required": ["description"], "status": ["active"]
+ }
+ policy_schema['dedup_policy'] = {
+ "canonical_priority": [
+ "pu/",
+ "sample/clients/",
+ "sample/contacts/",
+ "crm/sample/"
+ ],
+ "path_rules": [
+ {"prefix": "pu/", "domain": "sample-pu", "type": "power_user", "kind": "power_user"},
+ {"prefix": "sample/clients/", "domain": "sample-clients", "type": "client", "kind": "client"},
+ {"prefix": "sample/contacts/", "domain": "sample-contacts", "type": "contact", "kind": "contact"},
+ {"prefix": "crm/sample/", "domain": "sample-crm", "type": "crm", "kind": "crm"}
+ ]
+ }
+ policy_schema_path = tmp / 'policy-schema.json'
+ policy_schema_path.write_text(json.dumps(policy_schema))
+ policy_manifest = tmp / 'policy-manifest.json'
+ code, out, err = run([py, str(SCRIPTS_DIR / 'dedup.py'), str(policy_vault),
+ str(policy_schema_path), '--dry-run', '--manifest',
+ str(policy_manifest), '--verbose'])
+ test("dedup policy manifest exits 0", code == 0, err[:300])
+ test("dedup policy manifest writes file", policy_manifest.exists())
+ test("dedup policy does not move PU/client into contacts",
+ 'MOVE:' not in out and 'policy' in policy_manifest.read_text(),
+ out[:300])
+ manifest_data = json.loads(policy_manifest.read_text())
+ actions = {c['slug']: c['action'] for c in manifest_data['clusters']}
+ test("dedup policy keeps layered PU/contact", actions.get('alex') == 'keep_linked_layers', str(actions))
+ test("dedup policy keeps client/contact layers", actions.get('acme') == 'keep_linked_layers', str(actions))
+
+ # --- daily.py ---
+ print("\n--- daily.py ---")
+ _schema_cache.clear()
+ code, out, err = run([py, str(SCRIPTS_DIR / 'daily.py'), 'extract',
+ str(daily_dir), str(vault_dir), '2026-03-01'])
+ test("daily extract exits 0", code == 0, err[:300])
+ test("daily extract finds people", '"people"' in out, out[:200])
+ try:
+ daily_summary = json.loads(out)
+ test("daily extract summary includes linked_entities",
+ 'linked_entities' in daily_summary,
+ str(daily_summary))
+ except Exception as e:
+ test("daily extract summary includes linked_entities", False, str(e)[:200])
+
+ # ═══════════════════════════════════════════════════════
+ # 3. swarm_prepare.py + swarm_reduce.py tests
+ # ═══════════════════════════════════════════════════════
+
+ # --- swarm_prepare.py ---
+ print("\n--- swarm_prepare.py ---")
+ _schema_cache.clear()
+
+ # First run discover to get discovery JSON
+ code_disc, out_disc, _ = run([py, str(SCRIPTS_DIR / 'discover.py'), str(vault_dir)])
+ disc_json_path = tmp / 'discovery.json'
+ disc_json_path.write_text(out_disc)
+
+ # 3.1 swarm_prepare exits 0 and produces manifests
+ code, out, err = run([py, str(SCRIPTS_DIR / 'swarm_prepare.py'),
+ str(vault_dir), str(disc_json_path), '--budget', '50000'])
+ test("swarm_prepare exits 0", code == 0, err[:300])
+ test("swarm_prepare shows batch count", 'batch' in out.lower(), out[:200])
+
+ # Check manifests were created
+ manifests_dir = vault_dir / '.graph' / 'swarm' / 'manifests'
+ manifests = list(manifests_dir.glob('batch-*.json')) if manifests_dir.exists() else []
+ test("swarm_prepare creates manifests", len(manifests) > 0,
+ f"found {len(manifests)} manifests")
+
+ # 3.2 All files distributed (sum of batch file_counts = total files)
+ total_in_batches = 0
+ all_batch_files = set()
+ for mf in manifests:
+ manifest = json.loads(mf.read_text())
+ total_in_batches += manifest['file_count']
+ for f in manifest['files']:
+ all_batch_files.add(f)
+ # walk_vault finds all files (may include MOC files from moc.py test above)
+ from common import walk_vault as wv, rel_path as rp
+ vault_files_actual = [rp(f, vault_dir) for f in wv(vault_dir)]
+ test("swarm_prepare distributes all files",
+ total_in_batches == len(vault_files_actual),
+ f"batches have {total_in_batches}, vault has {len(vault_files_actual)}")
+
+ # 3.3 No file duplicated across batches
+ test("swarm_prepare no file duplicates",
+ len(all_batch_files) == total_in_batches,
+ f"unique={len(all_batch_files)}, total={total_in_batches}")
+
+ # 3.4 Budget respected per batch
+ budget = 50000
+ budget_ok = True
+ for mf in manifests:
+ manifest = json.loads(mf.read_text())
+ if manifest['estimated_tokens'] > budget and manifest['file_count'] > 1:
+ budget_ok = False
+ test("swarm_prepare respects token budget", budget_ok)
+
+ # 3.5 Seed types included
+ if manifests:
+ first_manifest = json.loads(manifests[0].read_text())
+ test("swarm_prepare includes seed_types",
+ len(first_manifest.get('seed_types', [])) > 0,
+ str(first_manifest.get('seed_types', [])))
+ else:
+ test("swarm_prepare includes seed_types", False, "no manifests")
+
+ # 3.6 swarm-meta.json exists
+ meta_path = vault_dir / '.graph' / 'swarm' / 'swarm-meta.json'
+ test("swarm_prepare creates swarm-meta.json", meta_path.exists())
+
+ # 3.7 Deterministic: two runs produce identical manifests
+ # Clean up and re-run
+ import shutil as _shutil
+ swarm_dir_1 = tmp / 'swarm-backup'
+ if manifests_dir.exists():
+ _shutil.copytree(manifests_dir, swarm_dir_1)
+ # Remove and re-run
+ _shutil.rmtree(vault_dir / '.graph' / 'swarm', ignore_errors=True)
+ code2, _, _ = run([py, str(SCRIPTS_DIR / 'swarm_prepare.py'),
+ str(vault_dir), str(disc_json_path), '--budget', '50000'])
+ manifests2 = sorted(manifests_dir.glob('batch-*.json')) if manifests_dir.exists() else []
+ manifests1 = sorted(swarm_dir_1.glob('batch-*.json')) if swarm_dir_1.exists() else []
+ deterministic = (len(manifests1) == len(manifests2))
+ if deterministic:
+ for m1, m2 in zip(manifests1, manifests2):
+ d1 = json.loads(m1.read_text())
+ d2 = json.loads(m2.read_text())
+ # Compare file lists (not timestamps)
+ if d1['files'] != d2['files']:
+ deterministic = False
+ break
+ test("swarm_prepare is deterministic", deterministic)
+
+ # 3.8 Empty vault
+ empty_swarm = tmp / 'empty-swarm-vault'
+ empty_swarm.mkdir(exist_ok=True)
+ code_e, out_e, err_e = run([py, str(SCRIPTS_DIR / 'swarm_prepare.py'),
+ str(empty_swarm)])
+ test("swarm_prepare handles empty vault", code_e == 0, err_e[:200])
+
+ # --- swarm_reduce.py ---
+ print("\n--- swarm_reduce.py ---")
+
+ # Create fake Wave 1 JSONL output for testing
+ classifications_dir = vault_dir / '.graph' / 'swarm' / 'classifications'
+ classifications_dir.mkdir(parents=True, exist_ok=True)
+ jsonl_lines = []
+ for rel_file in list(VAULT_FILES.keys())[:5]:
+ jsonl_lines.append(json.dumps({
+ "path": rel_file,
+ "proposed_type": "note",
+ "proposed_domain": "knowledge",
+ "summary": "Test file",
+ "seed_match": True,
+ "confidence": "high"
+ }))
+ for rel_file in list(VAULT_FILES.keys())[5:]:
+ jsonl_lines.append(json.dumps({
+ "path": rel_file,
+ "proposed_type": "contact",
+ "proposed_domain": "crm",
+ "summary": "Contact file",
+ "seed_match": True,
+ "confidence": "medium"
+ }))
+ (classifications_dir / 'batch-001.jsonl').write_text('\n'.join(jsonl_lines) + '\n')
+
+ # Add a malformed line
+ (classifications_dir / 'batch-002.jsonl').write_text(
+ '{"path":"ok.md","proposed_type":"note","proposed_domain":"personal","summary":"ok","seed_match":true,"confidence":"high"}\n'
+ 'THIS IS NOT JSON\n'
+ '{"no_path_field": true}\n'
+ )
+
+ # 3.9 swarm_reduce prepare reads JSONL and counts frequencies
+ code, out, err = run([py, str(SCRIPTS_DIR / 'swarm_reduce.py'), 'prepare',
+ str(vault_dir)])
+ test("swarm_reduce prepare exits 0", code == 0, err[:300])
+ test("swarm_reduce prepare shows types", 'note' in out, out[:300])
+
+ consolidation_path = vault_dir / '.graph' / 'swarm' / 'consolidation.json'
+ test("swarm_reduce prepare creates consolidation.json",
+ consolidation_path.exists())
+
+ if consolidation_path.exists():
+ consol = json.loads(consolidation_path.read_text())
+ test("swarm_reduce prepare has type_frequency",
+ 'note' in consol.get('type_frequency', {}),
+ str(consol.get('type_frequency', {})))
+ else:
+ test("swarm_reduce prepare has type_frequency", False, "no consolidation")
+
+ # 3.10 swarm_reduce prepare handles malformed lines (logged to stderr)
+ test("swarm_reduce prepare warns on malformed", 'malformed' in err.lower() or 'WARN' in err,
+ err[:200])
+
+ # 3.11 swarm_reduce finalize with valid schema → exits 0
+ valid_schema = {
+ "node_types": {
+ "note": {"description": "General note", "required": ["description", "tags"],
+ "status": ["active", "draft", "archived"]},
+ "contact": {"description": "Person", "required": ["description", "tags"],
+ "status": ["active", "inactive"]},
+ "project": {"description": "Project", "required": ["description", "status"],
+ "status": ["active", "done", "paused"]}
+ },
+ "type_aliases": {"person": "contact"},
+ "field_fixes": {},
+ "domain_inference": {"projects/": "work", "contacts/": "crm"},
+ "path_type_hints": {"contacts/": "contact"},
+ "status_order": {"active": 0, "draft": 1, "archived": 2, "inactive": 3, "done": 4, "paused": 5},
+ "status_defaults": {"default": "active"},
+ "richness_fields": {"bonus_fields": ["email", "telegram"]},
+ "entity_extraction": {"noise_words": ["TODO"]},
+ "decay": {"rate": 0.015, "floor": 0.1, "tiers": {"active": 7, "warm": 21, "cold": 60}},
+ "ignore_tags": []
+ }
+ valid_path = tmp / 'valid-wave2.json'
+ valid_path.write_text(json.dumps(valid_schema))
+ output_schema = tmp / 'output-schema.json'
+
+ code, out, err = run([py, str(SCRIPTS_DIR / 'swarm_reduce.py'), 'finalize',
+ str(valid_path), str(output_schema)])
+ test("swarm_reduce finalize valid exits 0", code == 0, err[:300])
+ test("swarm_reduce finalize writes output", output_schema.exists())
+
+ # 3.12 swarm_reduce finalize rejects missing sections → exits 1
+ bad_schema_missing = {"node_types": {"note": {"description": "x", "required": [], "status": ["active"]}}}
+ bad_path_missing = tmp / 'bad-missing.json'
+ bad_path_missing.write_text(json.dumps(bad_schema_missing))
+ code, _, err = run([py, str(SCRIPTS_DIR / 'swarm_reduce.py'), 'finalize',
+ str(bad_path_missing)])
+ test("swarm_reduce finalize rejects missing sections", code != 0,
+ err[:200])
+
+ # 3.13 swarm_reduce finalize rejects >15 node_types → exits 1
+ too_many_types = {f"type_{i}": {"description": f"Type {i}", "required": [], "status": ["active"]}
+ for i in range(16)}
+ bad_schema_many = dict(valid_schema)
+ bad_schema_many = json.loads(json.dumps(valid_schema)) # deep copy
+ bad_schema_many["node_types"] = too_many_types
+ bad_path_many = tmp / 'bad-many.json'
+ bad_path_many.write_text(json.dumps(bad_schema_many))
+ code, _, err = run([py, str(SCRIPTS_DIR / 'swarm_reduce.py'), 'finalize',
+ str(bad_path_many)])
+ test("swarm_reduce finalize rejects >15 types", code != 0,
+ err[:200])
+
+ # 3.14 swarm_reduce finalize checks alias targets exist
+ bad_schema_alias = json.loads(json.dumps(valid_schema))
+ bad_schema_alias["type_aliases"] = {"ghost": "nonexistent_type"}
+ bad_path_alias = tmp / 'bad-alias.json'
+ bad_path_alias.write_text(json.dumps(bad_schema_alias))
+ code, _, err = run([py, str(SCRIPTS_DIR / 'swarm_reduce.py'), 'finalize',
+ str(bad_path_alias)])
+ test("swarm_reduce finalize rejects bad alias target", code != 0,
+ err[:200])
+
+ # ═══════════════════════════════════════════════════════
+ # 4. Edge cases
+ # ═══════════════════════════════════════════════════════
+ print("\n--- edge cases ---")
+
+ # 3.1 Empty vault
+ empty_dir = tmp / "empty-vault"
+ empty_dir.mkdir(exist_ok=True)
+ _schema_cache.clear()
+
+ code, _, _ = run([py, str(SCRIPTS_DIR / 'graph.py'), 'health',
+ str(empty_dir), str(schema_path)])
+ test("graph handles empty vault", code == 0)
+
+ code, _, _ = run([py, str(SCRIPTS_DIR / 'moc.py'), 'generate',
+ str(empty_dir), str(schema_path)])
+ test("moc handles empty vault", code == 0)
+
+ code, _, _ = run([py, str(SCRIPTS_DIR / 'engine.py'), 'stats',
+ str(empty_dir), str(schema_path)])
+ test("engine handles empty vault", code == 0)
+
+ code, _, _ = run([py, str(SCRIPTS_DIR / 'dedup.py'), str(empty_dir)])
+ test("dedup handles empty vault", code == 0)
+
+ code, out, _ = run([py, str(SCRIPTS_DIR / 'discover.py'), str(empty_dir)])
+ test("discover handles empty vault", code == 0)
+ try:
+ disc_empty = json.loads(out)
+ test("discover empty vault returns 0 files",
+ disc_empty['meta']['total_files'] == 0)
+ except Exception:
+ test("discover empty vault returns 0 files", False, "bad JSON")
+
+ # 3.2 File without frontmatter (already in vault)
+ fm_none, body_none, _ = parse_frontmatter(VAULT_FILES["no-frontmatter.md"])
+ test("no-fm file: parse_fm returns None", fm_none is None)
+ test("no-fm file: body preserved", '# Just a Note' in body_none)
+
+ # 3.3 File with broken frontmatter
+ fm_broken, body_broken, _ = parse_frontmatter(VAULT_FILES["broken-frontmatter.md"])
+ test("broken-fm: parse_fm returns dict (not crash)",
+ fm_broken is not None or fm_broken is None)
+ # The parser should not crash; it may return partial data or None
+
+ # 3.4 No hardcoded domains/maps in scripts
+ # Note: common.py has get_domain_map() which is schema-driven, so we
+ # only check for DOMAIN_MAP (a hardcoded constant pattern)
+ for script_name in ['graph.py', 'moc.py', 'engine.py', 'common.py']:
+ src = (SCRIPTS_DIR / script_name).read_text()
+ test(f"{script_name} no hardcoded DOMAIN_MAP constant",
+ 'DOMAIN_MAP' not in src)
+
+ # ═══════════════════════════════════════════════════════
+ # 5. enrich.py tests (mock API)
+ # ═══════════════════════════════════════════════════════
+ print("\n--- enrich.py ---")
+ import enrich
+
+ # Mock API — returns canned results
+ _mock_call_count = 0
+ def mock_openrouter(messages, response_schema, model=enrich.DEFAULT_MODEL, schema_name="response"):
+ nonlocal _mock_call_count
+ _mock_call_count += 1
+ # Detect mode from schema_name or message content
+ user_msg = messages[-1]['content'] if messages else ''
+ if 'Classify' in user_msg or schema_name == 'tag_results':
+ # Tags mode — extract paths from user message
+ paths = []
+ for line in user_msg.split('\n'):
+ if line.startswith('### '):
+ paths.append(line[4:].strip())
+ return {"results": [{"path": p, "tags": ["ai", "test"]} for p in paths]}
+ elif schema_name == 'swarm_link_results':
+ # Swarm-links mode — return exact stems (some valid, some not)
+ paths = []
+ for line in user_msg.split('\n'):
+ if line.startswith('### '):
+ paths.append(line[4:].strip())
+ return {"results": [{"path": p, "links": ["alpha", "bob", "nonexistent-stem-xyz", Path(p).stem]} for p in paths]}
+ else:
+ # Links mode — extract paths, return suggestions
+ paths = []
+ for line in user_msg.split('\n'):
+ if line.startswith('### '):
+ paths.append(line[4:].strip())
+ return {"results": [{"path": p, "suggestions": ["ML Basics", "Alpha Project", "nonexistent note xyz"]} for p in paths]}
+
+ original_call = enrich.call_openrouter
+ enrich.call_openrouter = mock_openrouter
+
+ # 5.1 chunk_list
+ test("chunk_list normal", enrich.chunk_list([1,2,3,4,5], 2) == [[1,2],[3,4],[5]])
+ test("chunk_list empty", enrich.chunk_list([], 3) == [])
+ test("chunk_list smaller than chunk", enrich.chunk_list([1,2], 5) == [[1,2]])
+
+ # 5.8 collect_vault_tags
+ seed_tags = enrich.collect_vault_tags(vault_dir, schema)
+ test("collect_vault_tags returns list", isinstance(seed_tags, list))
+ test("collect_vault_tags finds tags", len(seed_tags) > 0)
+ test("collect_vault_tags excludes ignore_tags", 'imported' not in seed_tags)
+
+ # 5.9 build_tag_entries — filters files without tags
+ tag_entries = enrich.build_tag_entries(vault_dir, schema)
+ test("build_tag_entries finds untagged", len(tag_entries) > 0)
+ # Files with tags should be skipped (unless force)
+ tagged_paths = [e['path'] for e in tag_entries]
+ test("build_tag_entries skips tagged files",
+ 'projects/alpha.md' not in tagged_paths,
+ f"alpha.md should be skipped, got {tagged_paths}")
+
+ # 5.10 build_tag_entries --force includes all
+ force_entries = enrich.build_tag_entries(vault_dir, schema, force=True)
+ test("build_tag_entries force includes all",
+ len(force_entries) > len(tag_entries))
+
+ # 5.11 Tags dry run (no --apply) — results saved to disk
+ _mock_call_count = 0
+ enrich_tags_dir = vault_dir / '.graph' / 'enrich' / 'tags'
+ if enrich_tags_dir.exists():
+ shutil.rmtree(enrich_tags_dir)
+ enrich.cmd_tags(vault_dir, apply=False, budget=50000,
+ model="test-model", force=True, delay=0, workers=1)
+ tag_results = list(enrich_tags_dir.glob('batch-*-results.json'))
+ test("tags dry run creates result files", len(tag_results) > 0)
+ test("tags dry run called mock API", _mock_call_count > 0)
+
+ # 5.12 Tags --apply writes to files
+ # Add a file with no tags for apply test
+ test_apply_file = vault_dir / "apply-test.md"
+ test_apply_file.write_text("# Apply Test\n\nNo frontmatter here.\n")
+ if enrich_tags_dir.exists():
+ shutil.rmtree(enrich_tags_dir)
+ enrich.cmd_tags(vault_dir, apply=True, budget=50000,
+ model="test-model", force=True, delay=0, workers=1)
+ test_content = test_apply_file.read_text()
+ fm_check, _, _ = parse_frontmatter(test_content)
+ test("tags apply writes frontmatter",
+ fm_check is not None and 'tags' in fm_check,
+ f"got fm={fm_check}")
+
+ # 5.13 Tags resume (idempotent — skip existing results)
+ _mock_call_count = 0
+ enrich.cmd_tags(vault_dir, apply=False, budget=50000,
+ model="test-model", force=False, delay=0, workers=1)
+ test("tags resume skips existing batches", _mock_call_count == 0,
+ f"expected 0 API calls, got {_mock_call_count}")
+
+ # 5.14 scan_vault_for_links — unified scan
+ scan_stems, scan_stem_to_path, scan_catalog, scan_entries = enrich.scan_vault_for_links(vault_dir, force=False)
+ test("scan_vault_for_links returns stems", len(scan_stems) > 0)
+ test("scan_vault_for_links has alpha", 'alpha' in scan_stems)
+ test("scan_vault_for_links stem_to_path maps", 'bob' in scan_stem_to_path)
+ test("scan_vault_for_links catalog is dict", isinstance(scan_catalog, dict))
+ total_entries = sum(len(v) for v in scan_catalog.values())
+ test("scan_vault_for_links catalog has entries", total_entries > 0,
+ f"got {total_entries} entries across {len(scan_catalog)} domains")
+ test("scan_vault_for_links finds link entries", len(scan_entries) > 0)
+
+ # 5.15 scan_vault_for_links force mode includes all files
+ _, _, _, force_entries = enrich.scan_vault_for_links(vault_dir, force=True)
+ test("scan_vault_for_links force includes more",
+ len(force_entries) >= len(scan_entries))
+
+ # ═══════════════════════════════════════════════════════
+ # 5b. swarm-links tests
+ # ═══════════════════════════════════════════════════════
+ print("\n--- swarm-links ---")
+
+ # 5b.2 format_catalog formats entries
+ sample_entries = [
+ {'stem': 'test-note', 'type': 'note', 'tags': ['ai', 'test'], 'desc': 'A test note'},
+ {'stem': 'other', 'type': 'project', 'tags': ['dev'], 'desc': 'Other note'},
+ ]
+ formatted = enrich.format_catalog(sample_entries)
+ test("format_catalog contains stems", 'test-note' in formatted and 'other' in formatted)
+ test("format_catalog contains tags", 'ai, test' in formatted)
+
+ # 5b.3 format_catalog truncates at max_entries
+ big_entries = [{'stem': f'note-{i}', 'type': 'note', 'tags': [], 'desc': f'Note {i}'} for i in range(50)]
+ truncated = enrich.format_catalog(big_entries, max_entries=10)
+ test("format_catalog truncates", '(truncated, 40 more)' in truncated)
+
+ # 5b.4 format_catalog empty
+ empty_formatted = enrich.format_catalog([])
+ test("format_catalog empty", empty_formatted == "(empty catalog)")
+
+ # 5b.5 swarm-link validation: strict set membership
+ # Simulate what _process_swarm_link_batch does internally
+ test_stems = {'alpha', 'bob', 'ml-basics', 'python-tips'}
+ raw_links = ['alpha', 'bob', 'nonexistent-stem', 'invented-name']
+ valid = [s for s in raw_links if s in test_stems]
+ test("swarm strict validation passes exact stems",
+ valid == ['alpha', 'bob'],
+ f"got {valid}")
+
+ # 5b.6 swarm-link filters self-links
+ file_stem = 'alpha'
+ raw_with_self = ['alpha', 'bob', 'ml-basics']
+ filtered_self = [s for s in raw_with_self if s in test_stems and s != file_stem]
+ test("swarm filters self-links",
+ 'alpha' not in filtered_self and 'bob' in filtered_self)
+
+ # 5b.7 swarm-link filters existing links
+ existing_links = {'bob'}
+ raw_all = ['alpha', 'bob', 'ml-basics']
+ filtered_existing = [s for s in raw_all
+ if s in test_stems and s != file_stem and s not in existing_links]
+ test("swarm filters existing links",
+ 'bob' not in filtered_existing and 'ml-basics' in filtered_existing)
+
+ # 5b.8 swarm-links dry run
+ _mock_call_count = 0
+ enrich_swarm_dir = vault_dir / '.graph' / 'enrich' / 'swarm-links'
+ if enrich_swarm_dir.exists():
+ shutil.rmtree(enrich_swarm_dir)
+ enrich.cmd_swarm_links(vault_dir, apply=False, budget=100000,
+ model="test-model", force=True, delay=0, workers=1)
+ swarm_results = list(enrich_swarm_dir.glob('batch-*-results.json'))
+ test("swarm-links dry run creates result files", len(swarm_results) > 0)
+ test("swarm-links dry run called mock API", _mock_call_count > 0)
+
+ # 5b.9 swarm-links results have matched_links from strict validation
+ if swarm_results:
+ first_swarm = json.loads(swarm_results[0].read_text())
+ has_matched = any(
+ len(r.get('matched_links', [])) > 0
+ for r in first_swarm.get('results', [])
+ )
+ test("swarm-links strict match finds real stems", has_matched,
+ f"results: {first_swarm.get('results', [])[:2]}")
+ # Verify no nonexistent stems passed
+ all_matched = []
+ for r in first_swarm.get('results', []):
+ all_matched.extend(r.get('matched_links', []))
+ test("swarm-links no invented stems in matched",
+ 'nonexistent-stem-xyz' not in all_matched,
+ f"matched: {all_matched}")
+
+ # 5b.10 swarm-links --apply appends ## Related
+ swarm_test_file = vault_dir / "swarm-test.md"
+ swarm_test_file.write_text("---\ntype: note\ntags: [test]\n---\n# Swarm Test\n\nSome content.\n")
+ if enrich_swarm_dir.exists():
+ shutil.rmtree(enrich_swarm_dir)
+ enrich.cmd_swarm_links(vault_dir, apply=True, budget=100000,
+ model="test-model", force=True, delay=0, workers=1)
+ swarm_content = swarm_test_file.read_text()
+ test("swarm-links apply adds Related section",
+ '## Related' in swarm_content,
+ f"content: {swarm_content[:200]}")
+
+ # Restore original
+ enrich.call_openrouter = original_call
+
+ # ═══════════════════════════════════════════════════════
+ # 6. link_cleanup.py tests
+ # ═══════════════════════════════════════════════════════
+ print("\n--- link_cleanup.py ---")
+ import link_cleanup
+
+ # 6.1 build_stems_and_paths
+ valid_targets = link_cleanup.build_stems_and_paths(vault_dir)
+ test("cleanup stems includes alpha",
+ valid_targets.get('unique_stem', {}).get('alpha') == 'projects/alpha',
+ f"got: {valid_targets.get('unique_stem', {}).get('alpha')}")
+ test("cleanup stems includes path",
+ valid_targets.get('exact', {}).get('projects/alpha') == 'projects/alpha',
+ f"got: {valid_targets.get('exact', {}).get('projects/alpha')}")
+
+ # 6.2 check_link_target — valid
+ test("check_link valid stem", link_cleanup.check_link_target('alpha', valid_targets))
+ test("check_link valid path", link_cleanup.check_link_target('projects/alpha', valid_targets))
+
+ # 6.3 check_link_target — invalid
+ test("check_link invalid", not link_cleanup.check_link_target('nonexistent-file-xyz', valid_targets))
+ test("check_link ambiguous stem invalid", not link_cleanup.check_link_target('bob', valid_targets))
+
+ # 6.4 cleanup_related_section — removes broken links
+ content_with_broken = (
+ "# Test\n\nSome body.\n\n"
+ "## Related\n"
+ "- [[projects/alpha]]\n"
+ "- [[nonexistent-phantom-link]]\n"
+ "- [[knowledge/ml-basics]]\n"
+ )
+ new_content, removed, kept = link_cleanup.cleanup_related_section(
+ content_with_broken, valid_targets)
+ test("cleanup removes phantom link",
+ 'nonexistent-phantom-link' in removed)
+ test("cleanup keeps valid links", len(kept) >= 2)
+ test("cleanup output has no phantom",
+ '[[nonexistent-phantom-link]]' not in new_content)
+ test("cleanup output has valid links",
+ '[[projects/alpha]]' in new_content)
+
+ # 6.5 cleanup_related_section — all broken → delete section
+ content_all_broken = (
+ "# Test\n\nBody text.\n\n"
+ "## Related\n"
+ "- [[ghost-link-1]]\n"
+ "- [[ghost-link-2]]\n"
+ )
+ new_all, removed_all, kept_all = link_cleanup.cleanup_related_section(
+ content_all_broken, valid_targets)
+ test("cleanup deletes section when all broken",
+ '## Related' not in new_all)
+ test("cleanup reports all removed", len(removed_all) == 2)
+
+ # 6.6 cleanup doesn't touch body links
+ content_body_links = (
+ "# Test\n\n"
+ "See [[nonexistent-in-body]] for details.\n\n"
+ "## Related\n"
+ "- [[projects/alpha]]\n"
+ )
+ new_body, removed_body, _ = link_cleanup.cleanup_related_section(
+ content_body_links, valid_targets)
+ test("cleanup ignores body links",
+ '[[nonexistent-in-body]]' in new_body)
+ test("cleanup body link not in removed",
+ 'nonexistent-in-body' not in removed_body)
+
+ # 6.7 dry run doesn't modify files
+ # Create a file with phantom link
+ cleanup_test_file = vault_dir / "cleanup-test.md"
+ cleanup_test_file.write_text(
+ "---\ntype: note\n---\n# Cleanup Test\n\n"
+ "## Related\n- [[phantom-target-xyz]]\n"
+ )
+ original_content = cleanup_test_file.read_text()
+ report = link_cleanup.run_cleanup(vault_dir, apply=False)
+ test("cleanup dry run safe", cleanup_test_file.read_text() == original_content)
+ test("cleanup report has links_removed", len(report['links_removed']) > 0)
+
+ # 6.8 --apply modifies files
+ report_apply = link_cleanup.run_cleanup(vault_dir, apply=True)
+ new_cleanup_content = cleanup_test_file.read_text()
+ test("cleanup apply removes phantom",
+ '[[phantom-target-xyz]]' not in new_cleanup_content)
+
+ # 6.9 cleanup writes report json
+ report_path = vault_dir / '.graph' / 'link-cleanup-report.json'
+ test("cleanup writes report", report_path.exists())
+
+ # ═══════════════════════════════════════════════════════
+ # 7. Fix verification tests (code review findings)
+ # ═══════════════════════════════════════════════════════
+ print("\n--- code review fixes ---")
+
+ # 7.1 CRLF frontmatter parsing
+ crlf_content = "---\r\ntype: note\r\nstatus: active\r\ntags: [ai, test]\r\n---\r\n# CRLF Note\r\n\r\nBody with CRLF.\r\n"
+ fm_crlf, body_crlf, _ = parse_frontmatter(crlf_content)
+ test("parse_fm CRLF: extracts type", fm_crlf is not None and fm_crlf.get('type') == 'note')
+ test("parse_fm CRLF: extracts tags", fm_crlf is not None and fm_crlf.get('tags') == ['ai', 'test'])
+ test("parse_fm CRLF: body preserved", 'CRLF Note' in body_crlf)
+
+ # 7.2 Literal block (|-) preserves newlines
+ literal_content = "---\ntype: note\ndescription: |-\n Line one\n Line two\n Line three\n---\n# Test\n"
+ fm_literal, _, _ = parse_frontmatter(literal_content)
+ test("parse_fm literal block |- has newlines",
+ fm_literal is not None and '\n' in fm_literal.get('description', ''),
+ f"got: {fm_literal.get('description', '') if fm_literal else 'None'}")
+
+ # 7.3 Fold block (>-) joins with spaces
+ fold_content = "---\ntype: note\ndescription: >-\n First part\n second part\n---\n# Test\n"
+ fm_fold, _, _ = parse_frontmatter(fold_content)
+ test("parse_fm fold block >- joins with space",
+ fm_fold is not None and 'First part second part' in fm_fold.get('description', ''),
+ f"got: {fm_fold.get('description', '') if fm_fold else 'None'}")
+
+ # 7.4 format_field YAML special chars — colon
+ test("format_field escapes colon",
+ format_field('title', 'Key: Value') == 'title: "Key: Value"')
+
+ # 7.5 format_field YAML special chars — hash
+ test("format_field escapes hash",
+ format_field('title', 'Topic #1') == 'title: "Topic #1"')
+
+ # 7.6 format_field YAML special chars — brackets
+ test("format_field escapes brackets",
+ format_field('note', 'See [link]') == 'note: "See [link]"')
+
+ # 7.7 format_field YAML special chars — quotes
+ result_q = format_field('title', 'He said "hello"')
+ test("format_field escapes double quotes",
+ result_q == 'title: "He said \\"hello\\""',
+ f"got: {result_q}")
+
+ # 7.8 format_field safe string — no escaping
+ test("format_field no escape for safe string",
+ format_field('type', 'note') == 'type: note')
+
+ # 7.9 extract_wikilinks strips #anchor
+ anchor_links = extract_wikilinks("See [[target#heading]] and [[other#sec|display]]")
+ test("extract_wikilinks anchor stripped",
+ len(anchor_links) == 2 and anchor_links[0][0] == 'target',
+ f"got: {anchor_links}")
+ test("extract_wikilinks anchor with alias",
+ anchor_links[1] == ('other', 'display'),
+ f"got: {anchor_links[1] if len(anchor_links) > 1 else 'missing'}")
+
+ # 7.10 extract_wikilinks anchor-only link skipped
+ anchor_only = extract_wikilinks("See [[#heading-only]]")
+ test("extract_wikilinks anchor-only skipped",
+ len(anchor_only) == 0,
+ f"got: {anchor_only}")
+
+ # 7.11 resolve_link strips anchor (graph.py)
+ from graph import resolve_link, fix_broken_links, build_graph
+ from dedup import merge_content
+ from daily import (
+ build_vault_index as build_daily_index,
+ extract_entities as extract_daily_entities,
+ build_relationships as build_daily_relationships,
+ derive_legacy_buckets,
+ build_output_meta,
+ process_date as process_daily_date,
+ )
+ test_path_index = {'target': 'knowledge/target', 'foo': 'projects/foo'}
+ test("resolve_link strips anchor",
+ resolve_link('target#heading', test_path_index) == 'knowledge/target')
+ test("resolve_link empty after anchor strip",
+ resolve_link('#heading', test_path_index) is None)
+
+ # 7.12 graph fix rewrites only the exact broken wikilink
+ fix_vault = tmp / 'graph-fix-vault'
+ (fix_vault / 'notes').mkdir(parents=True, exist_ok=True)
+ (fix_vault / 'docs').mkdir(parents=True, exist_ok=True)
+ source_note = fix_vault / 'notes/source.md'
+ source_note.write_text("See [[visa]] and [[visa-guide]].\n")
+ (fix_vault / 'docs/visa.md').write_text("# Visa\n")
+ (fix_vault / 'docs/visa-guide.md').write_text("# Visa guide\n")
+ synthetic_graph = {
+ 'broken_link_list': [{'source': 'notes/source', 'target': 'visa'}]
+ }
+ fixes, applied = fix_broken_links(fix_vault, synthetic_graph, apply=True)
+ updated_source = source_note.read_text()
+ test("graph fix suggests unique stem target",
+ len(fixes) == 1 and fixes[0]['new'] == 'docs/visa',
+ f"got: {fixes}")
+ test("graph fix applies one exact replacement", applied == 1, f"got: {applied}")
+ test("graph fix does not mutate prefixed links",
+ '[[docs/visa]]' in updated_source and '[[visa-guide]]' in updated_source and '[[docs/visa-guide]]' not in updated_source,
+ f"got: {updated_source}")
+
+ # 7.13 nested hub notes are not orphans
+ hub_vault = tmp / 'graph-hub-vault'
+ (hub_vault / 'foo').mkdir(parents=True, exist_ok=True)
+ (hub_vault / 'agents/x').mkdir(parents=True, exist_ok=True)
+ (hub_vault / 'foo/_index.md').write_text("# Foo hub\n")
+ (hub_vault / 'agents/x/MEMORY.md').write_text("# Agent memory\n")
+ (hub_vault / 'foo/card.md').write_text("# Card\n")
+ hub_graph = build_graph(hub_vault, schema)
+ test("graph skips nested _index orphan",
+ 'foo/_index' not in hub_graph['orphan_list'],
+ f"got: {hub_graph['orphan_list']}")
+ test("graph skips nested MEMORY orphan",
+ 'agents/x/MEMORY' not in hub_graph['orphan_list'],
+ f"got: {hub_graph['orphan_list']}")
+ test("graph keeps regular orphan",
+ 'foo/card' in hub_graph['orphan_list'],
+ f"got: {hub_graph['orphan_list']}")
+
+ # 7.14 dedup merge writes YAML via shared serializer
+ dedup_merge_dir = tmp / 'dedup-merge-vault'
+ dedup_merge_dir.mkdir(parents=True, exist_ok=True)
+ canonical_note = dedup_merge_dir / 'canonical.md'
+ extra_note = dedup_merge_dir / 'extra.md'
+ long_desc = "Long description with colon: hash # brackets [x] and enough words to cross the serializer threshold for multiline output."
+ canonical_note.write_text(
+ "---\n"
+ "type: note\n"
+ "tags: [base]\n"
+ "description: Base note\n"
+ "---\n"
+ "# Canon\n\n"
+ "Body.\n"
+ )
+ extra_note.write_text(
+ "---\n"
+ f"description: {long_desc}\n"
+ "tags: [base, extra]\n"
+ "---\n"
+ "# Extra\n\n"
+ "## Imported Section\n"
+ "Merged text.\n"
+ )
+ merged = merge_content(canonical_note, [extra_note])
+ merged_content = canonical_note.read_text()
+ merged_fm, merged_body, _ = parse_frontmatter(merged_content)
+ test("dedup merge reports changed", merged, "merge_content returned False")
+ test("dedup merge keeps YAML parseable",
+ merged_fm is not None and merged_fm.get('description') == long_desc,
+ f"got: {merged_fm}")
+ test("dedup merge preserves list fields",
+ merged_fm.get('tags') == ['base', 'extra'],
+ f"got: {merged_fm.get('tags')}")
+ test("dedup merge uses multiline serializer",
+ 'description: >-' in merged_content,
+ f"got: {merged_content}")
+ test("dedup merge keeps original field order",
+ merged_content.find('type: note') < merged_content.find('tags: [base, extra]') < merged_content.find('description: >-'),
+ f"got: {merged_content}")
+ test("dedup merge appends unique body sections",
+ '## Imported Section' in merged_body,
+ f"got: {merged_body}")
+
+ # 7.15 daily extraction resolves typed linked entities
+ daily_index = build_daily_index(vault_dir)
+ daily_entities = extract_daily_entities(
+ "Worked with [[contacts/bob|Bob Smith]] on [[projects/alpha|Alpha]].",
+ daily_index
+ )
+ linked_types = {(item['name'], item['type']) for item in daily_entities['linked_entities']}
+ daily_relationships = build_daily_relationships(daily_entities)
+ test("daily typed links include contact",
+ ('Bob Smith', 'contact') in linked_types,
+ f"got: {daily_entities['linked_entities']}")
+ test("daily typed links include project",
+ ('Alpha', 'project') in linked_types,
+ f"got: {daily_entities['linked_entities']}")
+ test("daily projects bucket excludes contacts",
+ daily_entities['projects'] == [{'name': 'Alpha', 'link': 'projects/alpha'}],
+ f"got: {daily_entities['projects']}")
+ test("daily relationships use typed linked entities",
+ any(r['from_type'] == 'contact' and r['to_type'] == 'project' for r in daily_relationships) or
+ any(r['from_type'] == 'project' and r['to_type'] == 'contact' for r in daily_relationships),
+ f"got: {daily_relationships}")
+ legacy_buckets = derive_legacy_buckets([
+ {'name': 'Alpha', 'link': 'projects/alpha', 'type': 'project', 'domain': 'work'},
+ {'name': 'Acme', 'link': 'companies/acme', 'type': 'company', 'domain': 'crm'},
+ {'name': 'Bob Smith', 'link': 'contacts/bob', 'type': 'contact', 'domain': 'crm'},
+ ])
+ test("daily legacy buckets derive projects from linked_entities",
+ legacy_buckets['projects'] == [{'name': 'Alpha', 'link': 'projects/alpha'}],
+ f"got: {legacy_buckets}")
+ test("daily legacy buckets derive companies from linked_entities",
+ legacy_buckets['companies'] == [{'name': 'Acme', 'link': 'companies/acme'}],
+ f"got: {legacy_buckets}")
+ output_meta = build_output_meta()
+ test("daily output meta marks linked_entities primary",
+ output_meta.get('primary_entity_field') == 'entities.linked_entities',
+ f"got: {output_meta}")
+ test("daily output meta marks legacy buckets deprecated",
+ output_meta.get('deprecated_fields') == ['entities.projects', 'entities.companies'],
+ f"got: {output_meta}")
+ dated_result = process_daily_date(daily_dir, daily_index, '2026-03-01')
+ test("daily process_date includes output meta",
+ dated_result.get('_meta', {}).get('primary_entity_field') == 'entities.linked_entities',
+ f"got: {dated_result.get('_meta')}")
+
+ # 7.16 check_link_target strips anchor (link_cleanup.py)
+ test("check_link_target strips anchor",
+ link_cleanup.check_link_target('alpha#section', valid_targets))
+ test("check_link_target anchor-only is not broken",
+ link_cleanup.check_link_target('#heading', valid_targets))
+
+ # 7.17 load_schema local priority
+ # Create two schemas in a temp dir — schema.json and schema.local.json
+ _schema_cache.clear()
+ schema_test_dir = tmp / 'schema-priority-test'
+ schema_test_dir.mkdir(exist_ok=True)
+ scripts_dir_test = schema_test_dir / 'scripts'
+ scripts_dir_test.mkdir(exist_ok=True)
+ base_schema = {"node_types": {"note": {"description": "base", "required": [], "status": []}},
+ "type_aliases": {}, "field_fixes": {}, "domain_inference": {},
+ "path_type_hints": {}, "status_order": {}, "status_defaults": {},
+ "richness_fields": {}, "entity_extraction": {}, "decay": {"rate": 0.01, "floor": 0.1},
+ "ignore_tags": []}
+ local_schema = dict(base_schema)
+ local_schema["_marker"] = "local"
+ (schema_test_dir / 'schema.json').write_text(json.dumps(base_schema))
+ (schema_test_dir / 'schema.local.json').write_text(json.dumps(local_schema))
+ # load_schema with explicit path still works
+ loaded_base = load_schema(schema_test_dir / 'schema.json')
+ test("load_schema explicit path works", 'node_types' in loaded_base)
+ _schema_cache.clear()
+ loaded_local = load_schema(schema_test_dir / 'schema.local.json')
+ test("load_schema local has marker", loaded_local.get('_marker') == 'local')
+
+ # 7.18 load_schema cache by resolved path (not 'default')
+ _schema_cache.clear()
+ s1 = load_schema(schema_path)
+ s2 = load_schema(schema_path)
+ test("load_schema cache hit same object", s1 is s2)
+ # Different path → different cache entry
+ _schema_cache.clear()
+ s3 = load_schema(schema_test_dir / 'schema.json')
+ s4 = load_schema(schema_path)
+ test("load_schema different paths different cache", s3 is not s4)
+
+ # 7.19 swarm_reduce prompt matches REQUIRED_SCHEMA_SECTIONS
+ from swarm_reduce import WAVE2_PROMPT_TEMPLATE, REQUIRED_SCHEMA_SECTIONS
+ test("swarm_reduce prompt no region_fixes",
+ 'region_fixes' not in WAVE2_PROMPT_TEMPLATE)
+ # All required sections mentioned in prompt
+ for section in REQUIRED_SCHEMA_SECTIONS:
+ test(f"swarm_reduce prompt mentions {section}",
+ section in WAVE2_PROMPT_TEMPLATE,
+ f"'{section}' not found in prompt")
+
+ # ═══════════════════════════════════════════════════════
+ # SUMMARY
+ # ═══════════════════════════════════════════════════════
+ total = PASS + FAIL
+ print(f"\n{'='*60}")
+ print(f" RESULTS: {PASS}/{total} passed, {FAIL} failed")
+ print(f"{'='*60}")
+
+ finally:
+ shutil.rmtree(tmp, ignore_errors=True)
+
+ sys.exit(0 if FAIL == 0 else 1)
+
+
+if __name__ == '__main__':
+ main()