An always-on second brain you talk to. Voice notes in Telegram → typed, linked knowledge in your Obsidian vault. Runs 24/7 on the Claude subscription you already have.
Python ai-agentsai-automationclaudeclaude-codeebbinghausknowledge-graphllmmemory-systemobsidianpersonal-assistantpkmproductivitypythonsecond-braintelegram-botvoice-assistant
feat: add agent-memory skill — Ebbinghaus decay engine with tiered search
smixsfbba214360ff
diff --git a/vault/.claude/skills/agent-memory/SKILL.md b/vault/.claude/skills/agent-memory/SKILL.mdnew file mode 100644index 0000000..9fc2973--- /dev/null+++ b/vault/.claude/skills/agent-memory/SKILL.md@@ -0,0 +1,206 @@+---+type: note+description: >-+last_accessed: 2026-02-26+relevance: 1.0+tier: active+name: agent-memory+Use when: (1) setting up memory for a new agent, (2) diagnosing memory bloat or context+Triggers: "memory management", "organize vault", "memory decay", "forgetting curve",+---++# Agent Memory Management++Memory system with automatic decay, tiered search, and creative recall.+Works with any directory of markdown files.++## Quick Start++### 1. Scan existing files+```bash+python3 scripts/memory-engine.py scan <directory>+```+Reports: file count, YAML coverage, size, recommendations.++### 2. Bootstrap YAML frontmatter+```bash+python3 scripts/memory-engine.py init <directory> [--dry-run]+```+Adds `relevance`, `last_accessed`, `tier` to files missing frontmatter.+Infers `type` from directory path. Infers dates from YAML fields, git log, or file mtime.++### 3. Run decay+```bash+python3 scripts/memory-engine.py decay <directory> [--dry-run]+```+Updates all cards: recalculates relevance scores and reassigns tiers.+Schedule daily via cron for automatic forgetting.++### 4. Touch on read+```bash+python3 scripts/memory-engine.py touch <filepath>+```+Resets card to active (relevance=1.0). Call this when reading a card to answer a question.++### 5. Creative recall+```bash+python3 scripts/memory-engine.py creative <N> <directory>+```+Random sample from cold/archive tiers. Read these cards and look for unexpected connections.++### 6. Health check+```bash+python3 scripts/memory-engine.py stats <directory>+```+Shows tier distribution, context budget, stale card count.++## Core Concepts++### Three-Layer Architecture++| Layer | What | Size Target | Loaded |+|-------|------|-------------|--------|+| Hot context | State file (volatile focus, blockers) | <4KB | Every turn |+| Searchable vault | Cards with YAML, one per entity | Unlimited | On demand |+| Archive | Old logs, completed work | Unlimited | Deep/creative only |++**Rule:** Each fact lives in ONE place. If it's in a card, don't also put it in the state file.+See [references/architecture.md](references/architecture.md) for full design rationale.++### Forgetting Curve++Cards have `relevance: 0.0-1.0` that decays linearly over time:+- Day 0: 1.0 (just accessed)+- Day 7: 0.90 → tier: active+- Day 21: 0.69 → tier: warm+- Day 33: 0.50 → tier: cold+- Day 60+: 0.10 (floor) → tier: archive++`core` tier is manual-only — for identity, security, pricing. Never auto-demoted.++### Tier-Aware Search++| Mode | Tiers searched | When |+|------|---------------|------|+| heartbeat | core + active | Quick checks, monitoring |+| normal | active + warm | Most questions |+| deep | all tiers | Strategy, OSINT, complex analysis |+| creative | random cold+archive | Brainstorming, ideation |++See [references/search-protocols.md](references/search-protocols.md) for detailed protocols.++### YAML Frontmatter++Minimum required fields (managed by engine):+```yaml+---+relevance: 0.85+last_accessed: 2026-02-25+tier: active+---+```++See [references/yaml-schema.md](references/yaml-schema.md) for full schema with domain-specific fields.++## Daily Files (Episodic Memory)++Daily files (`memory/YYYY-MM-DD.md`) are the agent's episodic memory — what happened each day.+They follow the same decay system as vault cards. **Never delete them.**++### Lifecycle++```+Day 0: Created (end of day cron or manual) → tier: active, relevance: 1.0+Day 1-7: Auto-loaded at session start (today+yesterday) → tier: active+Day 8-21: Searchable but not auto-loaded → tier: warm+Day 22-60: Deep search only → tier: cold+Day 60+: Creative mode or explicit recall → tier: archive+```++### YAML Frontmatter for Daily Files++Add to each daily file (engine handles this via `init` or `decay`):+```yaml+---+type: daily+date: 2026-02-18+relevance: 0.76+last_accessed: 2026-02-18+tier: warm+---+```++### Session Loading Protocol++At session start, load only active-tier daily files:+```+1. Read today's daily file (if exists)+2. Read yesterday's daily file (if exists)+3. Stop. Don't load older files.+```++For questions about the past ("what happened last week?"), search warm/cold tier dailies on demand.++### Why Never Delete++1. **Disk cost ≈ zero.** 30 daily files ≈ 300KB. Irrelevant savings.+2. **Context is lost forever.** A knowledge graph captures entities but not reasoning, tone, or "why."+3. **Semantic search works on old files.** A query about "Red Bull tender" finds the relevant daily file regardless of age.+4. **Creative mode surfaces old context.** Forgotten daily entries become unexpected inspiration.+5. **Re-reading from source (Telegram, email) is expensive.** 100+ messages per day × API calls × LLM summarization > keeping a 10KB file.++### Compression (Optional)++If daily files grow large (>20KB), compress old ones instead of deleting:+1. Extract key decisions, contacts, and outcomes into vault cards+2. Trim the daily file to a 20-line summary+3. Keep the summary with its YAML frontmatter intact++This preserves searchability while reducing storage. But even without compression, a year of daily files is ~4MB — negligible.++## Configuration++Generate default config:+```bash+python3 scripts/memory-engine.py config <directory>+```++Creates `.memory-config.json`:+```json+{+ "tiers": {"active": 7, "warm": 21, "cold": 60},+ "decay_rate": 0.015,+ "relevance_floor": 0.1,+ "skip_patterns": ["_index.md"],+ "type_inference": {"crm/": "crm", "leads/": "lead"},+ "use_git_dates": true+}+```++Adjust tier thresholds and decay rate to match your domain's natural rhythm.+Fast-moving domains (sales): tighter thresholds (active=3, warm=10, cold=30).+Slow domains (research): wider thresholds (active=14, warm=45, cold=120).++## Cron Integration++Add decay to daily maintenance (OpenClaw example):+```json+{+ "name": "memory-decay",+ "schedule": {"kind": "cron", "expr": "0 22 * * *", "tz": "UTC"},+ "payload": {"kind": "agentTurn", "message": "Run: python3 scripts/memory-engine.py decay vault/crm/"}+}+```++## Anti-Patterns++| Don't | Do Instead |+|-------|------------|+| Load all contacts into state file | Keep them in vault cards, search on demand |+| Create "knowledge graph" that duplicates vault | Vault IS the graph. Use index files for navigation |+| Store same fact in 3 files | One card per entity, reference via links |+| Delete old daily files to "save space" | Keep all dailies, let tier decay handle visibility |+| Auto-load all daily files at session start | Load only today + yesterday; search older on demand |+| Search all 500 cards for every question | Check index first, filter by tier, then search |+| Touch every card during bulk operations | Only touch on meaningful read/update |+| Build elaborate review systems | Let decay handle it — if you don't use it, it fades |diff --git a/vault/.claude/skills/agent-memory/references/architecture.md b/vault/.claude/skills/agent-memory/references/architecture.mdnew file mode 100644index 0000000..c5125ab--- /dev/null+++ b/vault/.claude/skills/agent-memory/references/architecture.md@@ -0,0 +1,198 @@+---+type: note+title: Memory Architecture+last_accessed: 2026-02-26+relevance: 1.0+tier: active+---+# Memory Architecture++## The Problem++AI agents forget everything between sessions. Common solutions (dump everything into context, vector-search everything) create bloat and noise. Human memory works differently: it has layers, it decays, and it surprises you with random connections.++## Design Principles++### Single Source of Truth (DRY)+Each fact lives in ONE place. If a contact exists in a CRM card, don't also list them in a "key contacts" section of your state file. Instead, reference them: `see vault/crm/alice.md`.++**Violation test:** Search for any person/project name. If it appears in 3+ files with substantive detail, you have a DRY violation.++### Context is a Shared Resource (KISS)+Every byte loaded per turn costs tokens. An agent loading 40KB of context files is burning ~10,000 tokens before the conversation starts. Measure your context budget:+- State file (volatile): target <4KB+- Config files: target <15KB total+- Per-turn overhead = sum of all always-loaded files++### Build Only What You Use (YAGNI)+Don't create "knowledge graphs," "daily digests," or "weekly reviews" unless you actively query them. If a cron job generates a report nobody reads, it's waste.++## Three-Layer Architecture++```+Layer 1: HOT CONTEXT (always loaded, <4KB)+ └── State file — volatile: active focus, blockers, reminders+ NOT: contacts, history, reference, tools++Layer 2: SEARCHABLE VAULT (on-demand, unlimited)+ └── Cards with YAML frontmatter — one file per entity+ Searched via: semantic search, grep, graph traversal+ Organized by: domain directories + index files (MOC)++Layer 3: ARCHIVE (deep search only)+ └── Old daily logs, completed projects, cold contacts+ Still searchable but excluded from default queries+```++### Layer 1: Hot Context++This is what's loaded every turn. Must be ruthlessly slim.++**Include:** Current week's focus, active blockers, 3-5 pending items, security rules, navigation hints.++**Exclude:** Contact lists (use vault), tool docs (separate file), event history (daily files), anything searchable.++**Audit:** If your state file exceeds 80 lines, something belongs in Layer 2.++### Layer 2: Searchable Vault++One markdown file per entity (person, project, company, idea). YAML frontmatter enables filtering and automation.++**Directory structure:**+```+vault/+├── crm/ # contacts and companies+│ ├── clients/+│ ├── leads/+│ └── personal/+├── projects/ # active and past projects+├── MOC/ # Maps of Content (index files)+└── .graph/ # computed graph data+```++**Why one file per entity:** Enables granular decay, individual relevance scoring, and precise search results. A single "contacts.md" with 200 entries can't decay — the whole file is either loaded or not.++### Layer 2b: Daily Files (Episodic Memory)++Daily logs (`memory/YYYY-MM-DD.md`) sit between vault and archive. They capture the raw narrative of each day — conversations, decisions, reasoning, context that doesn't fit neatly into entity cards.++**Never delete daily files.** The decay system handles visibility:+- Active (0-7 days): loaded at session start+- Warm (8-21 days): searchable, not auto-loaded+- Cold (22-60 days): deep search only+- Archive (60+): creative mode, explicit queries++**Why keep everything:**+- Disk: 365 daily files ≈ 4MB. Irrelevant cost.+- Context: knowledge graphs capture entities but lose reasoning and tone.+- Search: semantic search finds old dailies just as well as new ones.+- Source rebuild: re-reading 100+ Telegram messages is far more expensive than keeping a 10KB summary.++**Compression:** If a daily file exceeds 20KB, extract key facts into vault cards and trim the daily to a 20-line summary. Keep the YAML frontmatter intact.++### Layer 3: Archive++One-off analysis reports. Completed project retrospectives. Generated artifacts that served a temporary purpose. These are still searchable but excluded from default queries. Daily files are NOT archive — they stay in `memory/` with tier-based visibility.++## Forgetting Curve++Inspired by Ebbinghaus (1885): memory strength decays over time without reinforcement.++### Relevance Score++Each card has `relevance: 0.0-1.0` in its frontmatter. Decays linearly:++```+relevance = max(floor, 1.0 - days_since_access × rate)++Default: rate=0.015, floor=0.1+→ After 7 days: 0.90+→ After 21 days: 0.69+→ After 33 days: 0.50+→ After 60 days: 0.10 (floor)+```++### Tier Assignment++Based on days since `last_accessed`:++| Tier | Days | Description |+|------|------|-------------|+| core | manual | Never auto-assigned. Identity, security, pricing. |+| active | 0-7 | Hot context. Searched in all modes. |+| warm | 8-21 | Default search radius. Gradually fading. |+| cold | 22-60 | Deep search only. Mostly forgotten. |+| archive | 60+ | Creative mode or explicit recall only. |++### Touch Protocol++When an agent reads or references a card, it should reset the card:+```bash+python3 memory-engine.py touch <filepath>+```+This sets `last_accessed` to today, `relevance` to 1.0, and `tier` to active.++**When to touch:**+- Agent reads card content to answer a question+- Agent updates card with new information+- User explicitly mentions the entity++**When NOT to touch:**+- Card appears in search results but isn't opened+- Automated scan (decay script itself)+- Bulk operations++## Search Protocols++Different tasks need different search depths:++### Heartbeat Mode (fast checks)+Search: core + active only.+Use for: quick status checks, routine monitoring, simple questions.+Cost: minimal — only hot cards in scope.++### Normal Mode (default)+Search: core + active + warm.+Use for: most questions, task execution, lookups.+Cost: moderate — includes fading but recent cards.++### Deep Mode (complex tasks)+Search: all tiers.+Use for: strategy, OSINT, complex analysis, "find everything about X."+Cost: high — full vault scan.++### Creative Mode (divergent thinking)+Method: random sample from cold + archive tiers.+Use for: brainstorming, finding unexpected connections, "what if" scenarios.+Not semantic search — deliberately random to surface forgotten associations.++```bash+python3 memory-engine.py creative 5 vault/+```++## Multi-Agent Shared Memory++When multiple agents share a vault:++1. **Shared YAML schema** — all agents use same frontmatter fields+2. **Each agent touches on read** — keeps decay accurate across agents+3. **One vault, one truth** — don't fork the vault per agent+4. **Conflict resolution** — last write wins for metadata; append-only for history sections++## Context Budget Calculator++Measure your actual per-turn cost:++```+Always-loaded files:+ STATE.md _____ bytes+ CONFIG.md _____ bytes+ IDENTITY.md _____ bytes+ RULES.md _____ bytes+ ─────────────────────────+ Total context: _____ bytes ÷ 4 ≈ _____ tokens++Target: <25KB (6,000 tokens) for always-loaded context+```++Every 1KB saved = ~250 tokens freed per turn for actual conversation.diff --git a/vault/.claude/skills/agent-memory/references/search-protocols.md b/vault/.claude/skills/agent-memory/references/search-protocols.mdnew file mode 100644index 0000000..95eef61--- /dev/null+++ b/vault/.claude/skills/agent-memory/references/search-protocols.md@@ -0,0 +1,125 @@+---+type: note+title: Search Protocols+last_accessed: 2026-02-26+relevance: 1.0+tier: active+---+# Search Protocols++## Protocol Selection++Before searching, classify the task:++| Signal | Mode | Search Radius |+|--------|------|---------------|+| Quick status check, monitoring | heartbeat | core + active |+| Normal question, lookup | normal | active + warm |+| "Find everything about X" | deep | all tiers |+| Strategy, OSINT, complex analysis | deep | all tiers |+| Brainstorm, "what if", ideation | creative | random from cold+archive |++## Search Order (All Modes)++1. **Index files first** — MOC or `_index.md` files are pre-built navigation. Check them before searching.+2. **Follow links** — If an index points to `[[crm/acme]]`, read the card directly. Don't search for "Acme."+3. **Semantic search** — Only when indexes don't cover the query.+4. **Grep** — Last resort for exact strings, IDs, phone numbers.++## Tier Filtering++### Pre-filter with grep+```bash+# Find all active cards+grep -rl "^tier: active" vault/crm/++# Find all cold+ cards for deep search+grep -rl "^tier: \(cold\|archive\)" vault/crm/++# Find high-relevance cards+grep -l "^relevance: 0\.9" vault/crm/+```++### Combine with semantic search+1. Grep for tier → get file list+2. Read relevant files → answer question++This is cheaper than searching all 400+ cards semantically.++## Heartbeat Protocol++Heartbeats are frequent, automated checks. Minimize cost:++1. Read state file (always loaded)+2. Check only `core` + `active` tier cards matching current projects+3. Skip warm/cold/archive entirely+4. Total reads: 0-5 files per heartbeat++## Normal Protocol++Default for user questions:++1. Check index/MOC for direct link+2. If not found: semantic search with default radius (active + warm)+3. Read top 3-5 results+4. Touch any card you read: `memory-engine.py touch <file>`++## Deep Protocol++For complex, multi-step analysis:++1. Check indexes+2. Semantic search across ALL tiers (no filtering)+3. Read all relevant results (up to 10-15 files)+4. Cross-reference cards for connections+5. Touch all read cards++## Creative Protocol++Deliberately non-directed. Goal: surface unexpected connections.++```bash+python3 memory-engine.py creative 5 vault/+```++1. Get 5 random cards from cold/archive+2. Read each one+3. For each card, ask: "How could this connect to my current task?"+4. If a connection exists → touch the card (promotes it back to active)+5. If no connection → leave it (continues to decay)++### When to use creative mode+- Stuck on a problem+- Looking for new business angles+- User says "brainstorm" or "what if"+- Exploring forgotten knowledge+- Weekly reflection sessions++### What makes it work+Human creativity often comes from random associations — shower thoughts, dreams, serendipitous encounters. Creative mode simulates this by pulling forgotten cards back into working memory. The randomness is the feature, not a bug.++## Touch Discipline++Touching a card resets its decay. Over-touching defeats the purpose.++**Touch when:**+- You read the card's content to answer a question+- You update the card with new information+- User explicitly discusses the entity++**Don't touch when:**+- Card appears in search results but you didn't open it+- Running automated scans or reports+- Bulk migration or formatting changes+- Card mentioned in passing but not substantively used++## Cost Awareness++| Mode | Typical reads | Token cost |+|------|---------------|------------|+| Heartbeat | 0-5 files | ~2K tokens |+| Normal | 3-8 files | ~5K tokens |+| Deep | 10-20 files | ~15K tokens |+| Creative | 5 files | ~3K tokens |++Optimize by reading index files first — they're cheap navigation that prevents expensive full-text searches.diff --git a/vault/.claude/skills/agent-memory/references/yaml-schema.md b/vault/.claude/skills/agent-memory/references/yaml-schema.mdnew file mode 100644index 0000000..0efab89--- /dev/null+++ b/vault/.claude/skills/agent-memory/references/yaml-schema.md@@ -0,0 +1,189 @@+---+type: note+title: YAML Frontmatter Schema+last_accessed: 2026-02-26+relevance: 1.0+tier: active+---+# YAML Frontmatter Schema++## Canonical Schema (all card types)++Every card MUST have this structure. Fields marked (auto) are managed by memory-engine.py.+Fields marked (required) must be written by the agent when creating a card.++```yaml+---+# ── identity ──+type: crm # (required) crm | lead | contact | project | personal | daily | note+description: >- # (required) One-line summary. What is this card about?+ Key client in [industry], active deal $XXK, [project description]++# ── classification ──+tags: [fmcg, active-deal, uz] # (required) 2-5 freeform tags for grep filtering+status: active # (required) universal: active|draft|pending|done|inactive; CRM-only: prospect|negotiation|won|lost+industry: FMCG # (optional) For CRM/leads+region: UZ # (optional) ISO country codes+source: referral # (optional) How this entity entered the system+priority: High # (optional) High | Medium | Low++# ── ownership ──+owner: [Your Name] # (optional) Who owns this relationship+responsible: [Your Name] # (optional) Who is doing the work++# ── dates ──+created: 2026-01-15 # (recommended) When card was first created+updated: 2026-02-20 # (recommended) When content was last meaningfully changed++# ── deal tracking ──+deal_status: negotiation # (optional) For active deals+deal_deadline: 2026-03-15 # (optional) Deal close date++# ── memory system (auto) ──+last_accessed: 2026-02-25 # (auto) When card was last read/touched+relevance: 0.85 # (auto) 0.0-1.0, decays over time+tier: active # (auto) core | active | warm | cold | archive+---+```++## Required Fields Explained++### `description` (string, one line)+The single most important field for search quality. Write a concise summary that answers:+"If someone searches for this entity, what should they see in results?"++Good: `"Event Specialist, [Company] [Country], tender 2026"`+Bad: `"контакт"` (too vague)+Bad: (empty — defeats the purpose of the entire system)++### `tags` (list, 2-5 items)+Cross-cutting labels for fast grep filtering. Use lowercase, hyphens.+```yaml+tags: [hot-lead, ai-training, enterprise, follow-up]+```+Search: `grep -rl "hot-lead" vault/crm/`++### `type` (enum)+| Type | When |+|------|------|+| crm | Existing client/company |+| lead | Potential client |+| contact | Person (not a lead/client) |+| project | Active or past project |+| personal | Family, friends |+| daily | Daily log file |+| note | Everything else |++### `status` (enum, normalized)+Domain-specific lifecycle. NOT the same as `tier` (which is memory-system lifecycle).++**Universal (all card types):**++| Status | Meaning |+|--------|---------|+| `active` | Currently relevant, in use, engaged |+| `draft` | Work in progress, not finalized |+| `pending` | Waiting for external input or decision |+| `done` | Completed, kept for reference |+| `inactive` | Was active, went quiet or outdated |++**CRM-specific (only for type: crm, lead, crm-lead, client):**++| Status | Meaning |+|--------|---------|+| `prospect` | Identified lead, no deep engagement yet |+| `negotiation` | Proposal/KP sent, in talks |+| `won` | Deal closed positively |+| `lost` | Rejected, didn't pursue, conflict |++**ONLY these 9 values.** No Russian, no mixed case, no free-text.++Typical lifecycle by type:+- **crm/lead:** prospect → active → negotiation → won/lost+- **project:** draft → active → done+- **contact:** active → inactive+- **note/knowledge:** draft → active → inactive (outdated)+- **personal:** active → inactive+- **daily:** no status needed (has `date` field)++When in doubt: CRM → `prospect`, everything else → `active`.++## Memory System Fields (auto-managed)++### `relevance` (float, 0.0-1.0)+Computed by decay engine. Do not manually edit unless marking as `core`.+- 1.0 = just accessed+- 0.5 = ~33 days old+- 0.1 = floor (60+ days)++### `tier` (enum)+Computed by decay engine based on `last_accessed`.+- `core` — only manually assigned, never auto-demoted. Use for: identity, security rules, pricing.+- `active` — 0-7 days since access. Searched in all modes.+- `warm` — 8-21 days. Searched in normal+ modes.+- `cold` — 22-60 days. Deep search only.+- `archive` — 60+ days. Creative mode or explicit queries.++### `last_accessed` (ISO date)+Updated by `touch` command (graduated: +1 tier per touch).++## Agent Protocol for New Cards++When creating ANY new card:+1. ALWAYS include: `type`, `description`, `tags`, `status`+2. Write `description` as if it's a search result snippet — concise, informative+3. Add 2-5 `tags` that cross-cut directory structure+4. Run `memory-engine.py touch <file>` after creation+5. The engine will auto-add `relevance`, `last_accessed`, `tier` on next decay run++## Field Semantics++### `relevance` (float, 0.0-1.0)+Computed by decay engine. Do not manually edit unless marking as `core`.+- 1.0 = just accessed+- 0.5 = ~33 days old+- 0.1 = floor (60+ days)++### `tier` (enum)+Computed by decay engine based on `last_accessed`.+- `core` — only manually assigned, never auto-demoted. Use for: identity, security rules, pricing, critical reference.+- `active` — 0-7 days since access. Searched in all modes.+- `warm` — 8-21 days. Searched in normal+ modes.+- `cold` — 22-60 days. Deep search only.+- `archive` — 60+ days. Creative mode or explicit queries.++### `last_accessed` (ISO date)+Updated by `touch` command when agent reads the card. Also updated by decay engine on first run (inferred from `updated`, `created`, git log, or file mtime).++### `type` (string)+Auto-inferred from directory path if not present. Configurable via `type_inference` in `.memory-config.json`.++### `tags` (list)+Freeform. Useful for cross-cutting concerns that don't fit directory structure.+```yaml+tags: [hot-lead, ai-training, enterprise, follow-up]+```+Search: `grep -rl "hot-lead" vault/crm/`++## Configuration++Type inference mapping in `.memory-config.json`:+```json+{+ "type_inference": {+ "crm/clients/": "crm",+ "crm/leads/": "lead",+ "crm/personal/": "personal",+ "projects/": "project"+ }+}+```++When `memory-engine.py init` encounters a file without `type`, it checks the file path against these patterns.++## Frontmatter Tips++1. **Don't duplicate content.** If the H1 heading says "# Acme Corp", you don't need `title: Acme Corp` — the engine infers it.+2. **Tags > nested directories.** A flat `crm/` with tags is more flexible than `crm/hot/enterprise/ai/`.+3. **Status is domain-specific.** The memory system uses `tier` for lifecycle; `status` is for your business logic (active/won/lost/churned).+4. **`updated` vs `last_accessed`**: `updated` = when content changed; `last_accessed` = when anyone read it. Both matter for decay; the engine uses whichever is most recent.diff --git a/vault/.claude/skills/agent-memory/scripts/memory-engine.py b/vault/.claude/skills/agent-memory/scripts/memory-engine.pynew file mode 100644index 0000000..6251022--- /dev/null+++ b/vault/.claude/skills/agent-memory/scripts/memory-engine.py@@ -0,0 +1,645 @@+#!/usr/bin/env python3+"""+memory-engine.py — Universal memory management for AI agents.++Implements Ebbinghaus-inspired forgetting curve with tiered recall.+Works with any directory of markdown files.++Commands:+ scan [dir] Analyze files, report stats (no changes)+ init [dir] Add YAML frontmatter to files missing it+ decay [dir] Update relevance scores and tiers+ touch <file> Reset file to active (on read/use)+ creative <N> [dir] Random N cards from cold/archive tiers+ stats [dir] Show tier distribution and health metrics++Options:+ --config <path> Config JSON (default: .memory-config.json in target dir)+ --dry-run Preview changes without writing+ --verbose Show per-file details++Config (.memory-config.json):+{+ "tiers": {+ "active": 7, // days threshold+ "warm": 21,+ "cold": 60+ // beyond cold = archive+ },+ "decay_rate": 0.015, // relevance loss per day (linear)+ "relevance_floor": 0.1, // minimum relevance+ "skip_patterns": ["_index.md", "MOC-*"],+ "type_inference": {+ "crm/": "crm",+ "leads/": "lead",+ "personal/": "personal"+ },+ "use_git_dates": true+}+"""++import os+import re+import sys+import json+import random+import subprocess+from datetime import datetime, date+from pathlib import Path+from typing import Optional++# ─── defaults ───────────────────────────────────────────────────++DEFAULT_CONFIG = {+ "tiers": {"active": 7, "warm": 21, "cold": 60},+ "decay_rate": 0.015,+ "relevance_floor": 0.1,+ "skip_patterns": ["_index.md"],+ "type_inference": {},+ "use_git_dates": True,+}++TODAY = date.today()++# ─── YAML frontmatter parsing ──────────────────────────────────++def parse_frontmatter(content: str) -> tuple[dict, str, bool]:+ """Parse YAML frontmatter. Returns (fields, body, had_yaml)."""+ if content.startswith("---\n"):+ end = content.find("\n---\n", 4)+ if end != -1:+ yaml_block = content[4:end]+ body = content[end + 5:]+ fields = {}+ for line in yaml_block.split("\n"):+ if ":" in line:+ key, _, val = line.partition(":")+ fields[key.strip()] = val.strip()+ return fields, body, True+ return {}, content, False+++def build_frontmatter(fields: dict, field_order: list[str] | None = None) -> str:+ """Build YAML frontmatter from dict, preserving field order."""+ if field_order is None:+ field_order = [+ "type", "title", "description", "tags",+ "industry", "source", "priority", "status", "region",+ "owner", "responsible", "domain", "related", "client",+ "deal_status", "deal_deadline", "deadline",+ "created", "updated", "last_accessed", "relevance", "tier",+ ]+ lines = []+ used = set()+ for key in field_order:+ if key in fields:+ lines.append(f"{key}: {fields[key]}")+ used.add(key)+ for key, val in fields.items():+ if key not in used:+ lines.append(f"{key}: {val}")+ return "---\n" + "\n".join(lines) + "\n---\n"+++# ─── date resolution ───────────────────────────────────────────++def get_git_date(filepath: Path) -> date | None:+ """Last git commit date for file."""+ try:+ result = subprocess.run(+ ["git", "log", "-1", "--format=%aI", "--", str(filepath)],+ capture_output=True, text=True,+ cwd=filepath.parent, timeout=5,+ )+ if result.returncode == 0 and result.stdout.strip():+ return date.fromisoformat(result.stdout.strip()[:10])+ except Exception:+ pass+ return None+++def get_best_date(fields: dict, filepath: Path, use_git: bool = True) -> date:+ """Most recent date from YAML fields, git history, or file mtime."""+ candidates = []+ for field in ["last_accessed", "updated", "created"]:+ val = fields.get(field, "")+ try:+ candidates.append(date.fromisoformat(val[:10]))+ except (ValueError, IndexError):+ continue+ if use_git:+ git_date = get_git_date(filepath)+ if git_date:+ candidates.append(git_date)+ if not candidates:+ mtime = os.path.getmtime(filepath)+ candidates.append(date.fromtimestamp(mtime))+ return max(candidates)+++# ─── core logic ─────────────────────────────────────────────────++def calc_relevance(days: int, rate: float, floor: float) -> float:+ """Linear decay with floor."""+ return round(max(floor, 1.0 - days * rate), 2)+++def calc_tier(days: int, tiers: dict, current_tier: str = "") -> str:+ """Assign tier based on days since last access."""+ if current_tier == "core":+ return "core" # never auto-demote core+ sorted_tiers = sorted(tiers.items(), key=lambda x: x[1])+ for tier_name, threshold in sorted_tiers:+ if days <= threshold:+ return tier_name+ return "archive"+++def infer_type(filepath: Path, type_map: dict) -> str:+ """Infer card type from path using configurable mapping."""+ path_str = str(filepath)+ for pattern, card_type in type_map.items():+ if pattern in path_str:+ return card_type+ return "note"+++def should_skip(filepath: Path, patterns: list[str]) -> bool:+ """Check if file matches skip patterns."""+ import fnmatch+ name = filepath.name+ for pattern in patterns:+ if fnmatch.fnmatch(name, pattern):+ return True+ return False+++def infer_title(body: str) -> str:+ """Extract title from first H1 heading."""+ for line in body.split("\n")[:10]:+ if line.startswith("# "):+ return line[2:].strip()+ return ""+++# ─── config ─────────────────────────────────────────────────────++def load_config(target_dir: Path, config_path: str | None = None) -> dict:+ """Load config from file or return defaults."""+ if config_path:+ p = Path(config_path)+ else:+ p = target_dir / ".memory-config.json"+ if p.exists():+ with open(p) as f:+ user = json.load(f)+ # Merge with defaults+ config = {**DEFAULT_CONFIG, **user}+ config["tiers"] = {**DEFAULT_CONFIG["tiers"], **user.get("tiers", {})}+ return config+ return DEFAULT_CONFIG.copy()+++def save_default_config(target_dir: Path):+ """Write default config file."""+ p = target_dir / ".memory-config.json"+ with open(p, "w") as f:+ json.dump(DEFAULT_CONFIG, f, indent=2)+ print(f" wrote default config: {p}")+++# ─── commands ───────────────────────────────────────────────────++def find_cards(target_dir: Path, config: dict) -> list[Path]:+ """Find all markdown files, respecting skip patterns."""+ cards = sorted(target_dir.rglob("*.md"))+ return [c for c in cards if c.exists() and not c.is_symlink() and not should_skip(c, config["skip_patterns"])]+++def cmd_scan(target_dir: Path, config: dict, verbose: bool = False):+ """Analyze files without changes."""+ cards = find_cards(target_dir, config)+ with_yaml = 0+ without_yaml = 0+ has_relevance = 0+ has_tier = 0+ total_bytes = 0++ for card in cards:+ content = card.read_text(encoding="utf-8", errors="replace")+ total_bytes += len(content.encode("utf-8"))+ fields, body, had_yaml = parse_frontmatter(content)+ if had_yaml:+ with_yaml += 1+ else:+ without_yaml += 1+ if "relevance" in fields:+ has_relevance += 1+ if "tier" in fields:+ has_tier += 1+ if verbose and not had_yaml:+ print(f" no yaml: {card.relative_to(target_dir)}")++ print(f"\n scan results for {target_dir}:")+ print(f" total files: {len(cards)}")+ print(f" with yaml: {with_yaml}")+ print(f" without yaml: {without_yaml}")+ print(f" has relevance: {has_relevance}")+ print(f" has tier: {has_tier}")+ print(f" total size: {total_bytes / 1024:.0f} KB")+ print(f" avg file size: {total_bytes / max(len(cards), 1) / 1024:.1f} KB")++ if without_yaml:+ print(f"\n → run 'init' to add YAML frontmatter to {without_yaml} files")+ if not has_relevance:+ print(f" → run 'decay' to add relevance scores and tiers")+++def cmd_init(target_dir: Path, config: dict, dry_run: bool = False, verbose: bool = False):+ """Add YAML frontmatter to files missing it."""+ cards = find_cards(target_dir, config)+ added = 0++ for card in cards:+ content = card.read_text(encoding="utf-8", errors="replace")+ fields, body, had_yaml = parse_frontmatter(content)+ if had_yaml:+ continue++ # Create minimal frontmatter+ new_fields = {+ "type": infer_type(card, config["type_inference"]),+ }+ title = infer_title(body)+ if title:+ new_fields["title"] = title++ ref_date = get_best_date({}, card, config["use_git_dates"])+ new_fields["last_accessed"] = ref_date.isoformat()+ days = max(0, (TODAY - ref_date).days)+ new_fields["relevance"] = str(calc_relevance(days, config["decay_rate"], config["relevance_floor"]))+ new_fields["tier"] = calc_tier(days, config["tiers"])++ new_content = build_frontmatter(new_fields) + body+ if not dry_run:+ card.write_text(new_content, encoding="utf-8")+ added += 1+ if verbose:+ print(f" {'[dry] ' if dry_run else ''}init: {card.relative_to(target_dir)} → {new_fields['tier']}")++ print(f"\n {'DRY RUN — ' if dry_run else ''}init results:")+ print(f" files processed: {len(cards)}")+ print(f" frontmatter added: {added}")+++def cmd_decay(target_dir: Path, config: dict, dry_run: bool = False, verbose: bool = False):+ """Update relevance and tiers based on time decay."""+ cards = find_cards(target_dir, config)+ results = []++ for card in cards:+ content = card.read_text(encoding="utf-8", errors="replace")+ fields, body, had_yaml = parse_frontmatter(content)++ ref_date = get_best_date(fields, card, config["use_git_dates"])+ days = max(0, (TODAY - ref_date).days)++ old_tier = fields.get("tier", "")+ new_relevance = calc_relevance(days, config["decay_rate"], config["relevance_floor"])+ new_tier = calc_tier(days, config["tiers"], old_tier)++ if "last_accessed" not in fields:+ fields["last_accessed"] = ref_date.isoformat()+ if "type" not in fields:+ fields["type"] = infer_type(card, config["type_inference"])++ fields["relevance"] = str(new_relevance)+ fields["tier"] = new_tier++ new_content = build_frontmatter(fields) + body+ changed = new_content != content++ if changed and not dry_run:+ card.write_text(new_content, encoding="utf-8")++ results.append({+ "path": str(card.relative_to(target_dir)),+ "days": days,+ "relevance": new_relevance,+ "tier": new_tier,+ "changed": changed,+ })++ if verbose and changed:+ print(f" {'[dry] ' if dry_run else ''}{card.relative_to(target_dir)}: {old_tier or '?'}→{new_tier} r={new_relevance}")++ # Stats+ tiers = {}+ changed_count = sum(1 for r in results if r["changed"])+ for r in results:+ tiers[r["tier"]] = tiers.get(r["tier"], 0) + 1+ avg_rel = sum(r["relevance"] for r in results) / max(len(results), 1)++ print(f"\n {'DRY RUN — ' if dry_run else ''}decay results:")+ print(f" total: {len(results)}, changed: {changed_count}")+ print(f" avg relevance: {avg_rel:.2f}")+ for tier in ["core", "active", "warm", "cold", "archive"]:+ count = tiers.get(tier, 0)+ bar = "█" * (count // 3)+ if count:+ print(f" {tier:8s}: {count:4d} {bar}")+++def cmd_touch(filepath: str, config: dict):+ """Promote a file one tier up (graduated recall).++ archive → cold → warm → active → active (refresh)+ Each touch promotes one level, not straight to top.+ Natural spaced repetition: multiple reads = stronger memory.+ """+ p = Path(filepath)+ if not p.exists():+ print(f" error: {filepath} not found")+ sys.exit(1)++ content = p.read_text(encoding="utf-8", errors="replace")+ fields, body, had_yaml = parse_frontmatter(content)++ if fields.get("tier") == "core":+ fields["last_accessed"] = TODAY.isoformat()+ fields["relevance"] = "1.0"+ new_content = build_frontmatter(fields) + body+ p.write_text(new_content, encoding="utf-8")+ print(f" touched: {filepath} → core (refreshed)")+ return++ tiers_cfg = config["tiers"]+ # Promotion targets: set last_accessed to midpoint of next-higher tier+ # archive → cold: midpoint of cold range+ # cold → warm: midpoint of warm range+ # warm → active: midpoint of active range+ # active → active: today (refresh)+ cold_threshold = tiers_cfg.get("cold", 60)+ warm_threshold = tiers_cfg.get("warm", 21)+ active_threshold = tiers_cfg.get("active", 7)++ current_tier = fields.get("tier", "archive")+ if current_tier == "archive":+ # Promote to cold: set last_accessed to midpoint of cold range+ target_days = (warm_threshold + cold_threshold) // 2+ new_tier = "cold"+ elif current_tier == "cold":+ # Promote to warm: midpoint of warm range+ target_days = (active_threshold + warm_threshold) // 2+ new_tier = "warm"+ elif current_tier == "warm":+ # Promote to active: midpoint of active range+ target_days = active_threshold // 2+ new_tier = "active"+ else:+ # Already active: refresh to today+ target_days = 0+ new_tier = "active"++ from datetime import timedelta+ new_date = TODAY - timedelta(days=target_days)+ new_relevance = calc_relevance(target_days, config["decay_rate"], config["relevance_floor"])++ fields["last_accessed"] = new_date.isoformat()+ fields["relevance"] = str(new_relevance)+ fields["tier"] = new_tier++ new_content = build_frontmatter(fields) + body+ p.write_text(new_content, encoding="utf-8")+ print(f" touched: {filepath} → {current_tier}→{new_tier}, relevance={new_relevance}")+++def cmd_creative(n: int, target_dir: Path, config: dict):+ """Random sample from cold/archive tiers for divergent thinking."""+ cards = find_cards(target_dir, config)+ cold_cards = []++ for card in cards:+ content = card.read_text(encoding="utf-8", errors="replace")+ fields, body, had_yaml = parse_frontmatter(content)+ tier = fields.get("tier", "")+ if tier in ("cold", "archive", "warm"):+ title = fields.get("title", "") or infer_title(body)+ cold_cards.append({+ "path": str(card.relative_to(target_dir)),+ "tier": tier,+ "relevance": fields.get("relevance", "?"),+ "title": title,+ "last_accessed": fields.get("last_accessed", "?"),+ })++ if not cold_cards:+ print(" no cold/archive cards found — memory is too fresh")+ return++ sample = random.sample(cold_cards, min(n, len(cold_cards)))+ print(f"\n 🎲 creative recall — {len(sample)} random cards:")+ for card in sample:+ print(f" [{card['tier']}] {card['title'] or card['path']}")+ print(f" {card['path']} (r={card['relevance']}, last={card['last_accessed']})")+ print(f"\n read these cards and look for unexpected connections to your current task")+++def cmd_daily(target_dir: Path, config: dict, dry_run: bool = False, verbose: bool = False):+ """Bootstrap and decay daily files (YYYY-MM-DD.md pattern)."""+ date_pattern = re.compile(r"^\d{4}-\d{2}-\d{2}\.md$")+ daily_files = sorted(+ f for f in target_dir.rglob("*.md")+ if date_pattern.match(f.name)+ )++ if not daily_files:+ print(f" no daily files (YYYY-MM-DD.md) found in {target_dir}")+ return++ results = []+ for f in daily_files:+ content = f.read_text(encoding="utf-8", errors="replace")+ fields, body, had_yaml = parse_frontmatter(content)++ # Extract date from filename+ file_date = date.fromisoformat(f.stem)+ days = max(0, (TODAY - file_date).days)++ # Use file_date as reference (not git/mtime — daily files are date-intrinsic)+ la = fields.get("last_accessed", "")+ try:+ last_acc = date.fromisoformat(la[:10])+ # Use most recent of: file_date, last_accessed+ ref_days = max(0, (TODAY - max(file_date, last_acc)).days)+ except (ValueError, IndexError):+ ref_days = days++ new_relevance = calc_relevance(ref_days, config["decay_rate"], config["relevance_floor"])+ old_tier = fields.get("tier", "")+ new_tier = calc_tier(ref_days, config["tiers"], old_tier)++ fields["type"] = "daily"+ fields["date"] = file_date.isoformat()+ if "last_accessed" not in fields:+ fields["last_accessed"] = file_date.isoformat()+ fields["relevance"] = str(new_relevance)+ fields["tier"] = new_tier++ new_content = build_frontmatter(fields) + body+ changed = new_content != content++ if changed and not dry_run:+ f.write_text(new_content, encoding="utf-8")++ results.append({+ "file": f.name,+ "date": file_date.isoformat(),+ "days": ref_days,+ "relevance": new_relevance,+ "tier": new_tier,+ "changed": changed,+ })++ if verbose:+ print(f" {'[dry] ' if dry_run else ''}{f.name}: {old_tier or '?'}→{new_tier} r={new_relevance} ({ref_days}d)")++ # Summary+ tiers = {}+ for r in results:+ tiers[r["tier"]] = tiers.get(r["tier"], 0) + 1+ changed_count = sum(1 for r in results if r["changed"])++ print(f"\n {'DRY RUN — ' if dry_run else ''}daily results:")+ print(f" files: {len(results)}, changed: {changed_count}")+ for tier in ["active", "warm", "cold", "archive"]:+ count = tiers.get(tier, 0)+ if count:+ dates = [r["date"] for r in results if r["tier"] == tier]+ print(f" {tier:8s}: {count:3d} ({dates[0]}..{dates[-1]})")+++def cmd_stats(target_dir: Path, config: dict):+ """Show comprehensive memory health stats."""+ cards = find_cards(target_dir, config)+ tiers = {}+ total_bytes = 0+ stale_count = 0+ no_yaml = 0++ for card in cards:+ content = card.read_text(encoding="utf-8", errors="replace")+ total_bytes += len(content.encode("utf-8"))+ fields, body, had_yaml = parse_frontmatter(content)+ if not had_yaml:+ no_yaml += 1+ tier = fields.get("tier", "unknown")+ tiers[tier] = tiers.get(tier, 0) + 1+ try:+ la = date.fromisoformat(fields.get("last_accessed", "")[:10])+ if (TODAY - la).days > 90:+ stale_count += 1+ except (ValueError, IndexError):+ pass++ print(f"\n memory health — {target_dir}")+ print(f" {'─' * 40}")+ print(f" total cards: {len(cards)}")+ print(f" total size: {total_bytes / 1024:.0f} KB")+ print(f" without yaml: {no_yaml}")+ print(f" stale (>90 days): {stale_count}")+ print(f" {'─' * 40}")+ print(f" tier distribution:")+ for tier in ["core", "active", "warm", "cold", "archive", "unknown"]:+ count = tiers.get(tier, 0)+ if count:+ pct = count / len(cards) * 100+ bar = "█" * int(pct / 2)+ print(f" {tier:8s}: {count:4d} ({pct:4.1f}%) {bar}")++ # Context budget estimate (assuming ~4 chars per token)+ active_bytes = 0+ for card in cards:+ content = card.read_text(encoding="utf-8", errors="replace")+ fields, _, _ = parse_frontmatter(content)+ if fields.get("tier") in ("core", "active"):+ active_bytes += len(content.encode("utf-8"))+ print(f" {'─' * 40}")+ print(f" active context: {active_bytes / 1024:.0f} KB (~{active_bytes // 4:,} tokens)")+ print(f" total context: {total_bytes / 1024:.0f} KB (~{total_bytes // 4:,} tokens)")+++# ─── main ───────────────────────────────────────────────────────++def main():+ args = sys.argv[1:]+ if not args or args[0] in ("-h", "--help"):+ print(__doc__)+ sys.exit(0)++ cmd = args[0]+ dry_run = "--dry-run" in args+ verbose = "--verbose" in args++ # Find config path+ config_path = None+ if "--config" in args:+ idx = args.index("--config")+ config_path = args[idx + 1] if idx + 1 < len(args) else None++ # Find target directory (first non-flag argument after command)+ target = None+ for a in args[1:]:+ if not a.startswith("-") and a != config_path:+ target = a+ break++ if cmd == "touch":+ if not target:+ print(" error: touch requires a file path")+ sys.exit(1)+ config = load_config(Path(target).parent, config_path)+ cmd_touch(target, config)+ return++ if cmd == "creative":+ n = 5+ creative_dir = None+ for a in args[1:]:+ if not a.startswith("-"):+ try:+ n = int(a)+ except ValueError:+ creative_dir = a+ target_dir = Path(creative_dir) if creative_dir else Path(".")+ config = load_config(target_dir, config_path)+ cmd_creative(n, target_dir, config)+ return++ target_dir = Path(target) if target else Path(".")+ if not target_dir.is_dir():+ print(f" error: {target_dir} is not a directory")+ sys.exit(1)++ config = load_config(target_dir, config_path)++ if cmd == "scan":+ cmd_scan(target_dir, config, verbose)+ elif cmd == "init":+ cmd_init(target_dir, config, dry_run, verbose)+ elif cmd == "decay":+ cmd_decay(target_dir, config, dry_run, verbose)+ elif cmd == "daily":+ cmd_daily(target_dir, config, dry_run, verbose)+ elif cmd == "stats":+ cmd_stats(target_dir, config)+ elif cmd == "config":+ save_default_config(target_dir)+ else:+ print(f" unknown command: {cmd}")+ print(" commands: scan, init, decay, daily, touch, creative, stats, config")+ sys.exit(1)+++if __name__ == "__main__":+ main()diff --git a/vault/.memory-config.json b/vault/.memory-config.jsonnew file mode 100644index 0000000..ac57f3c--- /dev/null+++ b/vault/.memory-config.json@@ -0,0 +1,13 @@+{+ "tiers": {"active": 7, "warm": 21, "cold": 60},+ "decay_rate": 0.015,+ "relevance_floor": 0.1,+ "skip_patterns": ["_index.md"],+ "type_inference": {+ "crm/": "crm",+ "leads/": "lead",+ "clients/": "client",+ "projects/": "project"+ },+ "use_git_dates": true+}