Skip to content
Shell
23.0 KB
Raw Blame

The Longform Guide to Everything Claude Code

Source: cogsec (@affaanmustafa) - Jan 2026 Status: Active - Core operational patterns Last Updated: 2026-01-21

This guide covers advanced Claude Code techniques for token economics, memory persistence, verification patterns, parallelization strategies, and compound effects of building reusable workflows.

Table of Contents


Context & Memory Management

Session Storage Pattern

  • Store session logs in ~/.claude/sessions/YYYY-MM-DD-topic.tmp
  • One file per session with: current state, completed items, blockers, key decisions
  • At session end, summarize and checkpoint progress
  • Next session: load previous file for continuity

Example structure:

# Session: 2026-01-21 - API Refactor

## Current State
- Moved from REST to GraphQL for orders endpoint
- 3/5 entities migrated

## Completed
- [x] User entity migration
- [x] Product entity migration
- [x] Order entity migration

## Blockers
- Need schema validation strategy
- Auth middleware needs GraphQL context

## Key Decisions
- Using DataLoader for N+1 prevention
- Separate schema files per domain module

## Next Session
- Migrate Payment and Shipping entities
- Implement field-level auth directives

Clearing Context Strategically

Default (Auto-Compact): Happens at arbitrary points, often mid-task Strategic (Manual): Compact at logical phase transitions

Strategic Compact Hook (PreToolUse):

#!/bin/bash
# ~/.claude/hooks/strategic-compact.sh
COUNTER_FILE="/tmp/claude-tool-count-$$"
THRESHOLD=${COMPACT_THRESHOLD:-50}

if [ -f "$COUNTER_FILE" ]; then
  count=$(cat "$COUNTER_FILE")
  count=$((count + 1))
  echo "$count" > "$COUNTER_FILE"
else
  echo "1" > "$COUNTER_FILE"
  count=1
fi

if [ "$count" -eq "$THRESHOLD" ]; then
  echo "[StrategicCompact] $THRESHOLD tool calls reached - consider /compact if transitioning phases" >&2
fi

When to compact:

  • After exploration, before execution
  • After completing a milestone, before starting next
  • When transitioning project phases

Dynamic System Prompt Injection

Basic pattern:

claude --system-prompt "$(cat memory.md)"

Why vs @ file references:

  • @memory.md = Read tool output (lower authority)
  • --system-prompt = Injected before conversation (higher authority)

Hierarchy: System Prompt > User Messages > Tool Results

Practical aliases:

# Daily development
alias claude-dev='claude --system-prompt "$(cat ~/.claude/contexts/dev.md)"'

# PR review mode
alias claude-review='claude --system-prompt "$(cat ~/.claude/contexts/review.md)"'

# Research/exploration mode
alias claude-research='claude --system-prompt "$(cat ~/.claude/contexts/research.md)"'

Memory Persistence Hooks

Hook lifecycle:

SESSION 1                              SESSION 2
─────────                              ─────────

[Start]                                [Start]
   │                                      │
   ▼                                      ▼
┌──────────────┐                    ┌──────────────┐
│ SessionStart │ ◄─── reads ─────── │ SessionStart │◄── loads previous
│    Hook      │     nothing yet    │    Hook      │    context
└──────┬───────┘                    └──────┬───────┘
       │                                   │
       ▼                                   ▼
   [Working]                           [Working]
       │                               (informed)
       ▼                                   │
┌──────────────┐                           ▼
│  PreCompact  │──► saves state       [Continue...]
│    Hook      │    before summary
└──────┬───────┘
       │
       ▼
   [Compacted]
       │
       ▼
┌──────────────┐
│  Stop Hook   │──► persists to ──────────►
│ (session-end)│    ~/.claude/sessions/
└──────────────┘

Hook configuration:

{
  "hooks": {
    "PreCompact": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "~/.claude/hooks/memory-persistence/pre-compact.sh"
      }]
    }],
    "SessionStart": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "~/.claude/hooks/memory-persistence/session-start.sh"
      }]
    }],
    "Stop": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "~/.claude/hooks/memory-persistence/session-end.sh"
      }]
    }]
  }
}

Continuous Learning Pattern

Problem: Repeated prompts for same mistakes, wasted tokens/context/time

Solution: Automatically save discovered patterns as skills

Why Stop hook vs UserPromptSubmit:

  • UserPromptSubmit runs on EVERY message (high overhead, latency)
  • Stop runs once at session end (lightweight, evaluates complete session)

Pattern:

  1. Stop hook triggers at session end
  2. Script analyzes session for extractable knowledge
  3. Saves patterns as skills in ~/.claude/skills/learned/
  4. Next session: skills load automatically via semantic matching

Manual extraction with /learn:

  • Run mid-session when solving something non-trivial
  • Prompts for pattern extraction, drafts skill file
  • Confirmation before saving

Other Self-Improving Patterns:

  • Reflection over logs: Build "diary" of what works/fails (RLanceMartin)
  • Proactive suggestions: Agent suggests improvements every 15 min (alexhillman)

Token Optimization

Primary Strategy: Subagent Architecture

Model Selection Quick Reference:

Task Type Model Why
Repetitive, clear instructions Haiku 5x cheaper than Opus, sufficient
Default coding tasks Sonnet 66.7% cheaper than Opus, minimal quality loss
5+ file changes, architecture, security-critical Opus Complex reasoning needed
"Worker" in multi-agent setup Haiku Saves cost on simple subtasks

Cost comparison:

  • Opus: $5 input / $25 output per M tokens
  • Sonnet: $3 input / $15 output per M tokens (1.67x savings)
  • Haiku: ~$1 input / ~$5 output per M tokens (5x savings vs Opus)

Agent definition with model:

---
name: quick-search
description: Fast file search
tools: Glob, Grep
model: haiku  # Cheap and fast
---

Tool-Specific Optimizations

Replace grep with mgrep:

Background processes:

  • Run outside Claude if full streaming output not needed
  • Use tmux for session management
  • Summarize or copy only relevant parts
  • Saves input tokens (where majority of cost comes from)

Modular Codebase Benefits

Problem: Long files (1000+ lines) require:

  • Multiple Read tool calls to finish
  • Claude loses information during reads
  • Stopping and rereading costs extra tokens

Solution: Modular architecture

  • Main files: hundreds of lines, not thousands
  • Reusable utilities, functions, hooks
  • Self-contained modules

Example structure:

root/
├── docs/
├── scripts/
├── src/
│   ├── apps/               # Entry points
│   │   ├── api-gateway/
│   │   └── cron-jobs/
│   ├── modules/            # Core business logic
│   │   ├── ordering/       # Self-contained
│   │   │   ├── api/
│   │   │   ├── domain/
│   │   │   ├── infrastructure/
│   │   │   ├── use-cases/
│   │   │   └── tests/
│   │   ├── catalog/
│   │   └── identity/
│   ├── shared/             # Cross-module utilities
│   │   ├── kernel/
│   │   ├── events/
│   │   └── utils/
│   └── main.ts
└── tests/

Benefits:

  • Cheaper token cost (less to read)
  • Higher first-try success rate
  • Easier for Claude to understand context

Lean Codebase = Cheaper Tokens

Continuous cleanup strategies:

  • Use skills to identify dead code
  • Regular refactoring passes
  • Manual codebase skimming for patterns
  • Feed context + refactor/dead-code skills to Claude

System Prompt Slimming (Advanced)

Current overhead: 18k tokens (9% of 200k context) With patches: ~10k tokens (saves ~7,300 tokens = 41% of static overhead)

Note: Most users don't need this optimization

  • Source: YK's system-prompt-patches

Verification Loops and Evals

Observability Methods

Option 1: tmux tracing

  • Hook tmux processes to trace thinking stream
  • Log output whenever skill triggers

Option 2: PostToolUse hook

  • Logs exact changes and outputs
  • Compare with/without skill benchmarks

Benchmarking workflow:

                    [Same Task]
                         │
            ┌────────────┴────────────┐
            ▼                         ▼
    ┌───────────────┐         ┌───────────────┐
    │  Worktree A   │         │  Worktree B   │
    │  WITH skill   │         │ WITHOUT skill │
    └───────┬───────┘         └───────┬───────┘
            │                         │
            ▼                         ▼
       [Output A]                [Output B]
            │                         │
            └──────────┬──────────────┘
                       ▼
                  [git diff]
                       │
                       ▼
              ┌────────────────┐
              │ Compare logs,  │
              │ token usage,   │
              │ output quality │
              └────────────────┘

Eval Pattern Types

CHECKPOINT-BASED vs CONTINUOUS:

CHECKPOINT-BASED                         CONTINUOUS
─────────────────                        ──────────

  [Task 1]                                 [Work]
     │                                        │
     ▼                                        ▼
  ┌─────────┐                            ┌─────────┐
  │Checkpoint│◄── verify                 │ Timer/  │
  │   #1    │    criteria                │ Change  │
  └────┬────┘                            └────┬────┘
       │ pass?                                │
   ┌───┴───┐                                  ▼
   │       │                            ┌──────────┐
  yes     no ──► fix ──┐                │Run Tests │
   │              │    │                │  + Lint  │
   ▼              └────┘                └────┬─────┘
  [Task 2]                                   │
     │                                  ┌────┴────┐
     ▼                                  │         │
  ┌─────────┐                          pass     fail
  │Checkpoint│                          │         │
  │   #2    │                           ▼         ▼
  └────┬────┘                        [Continue] [Stop & Fix]
       │                                          │
      ...                                    └────┘

Best for: Linear workflows              Best for: Long sessions
with clear milestones                   exploratory refactoring

Checkpoint-Based Evals:

  • Set explicit checkpoints in workflow
  • Verify against defined criteria at each
  • Fix before proceeding if verification fails
  • Good for: Linear workflows with clear milestones

Continuous Evals:

  • Run every N minutes or after major changes
  • Full test suite, build status, lint
  • Report regressions immediately
  • Stop and fix before continuing
  • Good for: Long-running sessions, exploratory work

Grader Types (from Anthropic)

Code-Based Graders:

  • String match, binary tests, static analysis
  • Fast, cheap, objective
  • Brittle to valid variations

Model-Based Graders:

  • Rubric scoring, natural language assertions
  • Flexible, handles nuance
  • Non-deterministic, more expensive

Human Graders:

  • SME review, crowdsourced judgment
  • Gold standard quality
  • Expensive and slow

Key Metrics

pass@k: At least ONE of k attempts succeeds

k=1: 70%  k=3: 91%  k=5: 97%
Higher k = higher odds of success

pass^k: ALL k attempts must succeed

k=1: 70%  k=3: 34%  k=5: 17%
Higher k = harder (consistency)

When to use:

  • pass@k: Just need it to work, any success counts
  • pass^k: Consistency essential, deterministic output quality

Building an Eval Roadmap

  1. Start early: 20-50 simple tasks from real failures
  2. Convert failures: User-reported issues → test cases
  3. Write unambiguous tasks: Two experts reach same verdict
  4. Build balanced sets: Test when behavior should AND shouldn't occur
  5. Build robust harness: Each trial starts from clean environment
  6. Grade outcomes: What agent produced, not the path it took
  7. Read transcripts: Review many trials for patterns
  8. Monitor saturation: 100% pass rate means add more tests

Parallelization

Philosophy

Goal: Maximum productivity with minimum viable parallelization

Avoid:

  • Arbitrary terminal counts (e.g., "always run 5 instances")
  • Over-parallelization before mastering single instance
  • Overlapping code changes without git worktrees

Preferred Pattern:

  • Main chat: Code changes
  • Forks: Questions, research, documentation scraping, external service queries

For Newcomers

Start with: Single instance, master workflow first

Scale when:

  • True necessity and purpose exist
  • Can't achieve task with script or tmux stream
  • Clear separation of concerns between instances

Most work: Usually 2-3 instances, rarely need more than 4

When Scaling Instances

If working on overlapping code:

  1. Use git worktrees:
git worktree add ../project-feature-a feature-a
git worktree add ../project-feature-b feature-b
git worktree add ../project-refactor refactor-branch

# Each worktree gets its own Claude instance
cd ../project-feature-a && claude

Benefits:

  • No git conflicts between instances
  • Each has clean working directory
  • Easy to compare outputs
  • Can benchmark same task with different approaches
  1. Use /rename for clarity:
/rename "Feature A: Auth Refactor"
/rename "Feature B: API Migration"
/rename "Refactor: Module Consolidation"

The Cascade Method

Organization pattern:

  1. Open new tasks in new tabs to the right
  2. Sweep left to right, oldest to newest
  3. Maintain consistent direction flow
  4. Check on specific tasks as needed
  5. Focus on at most 3-4 tasks simultaneously

Mental overhead increases faster than productivity beyond 3-4 tasks


Groundwork

The Two-Instance Kickoff Pattern

For new repos, start with 2 Claude instances:

Instance 1: Scaffolding Agent

  • Lay down scaffold and groundwork
  • Create project structure
  • Set up configs (CLAUDE.md, rules, agents)
  • Establish conventions
  • Get skeleton in place

Instance 2: Deep Research Agent

  • Connect to services, web search, etc.
  • Create detailed PRD
  • Create architecture mermaid diagrams
  • Compile references with actual documentation clips

Workflow:

  • Left terminal: Coding
  • Right terminal: Questions
  • Use /rename and /fork to organize

Starting Setup Philosophy

Start minimal:

  • Don't over-engineer from the start
  • Context7 / link scraping when already knee-deep
  • Build reusable patterns as you go

llms.txt Pattern

Many docs now provide LLM-optimized versions:

https://www.helius.dev/docs/llms.txt

Usage:

  • Navigate to docs page
  • Try /llms.txt at end of URL
  • Feed clean, optimized docs directly to Claude

Philosophy: Build Reusable Patterns

Insight from @omarsar0:

"Early on, I spent time building reusable workflows/patterns. Tedious to build, but this had a wild compounding effect as models and agent harnesses improved."

Invest in:

  • Subagents
  • Skills
  • Commands
  • Planning patterns
  • MCP tools
  • Context engineering patterns

Why it compounds:

  • Patterns transfer across model upgrades
  • Work with other agents (e.g., Codex)
  • Investment in patterns > investment in model tricks

Best Practices for Agents & Sub-Agents

The Sub-Agent Context Problem

Issue: Sub-agents lack semantic context

Analogy (@PerceptualPeak):

"Your boss sends you to a meeting and asks for a summary. You come back and give him the rundown. Nine times out of ten, he's going to have follow-up questions. Your summary won't include everything he needs because you don't have the implicit context he has."

Root cause:

  • Orchestrator has PURPOSE/REASONING behind request
  • Sub-agent only knows literal query
  • Summaries miss key details

Iterative Retrieval Pattern

┌─────────────────┐
│  ORCHESTRATOR   │
│  (has context)  │
└────────┬────────┘
         │ dispatch with query + objective
         ▼
┌─────────────────┐
│   SUB-AGENT     │
│ (lacks context) │
└────────┬────────┘
         │ returns summary
         ▼
┌─────────────────┐      ┌─────────────┐
│   EVALUATE      │─no──►│  FOLLOW-UP  │
│   Sufficient?   │      │  QUESTIONS  │
└────────┬────────┘      └──────┬──────┘
         │ yes                  │
         ▼                      │ sub-agent
    [ACCEPT]              fetches answers
                                │
         ◄──────────────────────┘
              (max 3 cycles)

Fix:

  1. Orchestrator evaluates every sub-agent return
  2. Asks follow-up questions before accepting
  3. Sub-agent goes back to source, gets answers
  4. Loop until sufficient (max 3 cycles to prevent infinite loops)
  5. Pass objective context, not just query

Pattern: Orchestrator with Sequential Phases

Phase 1: RESEARCH (use Explore agent)
- Gather context
- Identify patterns
- Output: research-summary.md

Phase 2: PLAN (use planner agent)
- Read research-summary.md
- Create implementation plan
- Output: plan.md

Phase 3: IMPLEMENT (use tdd-guide agent)
- Read plan.md
- Write tests first
- Implement code
- Output: code changes

Phase 4: REVIEW (use code-reviewer agent)
- Review all changes
- Output: review-comments.md

Phase 5: VERIFY (use build-error-resolver if needed)
- Run tests
- Fix issues
- Output: done or loop back

Key rules:

  1. Each agent gets ONE clear input, produces ONE clear output
  2. Outputs become inputs for next phase
  3. Never skip phases - each adds value
  4. Use /clear between agents to keep context fresh
  5. Store intermediate outputs in files (not just memory)

Agent Abstraction Tierlist (from @menhguin)

Tier 1: Direct Buffs (Easy to Use)

  1. Subagents

    • Direct buff for preventing context rot
    • Ad-hoc specialization
    • Half as useful as multi-agent but MUCH less complexity
  2. Metaprompting

    • "I take 3 minutes to prompt a 20-minute task"
    • Improves stability and sanity-checks assumptions
  3. Asking user more at beginning

    • Generally a buff
    • Must answer questions in plan mode

Tier 2: High Skill Floor (Harder to Use Well)

  1. Long-running agents

    • Need to understand shape/tradeoff of 15min vs 1.5hr vs 4hr tasks
    • Requires tweaking
    • Long trial-and-error
  2. Parallel multi-agent

    • Very high variance
    • Only useful on highly complex OR well-segmented tasks
    • "If 2 tasks take 10 minutes and you spend arbitrary time prompting or merging changes, it's counterproductive"
  3. Role-based multi-agent

    • "Models evolve too fast for hard-coded heuristics unless arbitrage is very high"
    • Hard to test
  4. Computer use agents

    • Very early paradigm, requires wrangling
    • "Getting models to do something they weren't meant to do a year ago"

Takeaway: Start with Tier 1 patterns. Graduate to Tier 2 only after mastering basics with genuine need.


Tips and Tricks

Some MCPs are Replaceable

Problem: MCPs eat context window and increase token usage

Solution: Convert MCP functionality to skills + CLI commands

Examples:

  • GitHub MCP → /gh-pr command wrapping gh pr create
  • Supabase MCP → Skills using Supabase CLI directly
  • Vercel/Railway MCP → CLI wrapper skills

With lazy loading (recent update):

  • Context window issue mostly solved
  • Token usage still optimizable via CLI + skills
  • Run MCP operations via CLI instead of in-context
  • Significantly reduces token usage for heavy operations (DB queries, deployments)

When to use CLI + skills over MCP:

  • Heavy repeated operations
  • Token cost is concern
  • Operation can be scripted
  • MCP wrapper adds minimal convenience

Trade-off:

  • MCP: More convenient, higher token cost
  • CLI + Skills: Slightly less convenient, much cheaper tokens

References


Integration with ID8Pipeline

This guide's patterns map directly to ID8Pipeline stages:

Stage 7 (Test Coverage):

  • Continuous evals
  • pass@k vs pass^k metrics
  • Verification loops

Stage 11 (Listen & Iterate):

  • Continuous learning hooks
  • Session reflection
  • Latent demand discovery from agent logs

All Stages:

  • Token optimization reduces cost
  • Memory persistence maintains continuity
  • Verification patterns prevent tech debt