DeepStack Knowledge Layer + AI Persona System
Overview
Build a semantic knowledge layer with RAG (Retrieval-Augmented Generation) and a 7-persona personality system for the DeepStack AI assistant.
User Choices:
- Embeddings: Local Ollama with
nomic-embed-text(768 dimensions, free) - Personas: Both sets - 7 personas (4 trading + 3 coaching)
- RAG Mode: Always retrieve on every message
- Storage: Supabase pgvector
Architecture: OpenAI 4-Pillar Style
┌─────────────────────────────────────────────────────────────────┐
│ USER CHAT MESSAGE │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ PILLAR 4: Context Assembly │
│ ├── Build persona prompt (based on selected persona) │
│ ├── Retrieve RAG context (semantic search) │
│ └── Inject runtime context (positions, theses, emotions) │
└─────────────────────────────────────────────────────────────────┘
│
┌───────────────────┴───────────────────┐
▼ ▼
┌──────────────────────┐ ┌──────────────────────┐
│ PILLAR 3: Retrieval │ │ PERSONA SYSTEM │
│ ├── Semantic search │ │ ├── 7 personas │
│ ├── Hybrid search │ │ ├── Prompt modifier │
│ └── RRF ranking │ │ └── UI selector │
└──────────────────────┘ └──────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ PILLAR 2: Embedding Pipeline │
│ ├── Ollama client (nomic-embed-text, 768d) │
│ ├── Auto-embed on create/update (journal, thesis, messages) │
│ └── Batch re-embedding utility │
└──────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ PILLAR 1: Knowledge Store (Supabase pgvector) │
│ ├── embeddings table (768d vectors) │
│ ├── HNSW index for fast similarity search │
│ └── RLS policies for user isolation │
└──────────────────────────────────────────────────────────────────┘
Agent & Skill Assignments
| Phase | Assigned Agent/Skill | Coordination Role |
|---|---|---|
| Phase 1 | database-architect + Supabase MCP |
Design & apply migrations |
| Phase 2 | backend-architect + ai-engineer |
Build embedding pipeline |
| Phase 3 | backend-architect |
Build retrieval service |
| Phase 4 | fullstack-developer |
Build persona system |
| Phase 5 | ai-engineer |
Integrate RAG into chat |
| Phase 6 | frontend-developer + Skill:ui-builder |
Build persona UI |
| Phase 7 | test-engineer + code-reviewer |
Test & review |
| Oversight | Claude (Coordinator) | Debug, verify, coordinate |
Plugins:
/supabase-toolkit:supabase-migration-assistant- Generate migrations/testing-suite:generate-tests- Generate test coverage/pr-review-toolkit:code-reviewer- Review completed code
Implementation Phases
Phase 1: Database Foundation
Assigned: database-architect agent + Supabase MCP
Files to create:
web/supabase/migrations/010_enable_pgvector.sqlweb/supabase/migrations/011_create_embeddings_table.sql
Key elements:
- Enable pgvector extension
- Create
embeddingstable with 768d vector column - HNSW index for cosine similarity
- RLS policies for user isolation
match_embeddings()SQL function for similarity search
Verification: Claude validates migration with mcp__supabase__execute_sql
Phase 2: Embedding Pipeline
Assigned: backend-architect + ai-engineer agents (parallel)
Files to create:
web/src/lib/embeddings/ollama-client.ts- Ollama API clientweb/src/lib/embeddings/embedding-service.ts- Content embedding logicweb/src/app/api/embeddings/route.ts- Embedding API endpoint
Key elements:
OllamaEmbeddingClientclass withembed()andembedBatch()- Content builders for journal, thesis, messages, patterns
upsertEmbedding()with content hash for change detection- Auto-embed triggers in existing API routes
Verification: Claude tests Ollama connectivity and embedding generation
Phase 3: Retrieval Service
Assigned: backend-architect agent
Files to create:
web/src/lib/embeddings/retrieval-service.ts- Search functionsweb/src/app/api/search/route.ts- Search API endpoint
Key elements:
semanticSearch()- Vector similarity searchkeywordSearch()- PostgreSQL full-text searchhybridSearch()- Reciprocal Rank Fusion (RRF) combining both- Filter by source type, symbol, threshold
Verification: Claude tests search API with sample queries
Phase 4: Persona System
Assigned: fullstack-developer agent + Skill:state-management
Files to create:
web/src/lib/types/persona.ts- TypeScript interfacesweb/src/lib/personas/persona-configs.ts- 7 persona definitionsweb/src/lib/personas/build-persona-prompt.ts- Prompt builderweb/src/lib/stores/persona-store.ts- Zustand store
7 Personas:
| ID | Name | Category | Focus |
|---|---|---|---|
value-investor |
Value Investor | Trading | Buffett-style fundamentals |
day-trader |
Day Trader | Trading | Momentum, technicals |
risk-manager |
Risk Manager | Trading | Capital preservation |
research-analyst |
Research Analyst | Trading | Neutral deep dives |
mentor |
Mentor | Coaching | Patient, educational |
coach |
Coach | Coaching | Direct, accountability |
analyst |
Analyst | Coaching | Data-focused, neutral |
Verification: Claude validates TypeScript types and store persistence
Phase 5: Context Assembly (RAG Integration)
Assigned: ai-engineer agent
Files to create:
web/src/lib/embeddings/context-builder.ts- RAG context builder
Files to modify:
web/src/app/api/chat/route.ts- Add RAG + persona to system prompt
Key elements:
buildRAGContext()- Retrieves and formats relevant knowledge- Token budget management (4000 tokens for RAG context)
- Integration with existing
buildContextMessage() - Persona prompt injection
Verification: Claude tests end-to-end chat with RAG context injection
Phase 6: UI Components
Assigned: frontend-developer agent + Skill:ui-builder
Files to create:
web/src/components/settings/PersonaCard.tsx- Persona selection cardweb/src/components/settings/PersonaSection.tsx- Settings sectionweb/src/components/chat/ActivePersonaIndicator.tsx- Chat indicator
Files to modify:
web/src/components/layout/SettingsPanel.tsx- Add persona sectionweb/src/components/chat/ChatInput.tsx- Show active personaweb/src/components/chat/ConversationView.tsx- Pass personaId to APIweb/src/app/globals.css- Add persona color variables
Verification: Claude uses Playwright MCP to test UI interactions
Phase 7: Tools & Testing
Assigned: test-engineer agent + code-reviewer agent
Files to modify:
web/src/lib/llm/tools.ts- Addsearch_knowledgetool
Testing:
- Verify Ollama connection and embedding generation
- Test semantic search accuracy
- Test persona prompt modifications
- Test UI persona selection persistence
Verification: Claude runs npm run type-check and npm run lint
Critical Files Reference
| Purpose | File Path |
|---|---|
| Chat API (main integration) | web/src/app/api/chat/route.ts |
| System prompt base | web/src/lib/llm/system-prompt.ts |
| LLM tools | web/src/lib/llm/tools.ts |
| Settings panel | web/src/components/layout/SettingsPanel.tsx |
| Chat input | web/src/components/chat/ChatInput.tsx |
| Conversation view | web/src/components/chat/ConversationView.tsx |
| Chat store pattern | web/src/lib/stores/chat-store.ts |
| Journal API pattern | web/src/app/api/journal/route.ts |
| Migration pattern | web/supabase/migrations/004_create_journal_entries.sql |
Environment Variables
# Add to .env.local
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_EMBED_MODEL=nomic-embed-text
Prerequisites
Ollama installed locally with
nomic-embed-textmodel:ollama pull nomic-embed-textSupabase project with ability to enable extensions
Coordination Workflow
┌─────────────────────────────────────────────────────────────────┐
│ CLAUDE (Coordinator) │
│ ├── Launches agents in parallel where possible │
│ ├── Monitors agent outputs for errors │
│ ├── Runs verification checks after each phase │
│ ├── Debugs and fixes issues before proceeding │
│ └── Commits working code incrementally │
└─────────────────────────────────────────────────────────────────┘
│
│ Launch Parallel Agents
▼
┌──────────┬──────────┬──────────┬──────────┐
│ Phase 1 │ Phase 4 │ Phase 6 │ Phase 7 │
│ DB │ Persona │ UI │ Testing │
│ Architect│ Fullstack│ Frontend │ Test Eng │
└──────────┴──────────┴──────────┴──────────┘
│
│ Sequential (Dependencies)
▼
┌──────────┬──────────┬──────────┐
│ Phase 2 │ Phase 3 │ Phase 5 │
│ Embed │ Retrieve │ Context │
│ Pipeline │ Service │ Assembly │
└──────────┴──────────┴──────────┘
Parallel Execution Strategy:
- Wave 1 (No dependencies): Phase 1 (DB) + Phase 4 (Personas) + Phase 6 (UI scaffolds)
- Wave 2 (After Phase 1): Phase 2 (Embedding) → Phase 3 (Retrieval)
- Wave 3 (After all): Phase 5 (Integration) + Phase 7 (Testing)
Verification Checkpoints:
- After each agent completes:
npm run type-check && npm run lint - After Phase 1: Verify pgvector extension enabled via Supabase MCP
- After Phase 2: Test Ollama embedding generation
- After Phase 5: End-to-end chat test with RAG context
- After Phase 6: Playwright UI test for persona selection
- Final: Full code review via
code-revieweragent
Success Criteria
- Embeddings auto-generate when journal/thesis created/updated
- Semantic search returns relevant past knowledge
- RAG context appears in AI responses referencing user's history
- 7 personas selectable from Settings
- Active persona visually indicated in chat
- Persona affects AI response style and focus
- All features work with existing emotional firewall
- TypeScript types pass (
npm run type-check) - Linting passes (
npm run lint) - Code reviewed by
code-revieweragent