🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
TypeScript agentic-aiagentic-frameworkagentic-workflowagentsai-agentsai-assistantai-skillsautonomous-agentsclaudecodexdsh-pluginharnessmcp-servermulti-agentmulti-agent-systemsnpmskillsswarmswarm-intelligencetypescript
checkpoint: File edit:
rUv5dbe8ec3a57d
diff --git a/v3/.agentic-flow/intelligence.json b/v3/.agentic-flow/intelligence.jsonindex 0cb0ec53f0..b7600d8748 100644--- a/v3/.agentic-flow/intelligence.json+++ b/v3/.agentic-flow/intelligence.json@@ -1,7 +1,8 @@{"patterns": {"command:": {- "success": 0.19+ "success": 0.5217031000000001,+ "failure": -0.05}},"sequences": {},@@ -9,8 +10,8 @@"dirPatterns": {},"errorPatterns": [],"metrics": {- "totalRoutes": 3,- "successfulRoutes": 3,+ "totalRoutes": 9,+ "successfulRoutes": 8,"routingHistory": [{"timestamp": "2026-01-04T17:04:00.672Z",diff --git a/v3/@claude-flow/embeddings/README.md b/v3/@claude-flow/embeddings/README.mdindex a2114805fe..5736e7d206 100644--- a/v3/@claude-flow/embeddings/README.md+++ b/v3/@claude-flow/embeddings/README.md@@ -1,17 +1,29 @@# @claude-flow/embeddings-Embedding generation and management module for Claude-Flow V3.+[](https://www.npmjs.com/package/@claude-flow/embeddings)+[](https://www.npmjs.com/package/@claude-flow/embeddings)+[](https://opensource.org/licenses/MIT)+[](https://www.typescriptlang.org/)+[](https://github.com/ruvnet/claude-flow)-## Overview--This module provides embedding generation, caching, and management for semantic search and memory systems.+> High-performance embedding generation module for Claude Flow V3 - multi-provider support, LRU caching, batch processing, and similarity computation.## Features-- **Multiple Providers**: OpenAI, Anthropic, local models-- **Caching**: LRU cache for frequently used embeddings-- **Batch Processing**: Efficient batch embedding generation-- **Dimensionality**: Support for various embedding dimensions+- **Multiple Providers** - OpenAI, Transformers.js (local), and Mock for testing+- **LRU Caching** - Intelligent caching with configurable size and hit rate tracking+- **Batch Processing** - Efficient batch embedding with partial cache hits+- **Similarity Functions** - Cosine, Euclidean, and dot product metrics+- **Event System** - Observable embedding operations with event listeners+- **Type-Safe** - Full TypeScript support with comprehensive type definitions++## Performance Targets++| Operation | API Provider | Local Provider |+|-----------|--------------|----------------|+| Single embedding | <100ms | <50ms |+| Batch (10 items) | <500ms | <200ms |+| Cache hit | <1ms | <1ms |## Installation@@ -19,62 +31,329 @@ This module provides embedding generation, caching, and management for semanticnpm install @claude-flow/embeddings```-## Usage+## Quick Start```typescript-import { EmbeddingService } from '@claude-flow/embeddings';+import { createEmbeddingService, cosineSimilarity } from '@claude-flow/embeddings';-const embeddings = new EmbeddingService({+// Create embedding service+const service = createEmbeddingService({provider: 'openai',+ apiKey: process.env.OPENAI_API_KEY!,model: 'text-embedding-3-small',dimensions: 1536,});// Generate embedding-const vector = await embeddings.embed('Hello world');+const result = await service.embed('Hello, world!');+console.log(`Embedding dimension: ${result.embedding.length}`);+console.log(`Latency: ${result.latencyMs}ms`);// Batch embed-const vectors = await embeddings.embedBatch([- 'First text',- 'Second text',- 'Third text',+const batchResult = await service.embedBatch([+ 'First document',+ 'Second document',+ 'Third document',]);+console.log(`Processed ${batchResult.embeddings.length} embeddings`);+console.log(`Cache hits: ${batchResult.cacheStats?.hits}`);// Calculate similarity-const similarity = embeddings.cosineSimilarity(vector1, vector2);+const similarity = cosineSimilarity(+ batchResult.embeddings[0],+ batchResult.embeddings[1]+);+console.log(`Similarity: ${similarity.toFixed(4)}`);```## API Reference-### EmbeddingService+### Factory Functions++```typescript+import { createEmbeddingService, getEmbedding } from '@claude-flow/embeddings';++// Create a service instance+const service = createEmbeddingService({+ provider: 'openai',+ apiKey: 'your-api-key',+ model: 'text-embedding-3-small',+ cacheSize: 1000,+});++// Quick one-off embedding+const embedding = await getEmbedding('Hello world', {+ provider: 'mock',+ dimensions: 384,+});+```++### OpenAI Provider++```typescript+import { OpenAIEmbeddingService } from '@claude-flow/embeddings';++const service = new OpenAIEmbeddingService({+ provider: 'openai',+ apiKey: process.env.OPENAI_API_KEY!,+ model: 'text-embedding-3-small', // or 'text-embedding-3-large'+ dimensions: 1536, // Adjustable for v3 models+ baseURL: 'https://api.openai.com/v1/embeddings',+ timeout: 30000,+ maxRetries: 3,+ cacheSize: 1000,+});++const result = await service.embed('Your text here');+console.log('Tokens used:', result.usage?.totalTokens);+```++### Transformers.js Provider (Local)-Main class for embedding operations.+```typescript+import { TransformersEmbeddingService } from '@claude-flow/embeddings';-#### Methods+const service = new TransformersEmbeddingService({+ provider: 'transformers',+ model: 'Xenova/all-MiniLM-L6-v2', // Any HuggingFace model+ cacheSize: 1000,+});-- `embed(text: string)` - Generate embedding for text-- `embedBatch(texts: string[])` - Batch embedding generation-- `cosineSimilarity(a, b)` - Calculate similarity score+// First call downloads the model+const result = await service.embed('Your text here');+console.log(`Local embedding generated in ${result.latencyMs}ms`);+```-## Configuration+### Mock Provider (Testing)```typescript-interface EmbeddingConfig {- provider: 'openai' | 'anthropic' | 'local';- model?: string;- dimensions?: number;- cacheEnabled?: boolean;- cacheSize?: number;+import { MockEmbeddingService } from '@claude-flow/embeddings';++const service = new MockEmbeddingService({+ provider: 'mock',+ dimensions: 384,+ simulatedLatency: 10, // Optional delay+ cacheSize: 100,+});++// Deterministic embeddings based on text hash+const result = await service.embed('Your text here');+```++### Batch Processing++```typescript+const result = await service.embedBatch([+ 'Document 1: Introduction to machine learning',+ 'Document 2: Deep learning fundamentals',+ 'Document 3: Natural language processing',+ 'Document 4: Computer vision basics',+]);++console.log('Batch Results:', {+ count: result.embeddings.length,+ totalLatency: `${result.totalLatencyMs}ms`,+ avgLatency: `${result.avgLatencyMs}ms`,+ cacheHits: result.cacheStats?.hits,+ cacheMisses: result.cacheStats?.misses,+ tokensUsed: result.usage?.totalTokens,+});+```++### Similarity Functions++```typescript+import {+ cosineSimilarity,+ euclideanDistance,+ dotProduct,+ computeSimilarity,+} from '@claude-flow/embeddings';++// Cosine similarity (most common for embeddings)+const cosine = cosineSimilarity(embedding1, embedding2);+// Returns: 0.0 to 1.0 (higher = more similar)++// Euclidean distance+const distance = euclideanDistance(embedding1, embedding2);+// Returns: 0.0 to infinity (lower = more similar)++// Dot product+const dot = dotProduct(embedding1, embedding2);+// Returns: unbounded (higher = more similar for normalized vectors)++// Generic similarity with metric selection+const result = computeSimilarity(embedding1, embedding2, 'cosine');+// Returns: { score: 0.95, metric: 'cosine' }+```++### Cache Management++```typescript+// Get cache statistics+const stats = service.getCacheStats();+console.log('Cache Stats:', {+ size: stats.size,+ maxSize: stats.maxSize,+ hitRate: `${(stats.hitRate * 100).toFixed(1)}%`,+});++// Clear cache+service.clearCache();++// Shutdown service+await service.shutdown();+```++### Event System++```typescript+// Subscribe to embedding events+service.addEventListener((event) => {+ switch (event.type) {+ case 'embed_start':+ console.log(`Starting: ${event.text.substring(0, 50)}...`);+ break;+ case 'embed_complete':+ console.log(`Completed in ${event.latencyMs}ms`);+ break;+ case 'embed_error':+ console.error(`Error: ${event.error}`);+ break;+ case 'cache_hit':+ console.log('Cache hit!');+ break;+ case 'batch_start':+ console.log(`Batch of ${event.count} started`);+ break;+ case 'batch_complete':+ console.log(`Batch completed in ${event.latencyMs}ms`);+ break;+ case 'cache_eviction':+ console.log(`Cache evicted ${event.size} entries`);+ break;+ }+});++// Remove listener+service.removeEventListener(listener);+```++## Provider Comparison++| Provider | Latency | Quality | Cost | Offline |+|----------|---------|---------|------|---------|+| **OpenAI** | ~50-100ms | Excellent | $0.02-0.13/1M tokens | No |+| **Transformers.js** | ~20-50ms | Good | Free | Yes |+| **Mock** | <1ms | N/A | Free | Yes |++### OpenAI Models++| Model | Dimensions | Max Tokens | Best For |+|-------|------------|------------|----------|+| `text-embedding-3-small` | 1536 | 8191 | General purpose, cost-effective |+| `text-embedding-3-large` | 3072 | 8191 | Highest quality |+| `text-embedding-ada-002` | 1536 | 8191 | Legacy support |++### Transformers.js Models++| Model | Dimensions | Size | Best For |+|-------|------------|------|----------|+| `Xenova/all-MiniLM-L6-v2` | 384 | 23MB | Fast, general purpose |+| `Xenova/all-mpnet-base-v2` | 768 | 110MB | Higher quality |+| `Xenova/bge-small-en-v1.5` | 384 | 33MB | Retrieval optimized |++## TypeScript Types++```typescript+import type {+ // Provider types+ EmbeddingProvider,+ EmbeddingConfig,+ OpenAIEmbeddingConfig,+ TransformersEmbeddingConfig,+ MockEmbeddingConfig,++ // Result types+ EmbeddingResult,+ BatchEmbeddingResult,++ // Service interface+ IEmbeddingService,++ // Event types+ EmbeddingEvent,+ EmbeddingEventListener,++ // Similarity types+ SimilarityMetric,+ SimilarityResult,+} from '@claude-flow/embeddings';+```++## Environment Variables++```bash+# OpenAI configuration+OPENAI_API_KEY=sk-...++# Optional: Custom base URL (for Azure OpenAI, etc.)+OPENAI_BASE_URL=https://your-endpoint.openai.azure.com/+```++## Error Handling++```typescript+try {+ const result = await service.embed('Your text');+} catch (error) {+ if (error instanceof Error) {+ if (error.message.includes('API error')) {+ // Handle API errors (rate limits, auth, etc.)+ } else if (error.message.includes('Failed to initialize')) {+ // Handle model loading errors (Transformers.js)+ }+ }}```-## Supported Providers+## Integration with Memory Module++```typescript+import { createEmbeddingService } from '@claude-flow/embeddings';+import { HNSWIndex } from '@claude-flow/memory';++// Create embedding service+const embeddings = createEmbeddingService({+ provider: 'openai',+ apiKey: process.env.OPENAI_API_KEY!,+ model: 'text-embedding-3-small',+});++// Create HNSW index+const index = new HNSWIndex({+ dimensions: 1536,+ metric: 'cosine',+});++// Index documents+const documents = ['Doc 1 content', 'Doc 2 content', 'Doc 3 content'];+const { embeddings: vectors } = await embeddings.embedBatch(documents);++vectors.forEach((vector, i) => {+ index.addPoint(`doc-${i}`, new Float32Array(vector));+});++// Search+const queryResult = await embeddings.embed('Search query');+const results = await index.search(new Float32Array(queryResult.embedding), 5);+```++## Related Packages-| Provider | Models | Dimensions |-|----------|--------|------------|-| OpenAI | text-embedding-3-small/large | 1536/3072 |-| Anthropic | claude-embed | 1024 |-| Local | sentence-transformers | 384-768 |+- [@claude-flow/memory](../memory) - HNSW indexing and vector storage+- [@claude-flow/providers](../providers) - Multi-LLM provider system+- [@claude-flow/neural](../neural) - SONA learning integration## License