π The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
TypeScript agentic-aiagentic-frameworkagentic-workflowagentsai-agentsai-assistantai-skillsautonomous-agentsclaudecodexdsh-pluginharnessmcp-servermulti-agentmulti-agent-systemsnpmskillsswarmswarm-intelligencetypescript
fix(embeddings): writable model cache dir + retryable degraded state (#3692) (#3831)
Nick Ruest6afffb777cf2
diff --git a/v3/@claude-flow/cli/__tests__/adr-454-memory-initializer-retry.test.ts b/v3/@claude-flow/cli/__tests__/adr-454-memory-initializer-retry.test.tsnew file mode 100644index 0000000000..a03cf5155f--- /dev/null+++ b/v3/@claude-flow/cli/__tests__/adr-454-memory-initializer-retry.test.ts@@ -0,0 +1,235 @@+/**+ * ADR-454 β writable embedding model cache + retryable degraded-embedder state.+ *+ * #3692: a system-prefix global install's package-relative model cache dir+ * is root-owned, so an ordinary user hits EACCES the first time+ * `loadLocalEmbeddingChain()` tries to initialize transformers.js. Before+ * this fix, ANY failure on that path (including a transient/fixable one)+ * memoized `embeddingModelState.loaded = true` with the hash fallback+ * permanently for the rest of the process's life β the "already-running+ * workers required a restart" symptom. This suite proves:+ *+ * 1. A real backend that THROWS during init (EACCES-shaped) is marked+ * `degradedReason: 'retryable'`, not permanent.+ * 2. A retryable failure does NOT re-attempt before `MODEL_INIT_RETRY_INTERVAL_MS`+ * elapses (no retry-storm regression).+ * 3. A retryable failure DOES re-attempt β and recovers to `backend: 'onnx'`+ * β once the interval has elapsed, without a process restart.+ * 4. No real backend importable at all is marked `degradedReason: 'permanent'`+ * and is NEVER retried, even long after the interval elapses.+ * 5. `env.cacheDir` is resolved and set before `pipelineFn(...)` is invoked.+ * 6. Hot path: once healthy (backend 'onnx'), repeated embed calls do not+ * re-run cache-dir resolution / re-invoke the pipeline factory.+ * 7. ADR-390's `backend: 'onnx' | 'mock'` contract keeps reporting accurately+ * in both the recovered and the permanently-degraded case.+ *+ * London-school: the transformers packages, agentic-flow, and ruvector are+ * all mocked via hoisted mutable state (mirrors the established pattern in+ * `issue-3375-rescue-local-embedder.test.ts`) so each test can drive the+ * exact failure shape without touching the network or a real ONNX model.+ * `generateLocalEmbedding()` is used directly β per its own contract it never+ * consults the AgentDB bridge, so no bridge setup is needed here.+ *+ * `embeddingModelState` is private module state, so every test gets an+ * isolated instance via `vi.resetModules()` + a fresh dynamic import.+ */+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';+import { existsSync, mkdtempSync, rmSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';++type PipelineMode = 'absent' | 'throw' | 'succeed';++const local = vi.hoisted(() => ({+ hfMode: 'absent' as PipelineMode,+ xenMode: 'absent' as PipelineMode,+ hfEnv: {} as Record<string, unknown>,+ xenEnv: {} as Record<string, unknown>,+ hfCacheDirAtCallTime: undefined as string | undefined,+ xenCacheDirAtCallTime: undefined as string | undefined,+ pipelineCallCount: 0,+ vec(text: string): number[] {+ let h = 0;+ for (let i = 0; i < text.length; i++) h = (h * 31 + text.charCodeAt(i)) | 0;+ return Array.from({ length: 384 }, (_, i) => Math.sin(h + i * 7) + 1.5);+ },+}));++function makePipelineGetter(which: 'hf' | 'xen') {+ return function pipeline() {+ const mode = which === 'hf' ? local.hfMode : local.xenMode;+ if (mode === 'absent') return undefined;+ return async (_task: string, _model?: string) => {+ local.pipelineCallCount++;+ const env = which === 'hf' ? local.hfEnv : local.xenEnv;+ if (which === 'hf') local.hfCacheDirAtCallTime = env.cacheDir as string | undefined;+ else local.xenCacheDirAtCallTime = env.cacheDir as string | undefined;+ if (mode === 'throw') {+ const err: NodeJS.ErrnoException = new Error('EACCES: permission denied, mkdir');+ err.code = 'EACCES';+ throw err;+ }+ // The resolved "extractor" β the callable returned by a real+ // pipeline('feature-extraction', ...) call.+ return async (text: string) => ({ data: Float32Array.from(local.vec(text)) });+ };+ };+}++vi.mock('@huggingface/transformers', () => ({+ get pipeline() {+ return makePipelineGetter('hf')();+ },+ get env() {+ return local.hfEnv;+ },+}));+vi.mock('@xenova/transformers', () => ({+ get pipeline() {+ return makePipelineGetter('xen')();+ },+ get env() {+ return local.xenEnv;+ },+}));+// Every fallback below the transformers branches is disabled so the hash+// fallback (the branch under test) is reliably reached.+vi.mock('agentic-flow/reasoningbank', () => ({ computeEmbedding: undefined }));+vi.mock('agentic-flow', () => ({ embeddings: undefined }));+vi.mock('ruvector', () => ({ initOnnxEmbedder: undefined, getOptimizedOnnxEmbedder: undefined }));++async function freshInit() {+ vi.resetModules();+ return await import('../src/memory/memory-initializer.js');+}++let tmpRoot: string;++beforeEach(() => {+ vi.useFakeTimers();+ local.hfMode = 'absent';+ local.xenMode = 'absent';+ local.hfEnv = {};+ local.xenEnv = {};+ local.hfCacheDirAtCallTime = undefined;+ local.xenCacheDirAtCallTime = undefined;+ local.pipelineCallCount = 0;+ delete process.env.TRANSFORMERS_CACHE;+ delete process.env.HF_HOME;+ delete process.env.RUFLO_REQUIRE_REAL_EMBEDDINGS;+ // generateLocalEmbedding() never touches the bridge per its own contract,+ // but generateEmbedding() (exercised once below) is bridge-first β force+ // the bridge off so that call stays on the local chain under test too,+ // instead of importing the real memory-bridge.js / AgentDB machinery.+ process.env.CLAUDE_FLOW_DISABLE_BRIDGE = '1';+ tmpRoot = mkdtempSync(join(tmpdir(), 'ruflo-mem-init-adr454-'));+});++afterEach(() => {+ vi.useRealTimers();+ delete process.env.CLAUDE_FLOW_DISABLE_BRIDGE;+ rmSync(tmpRoot, { recursive: true, force: true });+});++describe('ADR-454 permanent degraded state (no transformers package importable)', () => {+ it('marks the hash fallback permanent and never retries, even long after the retry interval elapses', async () => {+ const init = await freshInit();++ const out1 = await init.generateLocalEmbedding('hello');+ expect(out1.backend).toBe('mock');+ expect(out1.dimensions).toBe(128);+ expect(local.pipelineCallCount).toBe(0);++ // Advance well past MODEL_INIT_RETRY_INTERVAL_MS (60s) and even flip the+ // backend to "would now succeed" β a permanent state must not care.+ vi.advanceTimersByTime(10 * 60_000);+ local.hfMode = 'succeed';++ const out2 = await init.generateLocalEmbedding('hello again, much later');+ expect(out2.backend).toBe('mock');+ expect(local.pipelineCallCount).toBe(0);+ });+});++describe('ADR-454 retryable degraded state (a real backend existed but init threw)', () => {+ it('marks retryable on an EACCES-shaped pipeline failure, does not retry before the interval elapses, and recovers to onnx after it does', async () => {+ local.hfMode = 'throw';+ const init = await freshInit();++ const out1 = await init.generateLocalEmbedding('alpha task');+ expect(out1.backend).toBe('mock');+ expect(out1.dimensions).toBe(128);+ expect(local.pipelineCallCount).toBe(1);++ // Too soon (30s < 60s interval): must return the cached degraded state,+ // not re-attempt β even though the backend has since "recovered".+ vi.advanceTimersByTime(30_000);+ local.hfMode = 'succeed'; // e.g. an admin fixed the cache-dir permissions+ const out2 = await init.generateLocalEmbedding('beta task, still too soon');+ expect(out2.backend).toBe('mock');+ expect(local.pipelineCallCount).toBe(1); // not re-attempted++ // Interval elapsed (30_000 + 30_001 = 60_001ms since the failed attempt):+ // must re-attempt and recover without a process restart.+ vi.advanceTimersByTime(30_001);+ const out3 = await init.generateLocalEmbedding('gamma task, now it works');+ expect(out3.backend).toBe('onnx');+ expect(out3.dimensions).toBe(384);+ expect(local.pipelineCallCount).toBe(2);+ });+});++describe('ADR-454 env.cacheDir set before pipelineFn(...) is called (memory-initializer.ts copy)', () => {+ it('resolves TRANSFORMERS_CACHE, creates the directory, and sets env.cacheDir before the hf pipeline is invoked', async () => {+ const cacheDir = join(tmpRoot, 'models');+ process.env.TRANSFORMERS_CACHE = cacheDir;+ local.hfMode = 'succeed';+ const init = await freshInit();++ const out = await init.generateLocalEmbedding('cache dir probe');+ expect(out.backend).toBe('onnx');+ expect(local.hfCacheDirAtCallTime).toBe(cacheDir);+ expect(existsSync(cacheDir)).toBe(true);+ });+});++describe('ADR-454 hot path: healthy onnx state does no redundant work per embed call', () => {+ it('does not re-resolve the cache dir or re-invoke the pipeline factory on subsequent healthy calls', async () => {+ const cacheDir = join(tmpRoot, 'models');+ process.env.TRANSFORMERS_CACHE = cacheDir;+ local.hfMode = 'succeed';+ const init = await freshInit();++ const out1 = await init.generateLocalEmbedding('task one');+ expect(out1.backend).toBe('onnx');+ expect(local.pipelineCallCount).toBe(1);++ // Many more calls on an already-healthy state must not touch the+ // pipeline factory (which is where cache-dir resolution + mkdir live)+ // again β only the retry-interval `Date.now()` comparison should run.+ await init.generateLocalEmbedding('task two');+ await init.generateLocalEmbedding('task three');+ await init.generateLocalEmbedding('task four');+ expect(local.pipelineCallCount).toBe(1);+ });+});++describe('ADR-454 / ADR-390 backend field regression guard', () => {+ it('reports backend:"onnx" in the recovered case and backend:"mock" in the permanently-degraded case', async () => {+ // Permanently degraded: no backend importable at all.+ const degradedInit = await freshInit();+ const degraded = await degradedInit.generateLocalEmbedding('no backend available');+ expect(degraded.backend).toBe('mock');++ // Recovered: a real backend loads successfully from a cold state.+ local.hfMode = 'succeed';+ const recoveredInit = await freshInit();+ const recovered = await recoveredInit.generateLocalEmbedding('real backend available');+ expect(recovered.backend).toBe('onnx');++ // generateEmbedding() (bridge-first, but no bridge configured here so it+ // falls through to the local chain) must report the same backend truthfully.+ const viaGenerateEmbedding = await recoveredInit.generateEmbedding('real backend available via generateEmbedding');+ expect(viaGenerateEmbedding.backend).toBe('onnx');+ });+});diff --git a/v3/@claude-flow/cli/__tests__/adr-454-task-embedder-retry.test.ts b/v3/@claude-flow/cli/__tests__/adr-454-task-embedder-retry.test.tsnew file mode 100644index 0000000000..108c15cae6--- /dev/null+++ b/v3/@claude-flow/cli/__tests__/adr-454-task-embedder-retry.test.ts@@ -0,0 +1,151 @@+/**+ * ADR-454 β task-embedder.ts's own permanent-vs-retryable bug.+ *+ * Separate from `memory-initializer.ts`'s bug: `loadExtractor()` cached+ * `_extractorPromise` permanently even when it resolved to `null` on a+ * transient/fixable failure (e.g. EACCES on the model cache dir), with no+ * way to distinguish that from a genuinely missing dependency. This suite+ * proves the same retryable/permanent distinction holds here, plus the+ * cache-dir fix and the hot-path no-redundant-work requirement.+ *+ * `embedTaskWithCache()` never throws and never consults any bridge, so no+ * bridge setup is needed. Module state is reset via the file's own test+ * seam, `__resetTaskEmbedderForTests()`, rather than `vi.resetModules()`.+ */+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';+import { existsSync, mkdtempSync, rmSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import {+ __resetTaskEmbedderForTests,+ embedTaskWithCache,+ embedTaskWithCacheBatch,+} from '../src/ruvector/task-embedder.js';++type PipelineMode = 'absent' | 'throw' | 'succeed';++const local = vi.hoisted(() => ({+ mode: 'absent' as PipelineMode,+ env: {} as Record<string, unknown>,+ cacheDirAtCallTime: undefined as string | undefined,+ pipelineCallCount: 0,+}));++vi.mock('@xenova/transformers', () => ({+ get pipeline() {+ if (local.mode === 'absent') return undefined;+ return async (_task: string, _model?: string, _opts?: unknown) => {+ local.pipelineCallCount++;+ local.cacheDirAtCallTime = local.env.cacheDir as string | undefined;+ if (local.mode === 'throw') {+ const err: NodeJS.ErrnoException = new Error('EACCES: permission denied, mkdir');+ err.code = 'EACCES';+ throw err;+ }+ // The "extractor" callable returned by a real pipeline(...) call,+ // supporting both single-text and array-input modes.+ return async (input: string | string[], _opts: unknown) => {+ const texts = Array.isArray(input) ? input : [input];+ const dim = 384;+ const data = new Float32Array(texts.length * dim).fill(0.5);+ return { data, dims: [texts.length, dim] };+ };+ };+ },+ get env() {+ return local.env;+ },+}));++let tmpRoot: string;++beforeEach(() => {+ vi.useFakeTimers();+ __resetTaskEmbedderForTests();+ local.mode = 'absent';+ local.env = {};+ local.cacheDirAtCallTime = undefined;+ local.pipelineCallCount = 0;+ delete process.env.TRANSFORMERS_CACHE;+ delete process.env.HF_HOME;+ tmpRoot = mkdtempSync(join(tmpdir(), 'ruflo-task-embedder-adr454-'));+});++afterEach(() => {+ vi.useRealTimers();+ rmSync(tmpRoot, { recursive: true, force: true });+});++describe('ADR-454 permanent degraded state (no @xenova/transformers pipeline export)', () => {+ it('returns undefined and never retries, even long after the retry interval elapses', async () => {+ const out1 = await embedTaskWithCache('alpha task');+ expect(out1).toBeUndefined();+ expect(local.pipelineCallCount).toBe(0);++ vi.advanceTimersByTime(10 * 60_000);+ local.mode = 'succeed'; // "the package got installed" β must not matter++ const out2 = await embedTaskWithCache('beta task, much later');+ expect(out2).toBeUndefined();+ expect(local.pipelineCallCount).toBe(0);+ });+});++describe('ADR-454 retryable degraded state (pipeline present but init threw)', () => {+ it('returns undefined on an EACCES-shaped failure, does not retry before the interval elapses, and recovers after it does', async () => {+ local.mode = 'throw';++ const out1 = await embedTaskWithCache('alpha task');+ expect(out1).toBeUndefined();+ expect(local.pipelineCallCount).toBe(1);++ // Too soon: must not re-attempt.+ vi.advanceTimersByTime(30_000);+ local.mode = 'succeed';+ const out2 = await embedTaskWithCache('beta task, still too soon');+ expect(out2).toBeUndefined();+ expect(local.pipelineCallCount).toBe(1);++ // Interval elapsed: must re-attempt and recover.+ vi.advanceTimersByTime(30_001);+ const out3 = await embedTaskWithCache('gamma task, now it works');+ expect(out3).toBeDefined();+ expect(out3).toHaveLength(384);+ expect(local.pipelineCallCount).toBe(2);+ });+});++describe('ADR-454 env.cacheDir set before pipeline(...) is called (task-embedder.ts copy)', () => {+ it('resolves TRANSFORMERS_CACHE, creates the directory, and sets env.cacheDir before the pipeline is invoked', async () => {+ const cacheDir = join(tmpRoot, 'models');+ process.env.TRANSFORMERS_CACHE = cacheDir;+ local.mode = 'succeed';++ const out = await embedTaskWithCache('cache dir probe task');+ expect(out).toBeDefined();+ expect(local.cacheDirAtCallTime).toBe(cacheDir);+ expect(existsSync(cacheDir)).toBe(true);+ });+});++describe('ADR-454 hot path: healthy extractor does no redundant work per embed call', () => {+ it('loads the pipeline once and does not re-invoke it on subsequent cache-miss calls', async () => {+ const cacheDir = join(tmpRoot, 'models');+ process.env.TRANSFORMERS_CACHE = cacheDir;+ local.mode = 'succeed';++ const out1 = await embedTaskWithCache('task one');+ expect(out1).toBeDefined();+ expect(local.pipelineCallCount).toBe(1);++ // Distinct task text each time so these are cache MISSES, not hits β+ // the embedder itself runs again, but the pipeline factory (where+ // cache-dir resolution + mkdir live) must not re-run.+ await embedTaskWithCache('task two');+ await embedTaskWithCache('task three');+ const batch = await embedTaskWithCacheBatch(['task four', 'task five']);+ expect(batch.every((v) => v !== undefined)).toBe(true);++ expect(local.pipelineCallCount).toBe(1);+ });+});diff --git a/v3/@claude-flow/cli/__tests__/model-cache-dir-adr454.test.ts b/v3/@claude-flow/cli/__tests__/model-cache-dir-adr454.test.tsnew file mode 100644index 0000000000..93e88a28a4--- /dev/null+++ b/v3/@claude-flow/cli/__tests__/model-cache-dir-adr454.test.ts@@ -0,0 +1,47 @@+/**+ * ADR-454 β `@claude-flow/cli`'s own copy of `resolveModelCacheDir()`.+ *+ * `v3/@claude-flow/cli/src/memory/model-cache-dir.ts` is an intentional,+ * dependency-free duplicate of `@claude-flow/embeddings/src/transformers-loader.ts`'s+ * helper of the same name (the CLI package has no dependency on+ * `@claude-flow/embeddings`, to avoid reintroducing the ADR-094 circular+ * optional-dep issue). Verified independently per the ADR: testing one copy+ * does not cover the other.+ */+import { afterEach, beforeEach, describe, expect, it } from 'vitest';+import { homedir } from 'node:os';+import { join } from 'node:path';+import { resolveModelCacheDir } from '../src/memory/model-cache-dir.js';++const savedEnv: { TRANSFORMERS_CACHE?: string; HF_HOME?: string } = {};++beforeEach(() => {+ savedEnv.TRANSFORMERS_CACHE = process.env.TRANSFORMERS_CACHE;+ savedEnv.HF_HOME = process.env.HF_HOME;+ delete process.env.TRANSFORMERS_CACHE;+ delete process.env.HF_HOME;+});++afterEach(() => {+ if (savedEnv.TRANSFORMERS_CACHE === undefined) delete process.env.TRANSFORMERS_CACHE;+ else process.env.TRANSFORMERS_CACHE = savedEnv.TRANSFORMERS_CACHE;+ if (savedEnv.HF_HOME === undefined) delete process.env.HF_HOME;+ else process.env.HF_HOME = savedEnv.HF_HOME;+});++describe('ADR-454 resolveModelCacheDir() precedence (@claude-flow/cli copy)', () => {+ it('prefers TRANSFORMERS_CACHE when set', () => {+ process.env.TRANSFORMERS_CACHE = '/custom/transformers-cache';+ process.env.HF_HOME = '/custom/hf-home';+ expect(resolveModelCacheDir()).toBe('/custom/transformers-cache');+ });++ it('falls back to HF_HOME when TRANSFORMERS_CACHE is unset', () => {+ process.env.HF_HOME = '/custom/hf-home';+ expect(resolveModelCacheDir()).toBe('/custom/hf-home');+ });++ it('falls back to ~/.cache/ruflo/models when neither env var is set', () => {+ expect(resolveModelCacheDir()).toBe(join(homedir(), '.cache', 'ruflo', 'models'));+ });+});diff --git a/v3/@claude-flow/cli/src/memory/memory-initializer.ts b/v3/@claude-flow/cli/src/memory/memory-initializer.tsindex 38bb512473..85b2fffb2d 100644--- a/v3/@claude-flow/cli/src/memory/memory-initializer.ts+++ b/v3/@claude-flow/cli/src/memory/memory-initializer.ts@@ -22,6 +22,7 @@ import { createRequire } from 'node:module';import { readFileMaybeEncrypted, writeDbImagePreservingFormat, writeFileAtomic, writeFileRestricted } from '../fs-secure.js';import { restoreMemoryDbFromBackup } from '../services/memory-backup.js';import { validateIdentifier } from '../mcp-tools/validate-input.js';+import { resolveModelCacheDir } from './model-cache-dir.js';/*** ADR-323 β typed memory provenance. Distinguishes WHO/WHAT wrote a memory@@ -2260,8 +2261,19 @@ interface EmbeddingModel {model: unknown;tokenizer: unknown;dimensions: number;+ /**+ * ADR-454: distinguishes a hash-fallback state that's legitimately+ * permanent (no transformers package importable at all) from one caused+ * by an environmental/fixable failure (EACCES, ENOSPC, a transient+ * download failure) that should be retried rather than memoized forever.+ */+ degradedReason?: 'permanent' | 'retryable';+ lastAttemptAt?: number;}+/** ADR-454: minimum interval before a 'retryable' degraded state re-attempts real init. */+const MODEL_INIT_RETRY_INTERVAL_MS = 60_000;+/*** State of the LOCAL embedding chain only (transformers.js / agentic-flow /* ruvector ONNX / hash). #3375: the AgentDB bridge's result is cached@@ -2331,9 +2343,18 @@ async function loadLocalEmbeddingChain(verbose = false, startTime = Date.now()):error?: string;}> {if (embeddingModelState?.loaded) {- return { success: true, dimensions: embeddingModelState.dimensions, modelName: 'cached', loadTime: 0 };+ const isRetryableDue =+ embeddingModelState.degradedReason === 'retryable' &&+ Date.now() - (embeddingModelState.lastAttemptAt ?? 0) >= MODEL_INIT_RETRY_INTERVAL_MS;+ if (!isRetryableDue) {+ return { success: true, dimensions: embeddingModelState.dimensions, modelName: 'cached', loadTime: 0 };+ }+ // Fall through to re-attempt real initialization β the retryable+ // failure (e.g. EACCES on the model cache dir) may now be fixed.}+ let sawRetryableFailure = false;+try {// ADR-094: prefer @huggingface/transformers (clears protobufjs <7.5.5// critical RCE chain), fall back to legacy @xenova/transformers.@@ -2342,6 +2363,7 @@ async function loadLocalEmbeddingChain(verbose = false, startTime = Date.now()):// @claude-flow/embeddings/src/transformers-loader.ts.let transformersSource: '@huggingface/transformers' | '@xenova/transformers' | null = null;let pipelineFn: ((task: string, model?: string) => Promise<unknown>) | null = null;+ let transformersModule: Record<string, unknown> | null = null;{const tryLoad = async (specifier: string): Promise<Record<string, unknown> | null> => {@@ -2352,11 +2374,13 @@ async function loadLocalEmbeddingChain(verbose = false, startTime = Date.now()):if (hf && typeof hf.pipeline === 'function') {pipelineFn = hf.pipeline as (t: string, m?: string) => Promise<unknown>;transformersSource = '@huggingface/transformers';+ transformersModule = hf;} else {const xen = await tryLoad('@xenova/transformers');if (xen && typeof xen.pipeline === 'function') {pipelineFn = xen.pipeline as (t: string, m?: string) => Promise<unknown>;transformersSource = '@xenova/transformers';+ transformersModule = xen;}}}@@ -2374,6 +2398,15 @@ async function loadLocalEmbeddingChain(verbose = false, startTime = Date.now()):if (verbose) {console.log(`Loading ONNX embedding model via ${transformersSource} (all-MiniLM-L6-v2)...`);}++ // ADR-454: route model downloads to a user-writable cache dir β+ // the package-relative default is root-owned under a system-prefix+ // global install, which throws EACCES for the runtime user.+ const cacheDir = resolveModelCacheDir();+ fs.mkdirSync(cacheDir, { recursive: true });+ const env = transformersModule?.env as { cacheDir: string } | undefined;+ if (env) env.cacheDir = cacheDir;+const embedder = await pipelineFn('feature-extraction', 'Xenova/all-MiniLM-L6-v2');embeddingModelState = {@@ -2390,6 +2423,10 @@ async function loadLocalEmbeddingChain(verbose = false, startTime = Date.now()):loadTime: Date.now() - startTime};} catch (err) {+ // ADR-454: a real backend was importable and callable β this failure+ // (EACCES, ENOSPC, transient network error, etc.) is environmental,+ // not a missing dependency, so it must not be memoized permanently.+ sawRetryableFailure = true;if (verbose) {console.warn(`${transformersSource} pipeline init failed (${err instanceof Error ? err.message : String(err)}); ` +@@ -2468,12 +2505,18 @@ async function loadLocalEmbeddingChain(verbose = false, startTime = Date.now()):// No agentic-flow root-entry fallback: no published root exports// `embeddings`, and importing it runs its CLI main() in 3.0.0-alpha.x.- // No ONNX model available - use fallback+ // No ONNX model available - use fallback.+ // ADR-454: mark whether this is a permanent (no backend importable at+ // all) or retryable (a backend existed but init failed for an+ // environmental reason) degraded state, so a later call can re-attempt+ // real initialization once the condition is fixed.embeddingModelState = {loaded: true,model: null, // Will use simple hash-based fallbacktokenizer: null,- dimensions: 128 // Smaller fallback dimensions+ dimensions: 128, // Smaller fallback dimensions+ degradedReason: sawRetryableFailure ? 'retryable' : 'permanent',+ lastAttemptAt: Date.now()};return {@@ -2553,7 +2596,16 @@ export async function generateLocalEmbedding(text: string): Promise<{// loadEmbeddingModel(), which is bridge-first β when the bridge answered// there, no local model was ever loaded and this function always returned// the hash fallback.- if (!embeddingModelState?.loaded) {+ //+ // ADR-454: also re-attempt when the memoized state is a 'retryable'+ // degraded fallback whose retry interval has elapsed β otherwise this+ // guard short-circuits past loadLocalEmbeddingChain()'s own retry check+ // and the fix (e.g. a now-writable cache dir) never gets exercised.+ const shouldRetryLoad =+ !embeddingModelState?.loaded ||+ (embeddingModelState.degradedReason === 'retryable' &&+ Date.now() - (embeddingModelState.lastAttemptAt ?? 0) >= MODEL_INIT_RETRY_INTERVAL_MS);+ if (shouldRetryLoad) {await loadLocalEmbeddingChain();}diff --git a/v3/@claude-flow/cli/src/memory/model-cache-dir.ts b/v3/@claude-flow/cli/src/memory/model-cache-dir.tsnew file mode 100644index 0000000000..f86082e28c--- /dev/null+++ b/v3/@claude-flow/cli/src/memory/model-cache-dir.ts@@ -0,0 +1,19 @@+import * as os from 'node:os';+import * as path from 'node:path';++/**+ * Resolve a writable cache directory for ONNX model downloads.+ *+ * ADR-454: duplicated intentionally from+ * `@claude-flow/embeddings/src/transformers-loader.ts`'s+ * `resolveModelCacheDir()` rather than imported, since `@claude-flow/cli`+ * has no dependency on `@claude-flow/embeddings` (see the circular+ * optional-dep note in `memory-initializer.ts`).+ */+export function resolveModelCacheDir(): string {+ return (+ process.env.TRANSFORMERS_CACHE ||+ process.env.HF_HOME ||+ path.join(os.homedir(), '.cache', 'ruflo', 'models')+ );+}diff --git a/v3/@claude-flow/cli/src/ruvector/task-embedder.ts b/v3/@claude-flow/cli/src/ruvector/task-embedder.tsindex a8657c620d..376f65f449 100644--- a/v3/@claude-flow/cli/src/ruvector/task-embedder.ts+++ b/v3/@claude-flow/cli/src/ruvector/task-embedder.ts@@ -20,6 +20,9 @@* @module task-embedder*/+import * as fs from 'node:fs';+import { resolveModelCacheDir } from '../memory/model-cache-dir.js';+// ============================================================================// FNV-1a-32 hash (matches scripts/gen-seed-corpus.mjs + router-trajectory.ts)// ============================================================================@@ -52,15 +55,49 @@ type ExtractorFn = (input: string | string[], opts: { pooling: 'mean'; normalize// so we don't pay the ONNX model-load cost twice.let _extractorPromise: Promise<ExtractorFn | null> | null = null;+/**+ * ADR-454: distinguishes a null-extractor result caused by a genuinely+ * missing dependency (permanent β nothing changes without a reinstall) from+ * one caused by an environmental/fixable failure during `pipeline(...)`+ * init, such as EACCES on the model cache dir (retryable).+ */+let _extractorDegradedReason: 'permanent' | 'retryable' | null = null;+let _extractorLastAttemptAt = 0;+const EXTRACTOR_RETRY_INTERVAL_MS = 60_000;+function loadExtractor(): Promise<ExtractorFn | null> {- if (_extractorPromise !== null) return _extractorPromise;+ if (_extractorPromise !== null) {+ const isRetryableDue =+ _extractorDegradedReason === 'retryable' &&+ Date.now() - _extractorLastAttemptAt >= EXTRACTOR_RETRY_INTERVAL_MS;+ if (!isRetryableDue) return _extractorPromise;+ // Fall through to re-attempt β the retryable failure may now be fixed.+ }+ _extractorLastAttemptAt = Date.now();_extractorPromise = (async () => {+ let packageImportable = false;try {const specifier = '@xenova/transformers';const mod = await import(/* @vite-ignore */ specifier).catch(() => null);- if (!mod || typeof mod.pipeline !== 'function') return null;- return await mod.pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2', { quantized: true });+ if (!mod || typeof mod.pipeline !== 'function') {+ _extractorDegradedReason = 'permanent';+ return null;+ }+ packageImportable = true;++ // ADR-454: route model downloads to a user-writable cache dir β the+ // package-relative default is root-owned under a system-prefix+ // global install, which throws EACCES for the runtime user.+ const cacheDir = resolveModelCacheDir();+ fs.mkdirSync(cacheDir, { recursive: true });+ const env = (mod as Record<string, unknown>).env as { cacheDir: string } | undefined;+ if (env) env.cacheDir = cacheDir;++ const extractor = await mod.pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2', { quantized: true });+ _extractorDegradedReason = null;+ return extractor;} catch {+ _extractorDegradedReason = packageImportable ? 'retryable' : 'permanent';return null;}})();@@ -242,4 +279,6 @@ export function __resetTaskEmbedderForTests(): void {_hits = 0;_misses = 0;_extractorPromise = null;+ _extractorDegradedReason = null;+ _extractorLastAttemptAt = 0;}diff --git a/v3/@claude-flow/embeddings/__tests__/transformers-loader-adr454.test.ts b/v3/@claude-flow/embeddings/__tests__/transformers-loader-adr454.test.tsnew file mode 100644index 0000000000..7eed48f45b--- /dev/null+++ b/v3/@claude-flow/embeddings/__tests__/transformers-loader-adr454.test.ts@@ -0,0 +1,154 @@+/**+ * ADR-454 β writable embedding model cache dir.+ *+ * Covers `transformers-loader.ts`'s `resolveModelCacheDir()` precedence+ * (TRANSFORMERS_CACHE > HF_HOME > ~/.cache/ruflo/models) and the requirement+ * that `env.cacheDir` is set on the resolved transformers module BEFORE the+ * returned `pipeline` function is ever invoked by a caller, for both the+ * `@huggingface/transformers` and `@xenova/transformers` branches.+ *+ * London-school: both packages are mocked via hoisted mutable state so each+ * test can flip availability/behavior without re-declaring `vi.mock`. Module+ * state (`cached`/`cacheChecked`) is private to transformers-loader.ts, so+ * every test gets an isolated instance via `vi.resetModules()` + a fresh+ * dynamic import.+ */+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';+import { existsSync, mkdtempSync, rmSync } from 'node:fs';+import { homedir, tmpdir } from 'node:os';+import { join } from 'node:path';++const state = vi.hoisted(() => ({+ hfHasPipeline: false,+ xenHasPipeline: false,+ hfEnv: {} as Record<string, unknown>,+ xenEnv: {} as Record<string, unknown>,+ hfCacheDirAtCallTime: undefined as string | undefined,+ xenCacheDirAtCallTime: undefined as string | undefined,+}));++vi.mock('@huggingface/transformers', () => ({+ get pipeline() {+ if (!state.hfHasPipeline) return undefined;+ return async (_task: string, _model?: string) => {+ state.hfCacheDirAtCallTime = state.hfEnv.cacheDir as string | undefined;+ return async () => ({ data: new Float32Array(384) });+ };+ },+ get env() {+ return state.hfEnv;+ },+ version: '4.0.0-test',+}));++vi.mock('@xenova/transformers', () => ({+ get pipeline() {+ if (!state.xenHasPipeline) return undefined;+ return async (_task: string, _model?: string) => {+ state.xenCacheDirAtCallTime = state.xenEnv.cacheDir as string | undefined;+ return async () => ({ data: new Float32Array(384) });+ };+ },+ get env() {+ return state.xenEnv;+ },+ version: '2.17.0-test',+}));++let tmpRoot: string;++beforeEach(() => {+ vi.resetModules();+ state.hfHasPipeline = false;+ state.xenHasPipeline = false;+ state.hfEnv = {};+ state.xenEnv = {};+ state.hfCacheDirAtCallTime = undefined;+ state.xenCacheDirAtCallTime = undefined;+ delete process.env.TRANSFORMERS_CACHE;+ delete process.env.HF_HOME;+ tmpRoot = mkdtempSync(join(tmpdir(), 'ruflo-embeddings-adr454-'));+});++afterEach(() => {+ rmSync(tmpRoot, { recursive: true, force: true });+});++describe('ADR-454 resolveModelCacheDir() precedence', () => {+ it('prefers TRANSFORMERS_CACHE when set', async () => {+ process.env.TRANSFORMERS_CACHE = '/custom/transformers-cache';+ process.env.HF_HOME = '/custom/hf-home';+ const { resolveModelCacheDir } = await import('../src/transformers-loader.js');+ expect(resolveModelCacheDir()).toBe('/custom/transformers-cache');+ });++ it('falls back to HF_HOME when TRANSFORMERS_CACHE is unset', async () => {+ process.env.HF_HOME = '/custom/hf-home';+ const { resolveModelCacheDir } = await import('../src/transformers-loader.js');+ expect(resolveModelCacheDir()).toBe('/custom/hf-home');+ });++ it('falls back to ~/.cache/ruflo/models when neither env var is set', async () => {+ const { resolveModelCacheDir } = await import('../src/transformers-loader.js');+ expect(resolveModelCacheDir()).toBe(join(homedir(), '.cache', 'ruflo', 'models'));+ });+});++describe('ADR-454 env.cacheDir set before pipeline(...) is called', () => {+ it('@huggingface/transformers branch: cache dir is created and env.cacheDir is set before the pipeline function is invoked', async () => {+ const cacheDir = join(tmpRoot, 'models');+ process.env.TRANSFORMERS_CACHE = cacheDir;+ state.hfHasPipeline = true;++ const { loadTransformersPipeline } = await import('../src/transformers-loader.js');+ const handle = await loadTransformersPipeline();++ expect(handle?.source).toBe('@huggingface/transformers');+ // env.cacheDir must already be the resolved path immediately after+ // loadTransformersPipeline() resolves, before any downstream pipeline() call.+ expect(state.hfEnv.cacheDir).toBe(cacheDir);+ expect(existsSync(cacheDir)).toBe(true);++ // Invoking the returned pipeline function (as a real caller would) must+ // see the cache dir already configured β proves ordering, not just+ // end-state.+ await handle!.pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2');+ expect(state.hfCacheDirAtCallTime).toBe(cacheDir);+ });++ it('@xenova/transformers branch: cache dir is created and env.cacheDir is set before the pipeline function is invoked', async () => {+ const cacheDir = join(tmpRoot, 'models');+ process.env.HF_HOME = cacheDir;+ state.hfHasPipeline = false; // force fallback to xenova+ state.xenHasPipeline = true;++ const { loadTransformersPipeline } = await import('../src/transformers-loader.js');+ const handle = await loadTransformersPipeline();++ expect(handle?.source).toBe('@xenova/transformers');+ expect(state.xenEnv.cacheDir).toBe(cacheDir);+ expect(existsSync(cacheDir)).toBe(true);++ await handle!.pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2');+ expect(state.xenCacheDirAtCallTime).toBe(cacheDir);+ });++ it('prefers @huggingface/transformers over @xenova/transformers when both are available', async () => {+ const cacheDir = join(tmpRoot, 'models');+ process.env.TRANSFORMERS_CACHE = cacheDir;+ state.hfHasPipeline = true;+ state.xenHasPipeline = true;++ const { loadTransformersPipeline } = await import('../src/transformers-loader.js');+ const handle = await loadTransformersPipeline();++ expect(handle?.source).toBe('@huggingface/transformers');+ });++ it('returns null and caches the miss when neither package exposes a pipeline function', async () => {+ const { loadTransformersPipeline, getCachedTransformersSource } = await import('../src/transformers-loader.js');+ const handle = await loadTransformersPipeline();+ expect(handle).toBeNull();+ expect(getCachedTransformersSource()).toBeNull();+ });+});diff --git a/v3/@claude-flow/embeddings/src/transformers-loader.ts b/v3/@claude-flow/embeddings/src/transformers-loader.tsindex 17e59d8b92..16aa4c0eed 100644--- a/v3/@claude-flow/embeddings/src/transformers-loader.ts+++ b/v3/@claude-flow/embeddings/src/transformers-loader.ts@@ -14,6 +14,10 @@* users can see which package satisfied the runtime.*/+import * as fs from 'node:fs';+import * as os from 'node:os';+import * as path from 'node:path';+export type PipelineFn = (task: string,model?: string,@@ -29,6 +33,24 @@ export interface TransformersHandle {let cached: TransformersHandle | null = null;let cacheChecked = false;+/**+ * Resolve a writable cache directory for ONNX model downloads.+ *+ * ADR-454: a system-prefix global install's package directory is+ * root-owned, so defaulting the loader's cache into+ * `node_modules/@huggingface/transformers/.cache` fails with EACCES+ * for the runtime user. Honor an explicit env override first, then+ * fall back to the repo's existing `~/.cache/ruflo/<subpath>`+ * convention (see `gaia-loader.ts`, `gaia-judge.ts`).+ */+export function resolveModelCacheDir(): string {+ return (+ process.env.TRANSFORMERS_CACHE ||+ process.env.HF_HOME ||+ path.join(os.homedir(), '.cache', 'ruflo', 'models')+ );+}+/*** Load a working transformers pipeline. Returns null if neither* package is installed. Caches the first successful resolution so@@ -57,6 +79,9 @@ export async function loadTransformersPipeline(): Promise<TransformersHandle | n// Prefer the maintained successor.const hf = await tryLoad('@huggingface/transformers');if (hf && typeof hf.pipeline === 'function') {+ const cacheDir = resolveModelCacheDir();+ fs.mkdirSync(cacheDir, { recursive: true });+ if (hf.env) (hf.env as { cacheDir: string }).cacheDir = cacheDir;cached = {pipeline: hf.pipeline as PipelineFn,source: '@huggingface/transformers',@@ -68,6 +93,9 @@ export async function loadTransformersPipeline(): Promise<TransformersHandle | n// Fall back to the legacy package so existing installs keep working.const xen = await tryLoad('@xenova/transformers');if (xen && typeof xen.pipeline === 'function') {+ const cacheDir = resolveModelCacheDir();+ fs.mkdirSync(cacheDir, { recursive: true });+ if (xen.env) (xen.env as { cacheDir: string }).cacheDir = cacheDir;cached = {pipeline: xen.pipeline as PipelineFn,source: '@xenova/transformers',diff --git a/v3/docs/adr/ADR-454-writable-embedding-model-cache-and-retryable-degraded-state.md b/v3/docs/adr/ADR-454-writable-embedding-model-cache-and-retryable-degraded-state.mdnew file mode 100644index 0000000000..b743800c40--- /dev/null+++ b/v3/docs/adr/ADR-454-writable-embedding-model-cache-and-retryable-degraded-state.md@@ -0,0 +1,185 @@+# ADR-454 β Writable Embedding Model Cache and Retryable Degraded-Embedder State++**Status**: Proposed+**Date**: 2026-10-05+**Related**: ADR-094 (xenova β @huggingface/transformers migration), ADR-390 (semantic router uses the real embedder), #3692, `v3/@claude-flow/cli/src/benchmarks/gaia-loader.ts` / `gaia-judge.ts` (prior art for `~/.cache/ruflo/*`)+**Surfaces**: `v3/@claude-flow/embeddings/src/transformers-loader.ts`, `v3/@claude-flow/embeddings/src/embedding-service.ts`, `v3/@claude-flow/cli/src/memory/memory-initializer.ts`, `v3/@claude-flow/cli/src/ruvector/task-embedder.ts`++## Context++Issue #3692: on a system-prefix global install (`npm install -g` as root), an+ordinary user running `@claude-flow/cli` hits `EACCES` the first time any of+three independent call sites load the ONNX embedding model+(`Xenova/all-MiniLM-L6-v2`) via `@huggingface/transformers` /+`@xenova/transformers`'s `pipeline('feature-extraction', ...)`:++1. `v3/@claude-flow/embeddings/src/transformers-loader.ts`, resolved by+ `TransformersEmbeddingService.initialize()` in+ `v3/@claude-flow/embeddings/src/embedding-service.ts:398`.+2. `v3/@claude-flow/cli/src/memory/memory-initializer.ts:2337`,+ `loadLocalEmbeddingChain()`'s `pipelineFn('feature-extraction',+ 'Xenova/all-MiniLM-L6-v2')`.+3. `v3/@claude-flow/cli/src/ruvector/task-embedder.ts:62`, same pipeline call+ with `{ quantized: true }`.++None of the three passes a cache-dir override. Without one, the HF/Xenova+loader defaults to a path under its own package's+`node_modules/@huggingface/transformers/.cache`. Under a system-prefix global+install that directory tree is root-owned, so the runtime user's download+fails with `EACCES`. Reproduced by the reporter across five existing servers;+creating only the exact model-cache directory as the runtime user (`0700`),+without touching package ownership, let the normal loader download the model+and produce real embeddings β confirming the cache path, not the package+install, is the fixable boundary.++A second, compounding bug lives only in `memory-initializer.ts`. The `EACCES`+throw is caught by `loadLocalEmbeddingChain()`'s inner try/catch (around+line 2352, which carries a comment referencing #2461) and falls through to+the hash fallback at line ~2448:++```ts+embeddingModelState = {+ loaded: true,+ model: null, // Will use simple hash-based fallback+ tokenizer: null,+ dimensions: 128+};+```++Every caller short-circuits on `embeddingModelState?.loaded` β inside+`loadLocalEmbeddingChain` itself (line 2293) and inside+`generateLocalEmbedding` (line 2532). Because `loaded: true` is set+permanently on a transient, environmental failure, the degraded hash state is+memoized for the rest of the process's life β this is exactly the "already+running workers required a restart/refresh" symptom from the issue. Fixing+the cache-directory permissions mid-run does not help a worker that already+tripped this path; only a process restart does.++ADR-390 already established that a degraded embedder must be reported+truthfully rather than silently returning meaningless vectors, via an+explicit `backend: 'onnx' | 'mock'` field on `generateLocalEmbedding` /+`generateEmbedding`'s return value (`memory-initializer.ts`, line 2526 type+definition, lines 2562 and 2578 for assignments) and an `embedder: 'minilm' |+'hash'` field on router results. That reporting already exists and already+satisfies part of the issue's "report mock/degraded status truthfully"+expectation β the gap this ADR closes is specifically (a) where the model is+allowed to try to write, and (b) how long a fixable failure is allowed to+stay remembered.++The repo already has a convention for a user-writable, XDG-adjacent cache+outside any package directory: `path.join(os.homedir(), '.cache', 'ruflo',+...)`, used today by `gaia-loader.ts` (`~/.cache/ruflo/gaia`) and+`gaia-judge.ts` (`~/.cache/ruflo/gaia/judgments`).++## Decision++1. **One writable cache-dir default, applied at all three call sites.**+ Resolve a cache directory for ONNX model downloads with this precedence:+ - `TRANSFORMERS_CACHE` or `HF_HOME`, if the environment already sets one+ (don't override a choice the user or host environment already made);+ - otherwise `~/.cache/ruflo/models`, matching the existing+ `~/.cache/ruflo/<subpath>` convention.++ Centralize this resolution in one shared helper (in+ `transformers-loader.ts`, since it is the only one of the three call+ sites that is already a shared module) rather than tripling the+ precedence logic across `embedding-service.ts`, `memory-initializer.ts`,+ and `task-embedder.ts`. Each call site passes the resolved directory to+ the loaded package's cache-dir mechanism (`env.cacheDir` on the+ `@huggingface/transformers` / `@xenova/transformers` `env` singleton)+ before calling `pipeline('feature-extraction', ...)`.++2. **Never assume a system package directory is writable.** State this as+ the root cause, not an edge case: a global, system-prefix install is+ root-owned by construction, and a runtime user downloading a model into+ it is the fragile path, not a rare misconfiguration. The fix is to stop+ defaulting into package-relative storage at all, not to special-case+ root-owned installs.++3. **Make the degraded state retryable, not permanent.** Distinguish two+ failure classes in `loadLocalEmbeddingChain()`:+ - **No better backend exists at all** β e.g. neither+ `@huggingface/transformers` nor `@xenova/transformers` is installed, or+ `pipeline` isn't exported. This is legitimately permanent for the+ process's dependency set; memoizing `loaded: true` with the hash+ fallback is correct here, since nothing will change without a+ reinstall.+ - **A real backend exists but initialization failed for an+ environmental/fixable reason** (`EACCES`, `ENOSPC`, a transient+ network failure on first download, etc.). This must **not** be+ memoized as permanent degraded state. Either:+ - don't set `embeddingModelState.loaded = true` on this path β leave it+ `null`/unset so the next call to `loadLocalEmbeddingChain()`+ re-attempts initialization, or+ - keep the memoized hash state but record the failure class alongside+ it (e.g. `embeddingModelState.degradedReason: 'permanent' |+ 'retryable'`) and have `generateLocalEmbedding` re-attempt the real+ loader when the reason is `'retryable'`, bounded by a minimum retry+ interval so a persistently broken environment doesn't retry on every+ call.++ Either shape removes the "worker requires a restart after fixing cache+ permissions" symptom β the next real request should recover without a+ process restart once the writable cache directory exists.++4. **Keep ADR-390's truthful `backend` / `embedder` reporting unchanged.**+ This ADR extends it (the retryable/permanent distinction above should be+ visible through the existing status surface, e.g. alongside+ `embeddings_status.ruvectorStatus`) but does not replace or regress it.++## Consequences++**Positive:**+- Closes the actual failure mode in #3692 without requiring provider+ credentials or manual model downloads, matching the reporter's expected+ behavior.+- A single resolution point for the cache directory means a future change+ (e.g. supporting an additional env var) lands once instead of three times.+- Recovery after an admin fixes cache-dir permissions no longer requires+ restarting already-running workers.++**Negative:**+- `task-embedder.ts` and `memory-initializer.ts` each inline their own+ dynamic `import('@xenova/transformers')` rather than routing through+ `transformers-loader.ts`'s ADR-094 provider-agnostic loader (the+ `memory-initializer.ts` site does this deliberately, per ADR-094's+ "Implementation status" note, to avoid a circular optional-dependency at+ install time). The shared cache-dir helper this ADR proposes must be+ importable from all three without reintroducing that circularity β it+ should be a small, dependency-free utility (just path resolution plus+ setting `env.cacheDir`), not bundled with the loader's provider-selection+ logic.+- Honoring `TRANSFORMERS_CACHE` / `HF_HOME` means a host that already has+ one of those set to a non-writable location keeps failing β deliberately+ not overriding an explicit environment choice is weighed above+ auto-recovering from a user's own misconfiguration.++**Risk:**+- The retryable/permanent distinction adds a small amount of state-machine+ complexity to `loadLocalEmbeddingChain()`, which this ADR explicitly keeps+ inside `memory-initializer.ts`'s existing inline state rather than+ introducing a new shared abstraction, to avoid growing the file past the+ 500-line-per-file guidance beyond what's already there.++## Verification++- Unit tests (London-school, with the transformers package mocked):+ - cache-dir resolution prefers `TRANSFORMERS_CACHE`/`HF_HOME` when set, and+ falls back to `~/.cache/ruflo/models` otherwise.+ - `env.cacheDir` (or equivalent) is set before `pipeline(...)` is called,+ at all three call sites.+ - an `EACCES` (or similarly environmental) failure during pipeline init+ does **not** leave `embeddingModelState.loaded === true` pinned to the+ hash fallback in a way that blocks a subsequent real attempt β a+ follow-up call after the condition is fixed succeeds with+ `backend: 'onnx'`.+ - a genuinely missing dependency (no transformers package importable)+ still memoizes the hash fallback permanently β this ADR must not+ regress that case into unbounded retry loops.+ - `generateLocalEmbedding` / `generateEmbedding`'s `backend`/`embedder`+ fields keep reporting accurately in both the recovered and the+ permanently-degraded case (ADR-390 regression guard).+- Manual repro: on a system-prefix global install as root with no+ pre-existing writable cache, run as an unprivileged user; confirm the+ model downloads into `~/.cache/ruflo/models` (or `TRANSFORMERS_CACHE` if+ set) rather than attempting a write under the package directory.