Skip to content

armgit / cindy

Public Imported from GitHub
0

Consider it done. The open-source AI agent that works out of the box · 想到,就能做到。开源、开箱即用的 AI Agent。

fix(chat): 持久化聊天图片缓存并隔离清理范围 (#7442)

bobo4ce99ba24485
diff --git a/apps/desktop/src/main/cindy-media/__tests__/storageIpc.test.ts b/apps/desktop/src/main/cindy-media/__tests__/storageIpc.test.ts
index d93623586d..6351069e88 100644
--- a/apps/desktop/src/main/cindy-media/__tests__/storageIpc.test.ts
+++ b/apps/desktop/src/main/cindy-media/__tests__/storageIpc.test.ts
@@ -26,6 +26,7 @@ vi.mock('electron', () => ({
const schema = await import('../../localDb/schema');
const ledger = await import('../ledger');
const blobStore = await import('../blobStore');
+const { scanCache, evictCacheBlobs } = await import('../recycler');
const { createStorageIpcHandlers } = await import('../storageIpc');
const MIGRATION_0070 = path.resolve(__dirname, '../../../../drizzle/0070_woozy_harpoon.sql');
@@ -68,7 +69,10 @@ afterEach(() => {
const OLD_MS = 100 * 60 * 60 * 1000;
-it('clears only remote cache candidates, preserving attachments, shared bytes and other integrations', async () => {
+// PR #7442: shared remote-media refs were excluded from both cleanup paths (unit).
+// One blob with remote + other cache refs must lose only its remote index, free no
+// shared bytes, and remain reclaimable later. Existing assertions checked blobs only.
+it('clears remote cache indexes while preserving attachments, shared bytes and other integrations', async () => {
const remote = await seedBlob('remote', { isCache: true });
const other = await seedBlob('other', { isCache: true });
const shared = await seedBlob('shared', { isCache: true });
@@ -85,7 +89,8 @@ it('clears only remote cache candidates, preserving attachments, shared bytes an
const clearRemoteMediaMemory = vi.fn();
const handlers = makeHandlers({ clearRemoteMediaMemory });
const scan = await handlers.scan({ scope: 'remote-media', draftUrls: [live.url] });
- expect(scan.cache.evictable.map((entry) => entry.hash)).toEqual([remote.hash]);
+ expect(scan.cache.evictable.map((entry) => entry.hash)).toEqual([remote.hash, shared.hash]);
+ expect(scan.cache.evictable.find(entry => entry.hash === shared.hash)?.bytes).toBe(0);
expect(scan.zeroRef.hashes).toEqual([]);
const result = await handlers.cleanup({
scope: 'remote-media', draftUrls: [live.url],
@@ -96,7 +101,16 @@ it('clears only remote cache candidates, preserving attachments, shared bytes an
expect(result.freedBytes).toBe(remote.bytes);
expect(await ledger.getBlobInfo(remote.hash, db)).toBeNull();
for (const row of [other, shared, pinned, live]) expect(await ledger.getBlobInfo(row.hash, db)).not.toBeNull();
+ expect(await ledger.getIntegrationCacheHash(`remote-media:${shared.hash}`, db)).toBeNull();
+ expect(await ledger.getIntegrationCacheHash(`other:${shared.hash}`, db)).toBe(shared.hash);
+ expect(fs.readFileSync(blobStore.resolveHashRef(shared.hash, shared.ext).absPath).toString()).toBe('shared');
+ for (const row of [pinned, live])
+ expect(await ledger.getIntegrationCacheHash(`remote-media:${row.hash}`, db)).toBe(row.hash);
expect(clearRemoteMediaMemory).toHaveBeenCalledTimes(2);
+ expect((await handlers.scan({ scope: 'remote-media', draftUrls: [live.url] })).cache.evictable).toEqual([]);
+ expect((await scanCache({ live: new Set([live.hash]), limitBytes: 0 }, db)).evictable.map(row => row.hash))
+ .toContain(shared.hash);
+ expect((await evictCacheBlobs({ hashes: [shared.hash], live: new Set() }, db)).deleted).toBe(1);
});
async function seedBlob(
diff --git a/apps/desktop/src/main/cindy-media/ledger.ts b/apps/desktop/src/main/cindy-media/ledger.ts
index fa3486afc3..148c429163 100644
--- a/apps/desktop/src/main/cindy-media/ledger.ts
+++ b/apps/desktop/src/main/cindy-media/ledger.ts
@@ -624,23 +624,43 @@ export async function getIntegrationCacheBytes(integration: string, db: LedgerDb
/** cache 性质 blob 按 LRU 升序(最久未用在前),供逐出候选规划。 */
export async function listCacheBlobsByLru(db: LedgerDb = defaultDb(), integration?: string): Promise<CacheBlobRow[]> {
+ const hasOtherRefs = sql`EXISTS (SELECT 1 FROM ${mediaRefs} WHERE ${mediaRefs.hash} = ${mediaBlobs.hash}
+ AND (${mediaRefs.refKind} <> 'integration-cache' OR ${mediaRefs.originId} IS NULL OR ${mediaRefs.originId} <> ${integration ?? ''}))`;
+ // Manual image cleanup may unlink its own index without reclaiming shared bytes.
+ // Keep those candidates visible, but do not promise space that other refs protect.
+ const unlinkShared = integration === 'remote-media';
return await db
.select({
hash: mediaBlobs.hash,
ext: mediaBlobs.ext,
- bytes: mediaBlobs.bytes,
+ bytes: unlinkShared ? sql<number>`CASE WHEN ${hasOtherRefs} THEN 0 ELSE ${mediaBlobs.bytes} END` : mediaBlobs.bytes,
lastAccessAt: mediaBlobs.lastAccessAt,
})
.from(mediaBlobs)
.where(and(eq(mediaBlobs.isCache, true), integration ? sql`EXISTS (
SELECT 1 FROM ${mediaRefs} WHERE ${mediaRefs.hash} = ${mediaBlobs.hash}
AND ${mediaRefs.refKind} = 'integration-cache' AND ${mediaRefs.originId} = ${integration}
- ) AND NOT EXISTS (SELECT 1 FROM ${mediaRefs} WHERE ${mediaRefs.hash} = ${mediaBlobs.hash}
- AND (${mediaRefs.refKind} <> 'integration-cache' OR ${mediaRefs.originId} IS NULL OR ${mediaRefs.originId} <> ${integration}))` : undefined))
+ ) ${unlinkShared ? sql`` : sql`AND NOT ${hasOtherRefs}`}`
+ // Long-lived chat image copies have their own explicit cleanup category.
+ // Shared bytes stay protected even when an ordinary cache also references them.
+ : sql`NOT EXISTS (SELECT 1 FROM ${mediaRefs} WHERE ${mediaRefs.hash} = ${mediaBlobs.hash}
+ AND ${mediaRefs.refKind} = 'integration-cache' AND ${mediaRefs.originId} = 'remote-media')`))
.orderBy(asc(mediaBlobs.lastAccessAt))
.all();
}
+/** Drop only this cache's index; a pinned blob and every other owner's ref survive. */
+export async function removeIntegrationCacheRefs(
+ hash: string,
+ integration: string,
+ db: LedgerDb = defaultDb(),
+): Promise<void> {
+ await db.delete(mediaRefs).where(and(
+ eq(mediaRefs.hash, hash), eq(mediaRefs.refKind, 'integration-cache'), eq(mediaRefs.originId, integration),
+ sql`EXISTS (SELECT 1 FROM ${mediaBlobs} WHERE ${mediaBlobs.hash} = ${mediaRefs.hash} AND ${mediaBlobs.isCache} = 1)`,
+ )).run();
+}
+
/**
* 零引用条件删账(recycler 唯一的零引用账删口):零引用 + 缓冲期两个条件
* 在同一条 DELETE 里复查——扫描到执行之间若有人挂了引用或刷新了访问时间,
@@ -685,7 +705,9 @@ export async function deleteCacheBlobRecord(
eq(mediaBlobs.isCache, true),
sql`NOT EXISTS (SELECT 1 FROM ${mediaRefs} WHERE ${mediaRefs.hash} = ${mediaBlobs.hash} AND ${mediaRefs.refKind} <> 'integration-cache')`,
integration ? sql`NOT EXISTS (SELECT 1 FROM ${mediaRefs} WHERE ${mediaRefs.hash} = ${mediaBlobs.hash}
- AND (${mediaRefs.originId} IS NULL OR ${mediaRefs.originId} <> ${integration}))` : undefined,
+ AND (${mediaRefs.originId} IS NULL OR ${mediaRefs.originId} <> ${integration}))`
+ : sql`NOT EXISTS (SELECT 1 FROM ${mediaRefs} WHERE ${mediaRefs.hash} = ${mediaBlobs.hash}
+ AND ${mediaRefs.refKind} = 'integration-cache' AND ${mediaRefs.originId} = 'remote-media')`,
integration ? sql`EXISTS (SELECT 1 FROM ${mediaRefs} WHERE ${mediaRefs.hash} = ${mediaBlobs.hash}
AND ${mediaRefs.originId} = ${integration})` : undefined,
),
diff --git a/apps/desktop/src/main/cindy-media/recycler.ts b/apps/desktop/src/main/cindy-media/recycler.ts
index 9162857898..8d763883a1 100644
--- a/apps/desktop/src/main/cindy-media/recycler.ts
+++ b/apps/desktop/src/main/cindy-media/recycler.ts
@@ -221,6 +221,10 @@ export async function evictCacheBlobs(
}
const removed = await ledger.deleteCacheBlobRecord(hash, db, params.integration);
if (!removed) {
+ if (params.integration === 'remote-media') {
+ params.assertStillValid?.();
+ await ledger.removeIntegrationCacheRefs(hash, params.integration, db);
+ }
result.skipped++;
continue;
}
diff --git a/apps/desktop/src/main/cindy-media/storageIpc.ts b/apps/desktop/src/main/cindy-media/storageIpc.ts
index 7823fbf12f..c3a4a62a1c 100644
--- a/apps/desktop/src/main/cindy-media/storageIpc.ts
+++ b/apps/desktop/src/main/cindy-media/storageIpc.ts
@@ -28,7 +28,7 @@ import * as legacyDeadDirs from './legacyDeadDirs';
import type { FixedDirectoryStats } from './fixedDirectory';
import type { LiveHashSources } from './recycler';
import { getDbClient } from '../localDb/client/current';
-import { captureRemoteMediaOwner, invalidateRemoteMediaPersistentWrites, REMOTE_MEDIA_INTEGRATION } from '../device-link/remoteMediaPersistentCache';
+import { beginRemoteMediaCacheCleanup, captureRemoteMediaOwner, REMOTE_MEDIA_INTEGRATION } from '../device-link/remoteMediaPersistentCache';
const log = createLogger('cindy-media-storage-ipc');
@@ -296,19 +296,24 @@ export function createStorageIpcHandlers(deps: StorageIpcDeps) {
if (params.scope === 'remote-media') {
const assertCurrent = captureRemoteMediaOwner();
const db = deps.db ?? getDbClient().drizzle;
- invalidateRemoteMediaPersistentWrites();
- deps.clearRemoteMediaMemory?.();
- const live = await collectLive(deps, params.draftUrls ?? []);
- assertCurrent();
- const cacheEvicted = await recycler.evictCacheBlobs({
- hashes: params.evictCacheHashes ?? [], live,
- integration: REMOTE_MEDIA_INTEGRATION, assertStillValid: assertCurrent,
- }, db);
- assertCurrent();
- deps.clearRemoteMediaMemory?.();
- return { zeroRef: { deleted: 0, freedBytes: 0, skipped: 0 }, cacheEvicted,
- deadDirs: { removed: [], skipped: [], freedBytes: 0 }, tmpFilesRemoved: 0,
- freedBytes: cacheEvicted.freedBytes };
+ const finishCleanup = beginRemoteMediaCacheCleanup();
+ try {
+ deps.clearRemoteMediaMemory?.();
+ const live = await collectLive(deps, params.draftUrls ?? []);
+ assertCurrent();
+ const cacheEvicted = await recycler.evictCacheBlobs({
+ hashes: params.evictCacheHashes ?? [], live,
+ integration: REMOTE_MEDIA_INTEGRATION, assertStillValid: assertCurrent,
+ }, db);
+ assertCurrent();
+ return { zeroRef: { deleted: 0, freedBytes: 0, skipped: 0 }, cacheEvicted,
+ deadDirs: { removed: [], skipped: [], freedBytes: 0 }, tmpFilesRemoved: 0,
+ freedBytes: cacheEvicted.freedBytes };
+ } finally {
+ // Release this cleanup's fence and invalidate old projections even on failure.
+ finishCleanup();
+ deps.clearRemoteMediaMemory?.();
+ }
}
// 执行时重新取证:扫描→确认之间用户可能又粘了图/发了排队消息。
const live = await collectLive(deps, params.draftUrls ?? []);
diff --git a/apps/desktop/src/main/device-link/__tests__/remoteMediaPersistentCache.test.ts b/apps/desktop/src/main/device-link/__tests__/remoteMediaPersistentCache.test.ts
index 34e1a72ad8..ddf9493bd3 100644
--- a/apps/desktop/src/main/device-link/__tests__/remoteMediaPersistentCache.test.ts
+++ b/apps/desktop/src/main/device-link/__tests__/remoteMediaPersistentCache.test.ts
@@ -15,12 +15,14 @@ const state = vi.hoisted(() => ({
db: undefined as LedgerDb | undefined,
}));
const peer = vi.hoisted(() => vi.fn());
+const fetchMedia = vi.hoisted(() => vi.fn());
vi.mock('../filePeer', () => ({ tryPeerFile: peer }));
vi.mock('../mediaTransfer', () => ({ removeRemote: vi.fn() }));
-vi.mock('electron', () => ({ app: { getPath: () => state.root } }));
+vi.mock('electron', () => ({ app: { getPath: () => state.root }, net: { fetch: fetchMedia } }));
vi.mock('../../appSessionState', () => ({
getActiveAppSession: () => ({ mode: 'cloud', dataOwnerId: state.owner }),
activeOwnerScopeKey: () => `${state.owner}:${state.generation}`,
+ ownerScopedUserDataPath: () => state.root,
}));
vi.mock('../../localDb/client/current', () => ({ getDbClient: () => ({ drizzle: state.db }) }));
vi.mock('../broadcast-tap', () => ({
@@ -31,6 +33,9 @@ vi.mock('../broadcast-tap', () => ({
import * as schema from '../../localDb/schema';
import { listAllBlobRecords, listCacheBlobsByLru } from '../../cindy-media/ledger';
import { resolveHashRef } from '../../cindy-media/blobStore';
+import * as blobStore from '../../cindy-media/blobStore';
+import { evictCacheBlobs, scanCache } from '../../cindy-media/recycler';
+import { createStorageIpcHandlers } from '../../cindy-media/storageIpc';
import {
invalidateRemoteMediaPersistentWrites,
createSentImageRetainer,
@@ -38,6 +43,8 @@ import {
} from '../remoteMediaPersistentCache';
import { remoteFileVersionCache } from '../remoteFileVersionCache';
import { readRemoteDeviceFile } from '../fileAccess';
+import { createChatMedia } from '../../maker-ipc/chatServerMedia';
+import type { ChatApi } from '../../maker-ipc/chatServerMigration';
const { default: migration0071 } =
(await import('../../../../drizzle/scripts/0071_bright_ultron')) as {
@@ -46,6 +53,9 @@ const { default: migration0071 } =
let raw: Database.Database;
const png = Buffer.from('89504e470d0a1a0a0000000d49484452', 'hex');
const url = `cindy-media://blobs/${createHash('sha256').update(png).digest('hex')}.png`;
+const chatEndpoint = 'https://chat.example.test';
+const chatMetadata = () => ({ name: 'image.png', type: 'image/png', size: png.length,
+ url: 'https://download.example.test/signed' });
function reopen() {
raw = new Database(path.join(state.root, 'ledger.db'));
raw.pragma('foreign_keys = ON');
@@ -53,6 +63,7 @@ function reopen() {
}
beforeEach(async () => {
peer.mockReset();
+ fetchMedia.mockReset().mockImplementation(async () => new Response(new Uint8Array(png)));
state.root = await fs.mkdtemp(path.join(os.tmpdir(), 'remote-image-cache-'));
state.owner = 'alice';
state.generation = 1;
@@ -66,6 +77,207 @@ beforeEach(async () => {
migration0071.run(raw);
});
+// Regression: reopening a group/topic/direct chat downloaded every image again because
+// createChatMedia only indexed server media IDs in memory. Existing tests covered device
+// transfers, not Chat Server IDs. Minimum input: one image, then recreate the service and
+// reopen its real SQLite ledger (unit tier with an isolated temporary DB, no real network).
+it('reuses a Chat Server image across service and database restart without downloading its bytes again', async () => {
+ const api = vi.fn(async () => ({ name: 'image.png', type: 'image/png', size: png.length,
+ url: 'https://download.example.test/first-signature' })) as unknown as ChatApi;
+ const first = await createChatMedia(api, () => true, 'https://chat.example.test').download('room', 'media');
+ raw.close(); state.generation++; reopen();
+ vi.mocked(api).mockResolvedValue({ name: 'image.png', type: 'image/png', size: png.length,
+ url: 'https://download.example.test/new-signature' });
+ const restored = await createChatMedia(api, () => true, 'https://chat.example.test').download('room', 'media');
+ expect(restored).toEqual(first);
+ expect(fetchMedia).toHaveBeenCalledTimes(1);
+ expect(api).toHaveBeenCalledTimes(2); // Permission is still checked before local reuse.
+ expect(await listCacheBlobsByLru(state.db, 'remote-media')).toHaveLength(1);
+});
+
+it('retains the actual Chat Server upload so reopening its sent image needs no download', async () => {
+ const source = path.join(state.root, 'upload.png');
+ await fs.writeFile(source, png);
+ const api = vi.fn(async (route: string, method?: string) => method !== 'POST' ? chatMetadata()
+ : route.endsWith('/complete') ? {} : { id: 'media', uploadUrl: 'https://upload.example.test/signed' }) as unknown as ChatApi;
+ const service = createChatMedia(api, () => true, chatEndpoint);
+ expect(await service.upload('room', 'operation', [{ id: 'file', name: 'image.png', mimeType: 'image/png',
+ size: png.length, category: 'image', path: source, url: null }], 'actor'))
+ .toEqual([{ type: 'media', mediaId: 'media', caption: 'image.png' }]);
+ await fs.writeFile(source, Buffer.alloc(png.length));
+ raw.close(); state.generation++; reopen();
+ expect((await createChatMedia(api, () => true, chatEndpoint).download('room', 'media')).url).toBe(url);
+ expect(fetchMedia).toHaveBeenCalledTimes(1); // PUT only, never GET.
+ expect(fetchMedia.mock.calls[0][1].method).toBe('PUT');
+});
+
+it.each(['account', 'endpoint', 'room', 'media'] as const)('isolates Chat Server images by %s and still refuses revoked access', async (scope) => {
+ const api = vi.fn(async () => chatMetadata()) as unknown as ChatApi;
+ await createChatMedia(api, () => true, chatEndpoint).download('room', 'media');
+ if (scope === 'account') { state.owner = 'bob'; state.generation++; }
+ const service = createChatMedia(api, () => true, scope === 'endpoint' ? 'https://other.example.test' : chatEndpoint);
+ const room = scope === 'room' ? 'other-room' : 'room';
+ const mediaId = scope === 'media' ? 'other-media' : 'media';
+ await service.download(room, mediaId);
+ expect(fetchMedia).toHaveBeenCalledTimes(2);
+ vi.mocked(api).mockRejectedValue(new Error('CONVERSATION_NOT_FOUND'));
+ await expect(service.download(room, mediaId)).rejects.toThrow('CONVERSATION_NOT_FOUND');
+ expect(fetchMedia).toHaveBeenCalledTimes(2);
+});
+
+it.each(['missing', 'corrupt'] as const)('repairs a %s Chat Server image without restarting the service', async (damage) => {
+ const service = createChatMedia(vi.fn(async () => chatMetadata()) as unknown as ChatApi, () => true, chatEndpoint);
+ await service.download('room', 'media');
+ const [record] = await listAllBlobRecords(state.db);
+ const file = resolveHashRef(record.hash, record.ext).absPath;
+ if (damage === 'missing') await fs.rm(file);
+ else await fs.writeFile(file, Buffer.alloc(png.length));
+ expect((await service.download('room', 'media')).url).toBe(url);
+ expect(fetchMedia).toHaveBeenCalledTimes(2);
+ expect(await fs.readFile(file)).toEqual(png);
+});
+
+it('keeps Chat Server images out of automatic file eviction and lets manual image cleanup remove them', async () => {
+ const service = createChatMedia(vi.fn(async () => chatMetadata()) as unknown as ChatApi, () => true, chatEndpoint);
+ await service.download('room', 'media');
+ const [record] = await listAllBlobRecords(state.db);
+ expect((await scanCache({ live: new Set(), limitBytes: 0, integration: 'remote-file' }, state.db)).evictable).toEqual([]);
+ expect((await scanCache({ live: new Set(), limitBytes: 0 }, state.db)).evictable).toEqual([]);
+ expect((await evictCacheBlobs({ hashes: [record.hash], live: new Set() }, state.db)).deleted).toBe(0);
+ const generation = service.cacheGeneration;
+ invalidateRemoteMediaPersistentWrites();
+ expect((await evictCacheBlobs({ hashes: [record.hash], live: new Set(), integration: 'remote-media' }, state.db)).deleted).toBe(1);
+ expect(service.cacheGeneration).not.toBe(generation);
+ expect((await service.download('room', 'media')).url).toBe(url);
+ expect(fetchMedia).toHaveBeenCalledTimes(2);
+});
+
+// PR #7442: a lookup started after cleanup's initial invalidation could publish a
+// stale hit or recreate the cleared index. Unit timing regression through the real
+// cleanup handler + downloader; existing cases only started reads before cleanup.
+it.each(['success', 'failure'] as const)('rejects an image lookup started during cleanup after its %s', async outcome => {
+ const api = vi.fn(async () => chatMetadata()) as unknown as ChatApi;
+ const service = createChatMedia(api, () => true, chatEndpoint);
+ await service.download('room', 'media');
+ const [record] = await listAllBlobRecords(state.db);
+ let releaseScan!: () => void;
+ const scanBarrier = new Promise<void>(resolve => { releaseScan = resolve; });
+ const handlers = createStorageIpcHandlers({ db: state.db, getQueueScanTexts: () => [],
+ loadSnapshotPayloads: async () => {
+ await scanBarrier;
+ if (outcome === 'failure') throw new Error('scan failed');
+ return [];
+ } });
+ const cleanup = handlers.cleanup({ scope: 'remote-media', draftUrls: [], zeroRefHashes: [],
+ evictCacheHashes: [record.hash], deadDirNames: [], cleanTmpFiles: false })
+ .then(() => null, error => error);
+ let releaseMetadata!: (value: ReturnType<typeof chatMetadata>) => void;
+ vi.mocked(api).mockReturnValueOnce(new Promise(resolve => { releaseMetadata = resolve; }));
+ const pending = service.download('room', 'media');
+ releaseScan();
+ const cleanupError = await cleanup;
+ if (outcome === 'failure') expect(cleanupError).toBeInstanceOf(Error);
+ else expect(cleanupError).toBeNull();
+ releaseMetadata(chatMetadata());
+ await expect(pending).rejects.toThrow('REMOTE_MEDIA_CACHE_CLEARED');
+ expect(fetchMedia).toHaveBeenCalledTimes(1);
+ expect(await listAllBlobRecords(state.db)).toHaveLength(outcome === 'success' ? 0 : 1);
+});
+
+// PR #7442: end-generation invalidation misses writes committed during a multi-blob
+// clear. Unit regression: pause real cleanup after deleting its first file, complete
+// another cleanup, then try both the Chat Server read and the persistent put entry.
+it('fences reads and writes until every overlapping cleanup finishes', async () => {
+ const service = createChatMedia(vi.fn(async () => chatMetadata()) as unknown as ChatApi, () => true, chatEndpoint);
+ await service.download('room', 'media');
+ const [first] = await listAllBlobRecords(state.db);
+ const secondBytes = Buffer.concat([png, Buffer.from([1])]);
+ const secondHash = createHash('sha256').update(secondBytes).digest('hex');
+ const secondUrl = `cindy-media://blobs/${secondHash}.png`;
+ await remoteMediaPersistentCache('computer', secondUrl)!.put(secondBytes, 'image/png');
+ let deletedFirst!: () => void;
+ const firstDeleted = new Promise<void>(resolve => { deletedFirst = resolve; });
+ let release!: () => void;
+ const barrier = new Promise<void>(resolve => { release = resolve; });
+ const deleteFile = blobStore.deleteBlobFile;
+ const deleting = vi.spyOn(blobStore, 'deleteBlobFile').mockImplementation(async (...args) => {
+ await deleteFile(...args);
+ if (args[0] === first.hash) { deletedFirst(); await barrier; }
+ });
+ const handlers = createStorageIpcHandlers({ db: state.db, getQueueScanTexts: () => [], loadSnapshotPayloads: async () => [] });
+ const params = { scope: 'remote-media' as const, draftUrls: [], zeroRefHashes: [],
+ evictCacheHashes: [first.hash, secondHash], deadDirNames: [], cleanTmpFiles: false };
+ const cleanup = handlers.cleanup(params);
+ try {
+ await firstDeleted;
+ await handlers.cleanup({ ...params, evictCacheHashes: [] });
+ await expect(service.download('room', 'media')).rejects.toThrow('REMOTE_MEDIA_CACHE_CLEARED');
+ await expect(remoteMediaPersistentCache('computer', url)!.put(png, 'image/png'))
+ .rejects.toThrow('REMOTE_MEDIA_CACHE_CLEARED');
+ expect(fetchMedia).toHaveBeenCalledTimes(1);
+ } finally {
+ release();
+ await cleanup.finally(() => deleting.mockRestore());
+ }
+ expect(await listAllBlobRecords(state.db)).toEqual([]);
+ expect((await service.download('room', 'media')).url).toBe(url);
+ expect(fetchMedia).toHaveBeenCalledTimes(2);
+});
+
+// PR #7442 review: ordinary attachments have no persistent image index. Two reads of
+// one PDF must retain the original service-local reuse, including across image cleanup.
+// This unit fixture exercises the real downloader/filesystem; previous cases only used PNG.
+it('reuses ordinary Chat Server files across reads and image cleanup while rechecking access', async () => {
+ const pdf = Buffer.from('%PDF-1.4 test');
+ fetchMedia.mockImplementation(async () => new Response(new Uint8Array(pdf)));
+ const api = vi.fn(async () => ({ ...chatMetadata(), name: 'report.pdf', type: 'application/pdf', size: pdf.length })) as unknown as ChatApi;
+ const service = createChatMedia(api, () => true, chatEndpoint);
+ const first = await service.download('room', 'file');
+ expect(await service.download('room', 'file')).toEqual(first);
+ expect(fetchMedia).toHaveBeenCalledTimes(1);
+ invalidateRemoteMediaPersistentWrites();
+ expect(await service.download('room', 'file')).toEqual(first);
+ expect(await fs.readFile(first.path!)).toEqual(pdf);
+ expect(fetchMedia).toHaveBeenCalledTimes(1);
+ vi.mocked(api).mockRejectedValueOnce(new Error('CONVERSATION_NOT_FOUND'));
+ await expect(service.download('room', 'file')).rejects.toThrow('CONVERSATION_NOT_FOUND');
+ expect(api).toHaveBeenCalledTimes(4);
+});
+
+it('does not cancel an ordinary file transfer when the image cache is cleared', async () => {
+ const pdf = Buffer.from('%PDF-1.4 test');
+ fetchMedia.mockImplementationOnce(async () => {
+ invalidateRemoteMediaPersistentWrites();
+ return new Response(new Uint8Array(pdf));
+ });
+ const api = vi.fn(async () => ({ ...chatMetadata(), name: 'report.pdf', type: 'application/pdf', size: pdf.length })) as unknown as ChatApi;
+ const service = createChatMedia(api, () => true, chatEndpoint);
+ const file = await service.download('room', 'file');
+ expect(await fs.readFile(file.path!)).toEqual(pdf);
+ expect(await service.download('room', 'file')).toEqual(file);
+ expect(fetchMedia).toHaveBeenCalledTimes(1);
+});
+
+it.each(['account', 'clear'] as const)('rejects a Chat Server image arriving after %s changes', async (change) => {
+ const changeScope = () => {
+ if (change === 'account') { state.owner = 'bob'; state.generation++; }
+ else invalidateRemoteMediaPersistentWrites();
+ };
+ fetchMedia.mockImplementationOnce(async () => { changeScope(); return new Response(new Uint8Array(png)); });
+ const service = createChatMedia(vi.fn(async () => chatMetadata()) as unknown as ChatApi, () => state.owner === 'alice', chatEndpoint);
+ await expect(service.download('room', 'media')).rejects.toThrow();
+ expect(await listAllBlobRecords(state.db)).toEqual([]);
+});
+
+it('does not turn a sealed Chat Server upload into a failed send when image storage is cleared', async () => {
+ fetchMedia.mockImplementationOnce(async () => { invalidateRemoteMediaPersistentWrites(); return new Response(''); });
+ const api = vi.fn(async () => ({ id: 'media', uploadUrl: 'https://upload.example.test/signed' })) as unknown as ChatApi;
+ await expect(createChatMedia(api, () => true, chatEndpoint).upload('room', 'operation', [{ id: 'file', name: 'image.png',
+ mimeType: 'image/png', size: png.length, category: 'image', path: null, url: null, readBytes: async () => png }], 'actor'))
+ .resolves.toEqual([{ type: 'media', mediaId: 'media', caption: 'image.png' }]);
+ expect(await listAllBlobRecords(state.db)).toEqual([]);
+});
+
it.each(['prepare', 'peer-queue'])('does not recreate the disk index after clearing during %s', async (phase) => {
const metadata = { ossKey: '', transferRequired: true, version: 'b'.repeat(64), size: png.length, mimeType: 'image/png' };
const invoke = vi.fn(async () => {
diff --git a/apps/desktop/src/main/device-link/remoteMediaPersistentCache.ts b/apps/desktop/src/main/device-link/remoteMediaPersistentCache.ts
index c86e14ad2b..14447cef67 100644
--- a/apps/desktop/src/main/device-link/remoteMediaPersistentCache.ts
+++ b/apps/desktop/src/main/device-link/remoteMediaPersistentCache.ts
@@ -11,9 +11,25 @@ import { captureDataOwnerBroadcastScope, isDataOwnerBroadcastScopeCurrent } from
export const REMOTE_MEDIA_INTEGRATION = 'remote-media';
let cacheGeneration = 0;
+let activeCleanups = 0;
export function invalidateRemoteMediaPersistentWrites(): void {
cacheGeneration++;
}
+/** Keep reads/writes fenced until every overlapping cleanup has finished. */
+export function beginRemoteMediaCacheCleanup(): () => void {
+ activeCleanups++;
+ invalidateRemoteMediaPersistentWrites();
+ let finished = false;
+ return () => {
+ if (finished) return;
+ finished = true;
+ activeCleanups--;
+ invalidateRemoteMediaPersistentWrites();
+ };
+}
+export function remoteMediaCacheGeneration(): number {
+ return cacheGeneration;
+}
export function captureRemoteMediaOwner(): () => void {
const owner = captureDataOwnerBroadcastScope();
@@ -27,7 +43,7 @@ export function captureRemoteMediaCacheValidity(): () => void {
const generation = cacheGeneration;
return () => {
assertOwner();
- if (generation !== cacheGeneration) throw new Error('REMOTE_MEDIA_CACHE_CLEARED');
+ if (activeCleanups > 0 || generation !== cacheGeneration) throw new Error('REMOTE_MEDIA_CACHE_CLEARED');
};
}
@@ -79,12 +95,25 @@ export function createSentImageRetainer(device: string) {
* Capture the database and owner before downloading so late work cannot enter a new account. */
export function remoteMediaPersistentCache(device: string, url: string) {
if (!isImmutableRemoteMedia(url)) return null;
+ return persistentImageCache([device, url], url.startsWith('cindy-media://')
+ ? url.split('/').pop()!.split('.')[0] : undefined);
+}
+
+/** Chat Server media IDs are immutable, but their signed download URLs expire.
+ * Keep their index separate from device URLs, with the same manual cleanup lifecycle.
+ * The caller must authorize the room/media before using a hit. */
+export function chatMediaPersistentCache(endpoint: string, roomId: string, mediaId: string, assertValid: () => void) {
+ return persistentImageCache(['chat-server', endpoint, roomId, mediaId], undefined, assertValid);
+}
+
+function persistentImageCache(identity: string[], expectedHash?: string, assertValid?: () => void) {
const session = getActiveAppSession();
if (!session.dataOwnerId) return null;
- const assertCurrent = captureRemoteMediaCacheValidity();
+ const assertCache = captureRemoteMediaCacheValidity();
+ const assertCurrent = () => { assertValid?.(); assertCache(); };
const db = getDbClient().drizzle;
const key = `${REMOTE_MEDIA_INTEGRATION}:${createHash('sha256')
- .update(JSON.stringify([session.mode, session.dataOwnerId, device, url]))
+ .update(JSON.stringify([session.mode, session.dataOwnerId, ...identity]))
.digest('hex')}`;
return {
async get(maxBytes: number) {
@@ -92,10 +121,12 @@ export function remoteMediaPersistentCache(device: string, url: string) {
const hit = await integrationCacheGet(key, db);
assertCurrent();
if (!hit || !hit.mimeType.startsWith('image/')) return null;
- const file = await fs.open(hit.absPath, constants.O_RDONLY | (constants.O_NOFOLLOW ?? 0));
+ const file = await fs.open(hit.absPath, constants.O_RDONLY | (constants.O_NOFOLLOW ?? 0)).catch(() => null);
+ if (!file) { assertCurrent(); return null; }
try {
+ assertCurrent();
const stat = await file.stat();
- if (!stat.isFile() || stat.size > maxBytes) return null;
+ if (!stat.isFile() || stat.size <= 0 || stat.size > maxBytes) return null;
const bytes = Buffer.alloc(stat.size);
let offset = 0;
while (offset < bytes.length) {
@@ -110,7 +141,7 @@ export function remoteMediaPersistentCache(device: string, url: string) {
createHash('sha256').update(bytes).digest('hex') !== hit.hash
)
return null;
- return { bytes, mimeType: hit.mimeType };
+ return { bytes, mimeType: hit.mimeType, url: hit.url };
} finally {
await file.close();
}
@@ -121,11 +152,10 @@ export function remoteMediaPersistentCache(device: string, url: string) {
if (!verified?.startsWith('image/')) return;
// Never attach bytes to a remote content hash that they do not actually match.
if (
- url.startsWith('cindy-media://') &&
- createHash('sha256').update(bytes).digest('hex') !== url.split('/').pop()!.split('.')[0]
+ expectedHash && createHash('sha256').update(bytes).digest('hex') !== expectedHash
)
return;
- await integrationCachePut(
+ return integrationCachePut(
{
cacheKey: key,
integration: REMOTE_MEDIA_INTEGRATION,
diff --git a/apps/desktop/src/main/maker-ipc/__tests__/chatServerGroupOpen.test.ts b/apps/desktop/src/main/maker-ipc/__tests__/chatServerGroupOpen.test.ts
index ed00169cd2..0e25278602 100644
--- a/apps/desktop/src/main/maker-ipc/__tests__/chatServerGroupOpen.test.ts
+++ b/apps/desktop/src/main/maker-ipc/__tests__/chatServerGroupOpen.test.ts
@@ -9,13 +9,14 @@
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const fixture = vi.hoisted(() => ({ handle: vi.fn(), latencyMs: 100, profiles: [] as unknown[], localGroupIds: [] as string[],
- sockets: [] as Array<import('ws').WebSocket>, routes: [] as string[], download: vi.fn(), page: [] as unknown[],
+ sockets: [] as Array<import('ws').WebSocket>, routes: [] as string[], download: vi.fn(), cacheGeneration: 0, page: [] as unknown[],
/** When the server reads its state and when the answer reaches the client, in ms after the request. */
timing: null as null | ((route: string) => { serve: number; deliver: number }),
offline: false, onFrame: null as null | ((socket: unknown, frame: Record<string, unknown>) => void) }));
vi.mock('../../authManager.js', () => ({ getAccessToken: () => 'isolated-test-token', refresh: vi.fn(async () => true) }));
vi.mock('../../clientEndpointsService.js', () => ({ getClientEndpoint: () => 'https://chat.cindy.app' }));
-vi.mock('../chatServerMedia.js', () => ({ createChatMedia: () => ({ upload: vi.fn(async () => []), download: fixture.download }) }));
+vi.mock('../chatServerMedia.js', () => ({ createChatMedia: () => ({ upload: vi.fn(async () => []), download: fixture.download,
+ get cacheGeneration() { return fixture.cacheGeneration; } }) }));
vi.mock('../chatServerWorkspaces.js', () => ({ chatServerWorkspaces: () => ({ read: () => null, save: vi.fn() }) }));
vi.mock('../chatSendJournal.js', () => ({ chatSendJournal: () => ({ read: () => null, save: vi.fn(), confirm: vi.fn(), forgetRoom: vi.fn() }) }));
vi.mock('../chatLocalHistory.js', () => ({ chatLocalHistory: () => ({ read: () => null, save: vi.fn(), forget: vi.fn() }) }));
@@ -112,7 +113,7 @@ describe('Chat Server group open latency (simulated 100ms per request)', () => {
vi.setSystemTime(new Date('2026-10-03T00:00:00Z'));
vi.stubEnv('XDT_ISOLATED', '1');
fixture.routes = []; fixture.sockets = []; fixture.profiles = []; fixture.localGroupIds = []; fixture.latencyMs = 100;
- fixture.timing = null; fixture.offline = false; fixture.onFrame = null;
+ fixture.timing = null; fixture.offline = false; fixture.onFrame = null; fixture.cacheGeneration = 0;
fixture.page = [message];
fixture.handle.mockImplementation(response);
fixture.download.mockResolvedValue(null);
@@ -480,6 +481,85 @@ describe('Chat Server group open latency (simulated 100ms per request)', () => {
expect(reread.value.ok && reread.value.group.messages.every(item => item.attachments.length === 1)).toBe(true);
});
+ // Unit regression: getGroup kept an image's now-deleted local URL forever after manual
+ // cleanup. Existing attachment tests only retried failed transfers, not successful copies
+ // invalidated by cleanup. Minimum input: one image, one PDF and a cache-generation change.
+ it('reloads only image attachments after their dedicated cache is cleared', async () => {
+ const mediaId = '60000000-0000-4000-8000-000000000201';
+ const fileId = '60000000-0000-4000-8000-000000000202';
+ const file = { id: fileId, name: 'report.pdf', category: 'file', path: '/test/report.pdf', url: null };
+ fixture.page = [{ ...message, content: [mediaId, fileId].map(mediaId => ({ type: 'media', mediaId })) }];
+ let hash = 'a';
+ fixture.download.mockImplementation(async (_room: string, id: string) => id === fileId ? file
+ : { id, name: 'image.png', category: 'image', path: null, url: `cindy-media://blobs/${hash.repeat(64)}.png` });
+ await timed(() => service.getGroup(rooms[0]));
+ const loaded = await timed(() => service.getGroup(rooms[0]));
+ expect(loaded.value.ok && loaded.value.group.messages[0].attachments[0].url).toContain('a'.repeat(64));
+ expect(fixture.download).toHaveBeenCalledTimes(2);
+ fixture.cacheGeneration++;
+ hash = 'b';
+ await timed(() => service.getGroup(rooms[0]));
+ const reloaded = await timed(() => service.getGroup(rooms[0]));
+ expect(reloaded.value.ok && reloaded.value.group.messages[0].attachments[0].url).toContain('b'.repeat(64));
+ expect(reloaded.value.ok && reloaded.value.group.messages[0].attachments[1]).toEqual(file);
+ expect(fixture.download.mock.calls.map(([, id]) => id)).toEqual([mediaId, fileId, mediaId]);
+ });
+
+ // PR #7442: cleanup must not duplicate or discard a pending ordinary file transfer.
+ // Unit, minimum input: one PDF still downloading when the image generation changes.
+ it('keeps a pending ordinary attachment projection across image cleanup', async () => {
+ const mediaId = '60000000-0000-4000-8000-000000000202';
+ const file = { id: mediaId, name: 'report.pdf', category: 'file', path: '/test/report.pdf', url: null };
+ let complete!: (value: typeof file) => void;
+ fixture.download.mockReturnValue(new Promise<typeof file>(resolve => { complete = resolve; }));
+ fixture.page = [{ ...message, content: [{ type: 'media', mediaId }] }];
+ await timed(() => service.getGroup(rooms[0]));
+ fixture.cacheGeneration++;
+ await timed(() => service.getGroup(rooms[0]));
+ expect(fixture.download).toHaveBeenCalledTimes(1);
+ complete(file);
+ const loaded = await timed(() => service.getGroup(rooms[0]));
+ expect(loaded.value.ok && loaded.value.group.messages[0].attachments[0]).toEqual(file);
+ });
+
+ // Unit timing regression: preserving pending files must not retain a late image URL.
+ // Existing settled-image coverage cannot exercise cleanup before download completion.
+ it('retries a pending image immediately when it completes after cleanup', async () => {
+ const mediaId = '60000000-0000-4000-8000-000000000201';
+ const image = { id: mediaId, name: 'image.png', category: 'image', path: null,
+ url: `cindy-media://blobs/${'a'.repeat(64)}.png` };
+ let complete!: (value: typeof image) => void;
+ fixture.download.mockReturnValueOnce(new Promise<typeof image>(resolve => { complete = resolve; }))
+ .mockResolvedValue({ ...image, url: `cindy-media://blobs/${'b'.repeat(64)}.png` });
+ fixture.page = [{ ...message, content: [{ type: 'media', mediaId }] }];
+ await timed(() => service.getGroup(rooms[0]));
+ fixture.cacheGeneration++;
+ complete(image);
+ const retrying = await timed(() => service.getGroup(rooms[0]));
+ expect(retrying.value.ok && retrying.value.group.messages[0].attachments[0].url).not.toBe(image.url);
+ const loaded = await timed(() => service.getGroup(rooms[0]));
+ expect(loaded.value.ok && loaded.value.group.messages[0].attachments[0].url).toContain('b'.repeat(64));
+ expect(fixture.download).toHaveBeenCalledTimes(2);
+ });
+
+ // Unit: cleanup-fenced reads must recover at cleanup completion, not wait 30s.
+ // Existing timing coverage only changes generation before a download settles.
+ it('retries an image rejected during cleanup as soon as its generation changes', async () => {
+ const mediaId = '60000000-0000-4000-8000-000000000201';
+ fixture.page = [{ ...message, content: [{ type: 'media', mediaId }] }];
+ fixture.download.mockRejectedValueOnce(new Error('REMOTE_MEDIA_CACHE_CLEARED'))
+ .mockResolvedValue({ id: mediaId, name: 'image.png', category: 'image', path: null,
+ url: `cindy-media://blobs/${'a'.repeat(64)}.png` });
+ await timed(() => service.getGroup(rooms[0]));
+ await timed(() => service.getGroup(rooms[0]));
+ expect(fixture.download).toHaveBeenCalledTimes(1);
+ fixture.cacheGeneration++;
+ await timed(() => service.getGroup(rooms[0]));
+ const loaded = await timed(() => service.getGroup(rooms[0]));
+ expect(loaded.value.ok && loaded.value.group.messages[0].attachments[0].url).toContain('a'.repeat(64));
+ expect(fixture.download).toHaveBeenCalledTimes(2);
+ });
+
it('keeps attachment order, failures and a later retry when completions share one re-read', async () => {
const ids = ['60000000-0000-4000-8000-000000000201', '60000000-0000-4000-8000-000000000202', '60000000-0000-4000-8000-000000000203'];
fixture.page = [{ ...message, content: [{ type: 'text', text: 'three files' }, ...ids.map(mediaId => ({ type: 'media', mediaId }))] }];
diff --git a/apps/desktop/src/main/maker-ipc/__tests__/chatServerMedia.test.ts b/apps/desktop/src/main/maker-ipc/__tests__/chatServerMedia.test.ts
index 1ab9ddf786..6214c55ddd 100644
--- a/apps/desktop/src/main/maker-ipc/__tests__/chatServerMedia.test.ts
+++ b/apps/desktop/src/main/maker-ipc/__tests__/chatServerMedia.test.ts
@@ -2,7 +2,11 @@ import { describe, expect, it, vi } from 'vitest';
const f = vi.hoisted(() => ({ fetch: vi.fn(), read: vi.fn(), ingest: vi.fn() }));
vi.mock('electron', () => ({ net: { fetch: f.fetch } }));
vi.mock('../../cindy-media/blobStore.js', () => ({ readFile: f.read, supportedMime: () => true }));
-vi.mock('../../cindy-media/ingest.js', () => ({ ingestMedia: f.ingest }));
+vi.mock('../../device-link/remoteMediaPersistentCache.js', () => ({
+ captureRemoteMediaCacheValidity: () => () => {},
+ remoteMediaCacheGeneration: () => 0,
+ chatMediaPersistentCache: () => ({ get: async () => null, put: f.ingest }),
+}));
vi.mock('../../localDb/client/current.js', () => ({ getDbClient: () => ({ drizzle: {} }) }));
vi.mock('../../appSessionState.js', () => ({ ownerScopedUserDataPath: () => '/unused-test-owner' }));
vi.mock('../botGroupAttachments.js', () => ({ safeAttachmentFileName: (name: string) => name }));
@@ -21,7 +25,7 @@ describe('chat private media', () => {
return {};
}) as unknown as ChatApi;
const validate = async () => { await Promise.resolve(); if (stage === deniedStage) throw new Error('REVOKED'); };
- await expect(createChatMedia(api, () => true).upload('room', 'operation', [{ ...attachment, readBytes }], 'actor', validate)).rejects.toThrow('REVOKED');
+ await expect(createChatMedia(api, () => true, 'https://chat.example.test').upload('room', 'operation', [{ ...attachment, readBytes }], 'actor', validate)).rejects.toThrow('REVOKED');
if (deniedStage === 'read') expect(readBytes).not.toHaveBeenCalled();
if (deniedStage !== 'seal') expect(f.fetch).not.toHaveBeenCalled();
expect(vi.mocked(api).mock.calls.some(([route]) => route.endsWith('/complete'))).toBe(false);
@@ -32,7 +36,7 @@ describe('chat private media', () => {
f.fetch.mockResolvedValue(new Response('', { status: 429, headers: retryAfter ? { 'Retry-After': retryAfter } : {} }));
const api = vi.fn(async () => ({ id: 'media', uploadUrl: 'https://upload.example.test/signed' })) as unknown as ChatApi;
const before = Date.now();
- const error = await createChatMedia(api, () => true).upload('room', 'reply', [attachment], 'actor').catch(error => error);
+ const error = await createChatMedia(api, () => true, 'https://chat.example.test').upload('room', 'reply', [attachment], 'actor').catch(error => error);
expect(error).toMatchObject({ message: 'MEDIA_UPLOAD_FAILED', status: 429 });
if (retryAfter) expect(error.retryAt).toBeGreaterThanOrEqual(before + 120_000);
else expect(error.retryAt).toBeUndefined();
@@ -42,7 +46,7 @@ describe('chat private media', () => {
it('does not PUT an ordinary reply after execution changes during prepare', async () => {
let current = true; f.read.mockResolvedValueOnce({ buffer: Buffer.from('png') }); f.fetch.mockClear();
const api = vi.fn(async () => { current = false; return { id: 'media', uploadUrl: 'https://upload.example.test/signed' }; }) as unknown as ChatApi;
- await expect(createChatMedia(api, () => true).upload('room', 'reply', [attachment], 'actor', () => { if (!current) throw new Error('STALE_EXECUTOR'); })).rejects.toThrow('STALE_EXECUTOR');
+ await expect(createChatMedia(api, () => true, 'https://chat.example.test').upload('room', 'reply', [attachment], 'actor', () => { if (!current) throw new Error('STALE_EXECUTOR'); })).rejects.toThrow('STALE_EXECUTOR');
expect(f.fetch).not.toHaveBeenCalled();
});
@@ -50,7 +54,7 @@ describe('chat private media', () => {
f.read.mockResolvedValue({ buffer: Buffer.from('png'), mimeType: 'image/png' });
f.fetch.mockResolvedValue(new Response('', { status: 200 }));
const api = vi.fn(async route => route.endsWith('/media') ? { id: 'media', uploadUrl: 'https://upload.example.test/signed' } : {}) as unknown as ChatApi;
- const media = createChatMedia(api, () => true);
+ const media = createChatMedia(api, () => true, 'https://chat.example.test');
expect(await media.upload('room', 'operation', [attachment], 'actor')).toEqual([{ type: 'media', mediaId: 'media', caption: 'picture.png' }]);
expect(f.fetch).toHaveBeenLastCalledWith('https://upload.example.test/signed', expect.objectContaining({ redirect: 'error', headers: { 'Content-Type': 'image/png' } }));
expect(vi.mocked(api).mock.calls.at(-1)?.[0]).toBe('/conversations/room/media/media/complete');
@@ -62,7 +66,7 @@ describe('chat private media', () => {
const file = { ...attachment, name: 'report.pdf', mimeType: 'application/pdf', url: null, path: null, readBytes,
assertCurrent: () => { if (!valid) throw new Error('STALE_EXECUTOR'); } };
const api = vi.fn(async (route: string) => route.endsWith('/media') ? { id: 'pdf', uploadUrl: 'https://upload.example.test/signed' } : {}) as unknown as ChatApi;
- const media = createChatMedia(api, () => true);
+ const media = createChatMedia(api, () => true, 'https://chat.example.test');
expect(await media.upload('room', 'operation', [file], 'actor')).toEqual([{ type: 'media', mediaId: 'pdf', caption: 'report.pdf' }]);
expect(f.fetch).toHaveBeenCalledWith(expect.any(String), expect.objectContaining({ body: new Uint8Array(Buffer.from('%PDF-1.4')) }));
f.fetch.mockClear();
@@ -80,7 +84,7 @@ describe('chat private media', () => {
throw Object.assign(new Error('REQUEST_FAILED'), { status: 500 });
return { id: 'media', uploadUrl: 'https://upload.example.test/private?token=secret' };
}) as unknown as ChatApi;
- await expect(createChatMedia(api, () => true, log).upload('room', 'operation', [attachment], 'actor')).rejects.toThrow();
+ await expect(createChatMedia(api, () => true, 'https://chat.example.test', log).upload('room', 'operation', [attachment], 'actor')).rejects.toThrow();
expect(log.warn).toHaveBeenCalledWith('Chat media upload failed', expect.objectContaining({ stage, status: stage === 'upload' ? 403 : 500 }));
expect(JSON.stringify(log.warn.mock.calls)).not.toMatch(/secret|picture.png|https:/);
});
@@ -90,7 +94,7 @@ describe('chat private media', () => {
f.fetch.mockClear();
f.read.mockImplementation(async () => { current = false; return { buffer: Buffer.from('png') }; });
const api = vi.fn() as unknown as ChatApi;
- await expect(createChatMedia(api, () => current).upload('room', 'op', [attachment], 'actor')).rejects.toThrow('OWNER_CHANGED');
+ await expect(createChatMedia(api, () => current, 'https://chat.example.test').upload('room', 'op', [attachment], 'actor')).rejects.toThrow('OWNER_CHANGED');
expect(api).not.toHaveBeenCalled();
expect(f.fetch).not.toHaveBeenCalled();
});
@@ -102,7 +106,7 @@ describe('chat private media', () => {
if (!allowed) throw new Error('CONVERSATION_NOT_FOUND');
return { name: 'p.png', type: 'image/png', size: 3, url: 'https://download.example.test/signed' };
}) as unknown as ChatApi;
- const media = createChatMedia(api, () => true);
+ const media = createChatMedia(api, () => true, 'https://chat.example.test');
expect((await media.download('room', 'media')).url).toBe('cindy-media://cached');
allowed = false;
await expect(media.download('room', 'media')).rejects.toThrow('CONVERSATION_NOT_FOUND');
@@ -111,9 +115,9 @@ describe('chat private media', () => {
it('keeps the status of a failed download, so a file that is gone is told from a failing server', async () => {
const api = vi.fn(async () => ({ name: 'p.png', type: 'image/png', size: 3, url: 'https://download.example.test/signed' })) as unknown as ChatApi;
f.fetch.mockResolvedValueOnce(new Response('', { status: 404 }));
- await expect(createChatMedia(api, () => true).download('room', 'gone')).rejects.toMatchObject({ message: 'MEDIA_DOWNLOAD_FAILED', status: 404 });
+ await expect(createChatMedia(api, () => true, 'https://chat.example.test').download('room', 'gone')).rejects.toMatchObject({ message: 'MEDIA_DOWNLOAD_FAILED', status: 404 });
f.fetch.mockResolvedValueOnce(new Response('', { status: 503 }));
- await expect(createChatMedia(api, () => true).download('room', 'later')).rejects.toMatchObject({ message: 'MEDIA_DOWNLOAD_FAILED', status: 503 });
+ await expect(createChatMedia(api, () => true, 'https://chat.example.test').download('room', 'later')).rejects.toMatchObject({ message: 'MEDIA_DOWNLOAD_FAILED', status: 503 });
});
it('uploads a send\'s files three at a time and keeps their order', async () => {
@@ -129,7 +133,7 @@ describe('chat private media', () => {
inFlight -= 1;
return { id: `media-${body.name}`, uploadUrl: 'https://upload.example.test/signed' };
}) as unknown as ChatApi;
- const blocks = await createChatMedia(api, () => true).upload('room', 'operation', files, 'actor');
+ const blocks = await createChatMedia(api, () => true, 'https://chat.example.test').upload('room', 'operation', files, 'actor');
expect(blocks.map(block => block.caption)).toEqual(['a.png', 'b.png', 'c.png', 'd.png']);
expect(most).toBe(3);
});
@@ -137,10 +141,10 @@ describe('chat private media', () => {
f.read.mockResolvedValue({ buffer: Buffer.from('png') });f.fetch.mockResolvedValue(new Response('', { status: 200 }));
let uploadUrl = 'http://127.0.0.1:3018/dev-media/fixture';
const api = vi.fn(async () => ({ id: 'media', uploadUrl })) as unknown as ChatApi;
- await expect(createChatMedia(api, () => true).upload('room','op',[attachment],'actor')).rejects.toThrow('INVALID_MEDIA_URL');
- await expect(createChatMedia(api, () => true, undefined, { allowLoopback: true }).upload('room','op',[attachment],'actor')).resolves.toHaveLength(1);
+ await expect(createChatMedia(api, () => true, 'https://chat.example.test').upload('room','op',[attachment],'actor')).rejects.toThrow('INVALID_MEDIA_URL');
+ await expect(createChatMedia(api, () => true, 'https://chat.example.test', undefined, { allowLoopback: true }).upload('room','op',[attachment],'actor')).resolves.toHaveLength(1);
uploadUrl = 'http://127.0.0.1:3019/private';
- await expect(createChatMedia(api, () => true, undefined, { allowLoopback: true }).upload('room','op',[attachment],'actor')).rejects.toThrow('INVALID_MEDIA_URL');
+ await expect(createChatMedia(api, () => true, 'https://chat.example.test', undefined, { allowLoopback: true }).upload('room','op',[attachment],'actor')).rejects.toThrow('INVALID_MEDIA_URL');
});
});
diff --git a/apps/desktop/src/main/maker-ipc/chatServer.ts b/apps/desktop/src/main/maker-ipc/chatServer.ts
index bebc4c3548..b308f9e2d3 100644
--- a/apps/desktop/src/main/maker-ipc/chatServer.ts
+++ b/apps/desktop/src/main/maker-ipc/chatServer.ts
@@ -668,7 +668,7 @@ function createChatServer(local: BotGroupChatService, deps: BotGroupChatServiceD
throw error;
});
}
- const media = createChatMedia(api, current, deps.log, { allowLoopback: !app.isPackaged && process.env.XDT_ISOLATED === '1' && config.baseUrl === 'http://127.0.0.1:3018' });
+ const media = createChatMedia(api, current, config.baseUrl, deps.log, { allowLoopback: !app.isPackaged && process.env.XDT_ISOLATED === '1' && config.baseUrl === 'http://127.0.0.1:3018' });
const threads = createThreadProjection<Message>({
threadPage: (roomId, rootId, limit, before) => history()?.page<Message>(roomId,
new URLSearchParams({ threadRootId: rootId, limit: String(limit), ...(before ? { before } : {}) }))
@@ -1337,7 +1337,7 @@ function createChatServer(local: BotGroupChatService, deps: BotGroupChatServiceD
// request never rejects a room/Thread; subsequent reads retry after a backoff.
// Until the bytes are here the view keeps a named placeholder in its place, so
// a slow or failed download is shown as such instead of disappearing.
- const downloads = new Map<string, { value?: BotGroupAttachment; retryAt: number }>();
+ const downloads = new Map<string, { value?: BotGroupAttachment; retryAt: number; generation: number; imageCacheCleared?: boolean }>();
// A page's attachments settle close together; one re-read of the room and sidebar shows
// them all instead of one full re-read per file.
const attachmentRefreshes = new Map<string, ReturnType<typeof setTimeout>>();
@@ -1359,13 +1359,21 @@ function createChatServer(local: BotGroupChatService, deps: BotGroupChatServiceD
const mediaId = parsed.data;
const key = `${roomId}:${mediaId}`;
let cached = downloads.get(key);
+ // Image cleanup does not own ordinary files, including still-pending transfers.
+ if (cached && cached.generation !== media.cacheGeneration
+ && (cached.value?.category === 'image' || cached.imageCacheCleared)) cached = undefined;
if (!cached || (!cached.value && Date.now() >= cached.retryAt)) {
- cached = { retryAt: Infinity };
+ cached = { retryAt: Infinity, generation: media.cacheGeneration };
downloads.set(key, cached);
const entry = cached;
void media.download(roomId, mediaId).then(value => {
- if (current()) entry.value = value;
- }).catch(() => { entry.retryAt = Date.now() + 30000; })
+ if (!current()) return;
+ if (value.category === 'image' && entry.generation !== media.cacheGeneration) entry.retryAt = 0;
+ else entry.value = value;
+ }).catch(error => {
+ entry.imageCacheCleared = error instanceof Error && error.message === 'REMOTE_MEDIA_CACHE_CLEARED';
+ entry.retryAt = entry.generation !== media.cacheGeneration ? 0 : Date.now() + 30000;
+ })
.finally(() => attachmentsChanged(roomId));
}
view.attachments.push(cached.value ?? placeholder(cached.retryAt === Infinity ? 'loading' : 'unavailable'));
diff --git a/apps/desktop/src/main/maker-ipc/chatServerMedia.ts b/apps/desktop/src/main/maker-ipc/chatServerMedia.ts
index 0d50848053..ff233ef578 100644
--- a/apps/desktop/src/main/maker-ipc/chatServerMedia.ts
+++ b/apps/desktop/src/main/maker-ipc/chatServerMedia.ts
@@ -5,8 +5,7 @@ import fs from 'node:fs/promises';
import path from 'node:path';
import { createHash, randomUUID } from 'node:crypto';
import { readFile as readMedia, supportedMime } from '../cindy-media/blobStore.js';
-import { ingestMedia } from '../cindy-media/ingest.js';
-import { getDbClient } from '../localDb/client/current.js';
+import { captureRemoteMediaCacheValidity, chatMediaPersistentCache, remoteMediaCacheGeneration } from '../device-link/remoteMediaPersistentCache.js';
import { ownerScopedUserDataPath } from '../appSessionState.js';
import { botGroupAttachmentsPath } from './botGroupWorkDir.js';
import { safeAttachmentFileName } from './botGroupAttachments.js';
@@ -28,7 +27,8 @@ export type ChatUploadAttachment = BotGroupAttachment & {
const MAX_SIZE = 100 * 1024 * 1024;
/** Files of one send uploaded at once; each holds its bytes in memory until sealed. */
const UPLOAD_CONCURRENCY = 3;
-export function createChatMedia(api: ChatApi, current: () => boolean, log?: { warn(message: string, meta?: Record<string, unknown>): void }, options: { allowLoopback?: boolean } = {}) {
+export function createChatMedia(api: ChatApi, current: () => boolean, endpoint: string, log?: { warn(message: string, meta?: Record<string, unknown>): void }, options: { allowLoopback?: boolean } = {}) {
+ // Images keep only in-flight transfers here; ordinary files retain service-local reuse.
const cache = new Map<string, Promise<BotGroupAttachment>>();
const check = () => { if (!current()) throw new Error('OWNER_CHANGED'); };
const signedUrl = (value: string) => {
@@ -37,12 +37,14 @@ export function createChatMedia(api: ChatApi, current: () => boolean, log?: { wa
return url.href;
};
return {
+ get cacheGeneration() { return remoteMediaCacheGeneration(); },
/**
* Uploads a send's files a few at a time (prepare → PUT → complete each), keeping the
* blocks in the files' order. Every file settles before a failure is reported, so no
* upload keeps running after the send has given up; the first failure in order wins.
*/
async upload(roomId: string, source: string, attachments: ChatUploadAttachment[], actorId: string, assertCurrent?: () => void | Promise<void>): Promise<ChatContentBlock[]> {
+ const assertCacheCurrent = captureRemoteMediaCacheValidity();
const checkUpload = async () => { check(); await assertCurrent?.(); check(); };
const results: Array<PromiseSettledResult<ChatContentBlock>> = new Array(attachments.length);
let next = 0;
@@ -97,6 +99,17 @@ export function createChatMedia(api: ChatApi, current: () => boolean, log?: { wa
await checkFile();
stage = 'complete';
await api(`/conversations/${roomId}/media/${upload.id}/complete`, 'POST', { operationId: `seal:${upload.id}` }, actorId);
+ check(); attachment.assertCurrent?.();
+ if (attachment.mimeType.startsWith('image/') && supportedMime(attachment.mimeType)) {
+ // Keep the bytes actually uploaded, not a later re-read of a mutable source.
+ // Cache failure must not turn an already sealed upload into a failed send.
+ try {
+ assertCacheCurrent();
+ await chatMediaPersistentCache(endpoint, roomId, upload.id, () => { check(); assertCacheCurrent(); })
+ ?.put(bytes, attachment.mimeType);
+ } catch { log?.warn('Chat image cache write failed', { stage: 'retain' }); }
+ check(); attachment.assertCurrent?.();
+ }
return { type: 'media', mediaId: upload.id, caption: attachment.name };
} catch (error) {
log?.warn('Chat media upload failed', { stage, groupId: roomId,
@@ -107,15 +120,27 @@ export function createChatMedia(api: ChatApi, current: () => boolean, log?: { wa
},
async download(roomId: string, mediaId: string): Promise<BotGroupAttachment> {
check();
+ const generation = remoteMediaCacheGeneration();
+ const assertCacheCurrent = captureRemoteMediaCacheValidity();
// Authorization is checked for each projection, even when bytes are cached.
const media = await api<{ name: string; type: string; size: string | number; url: string }>(`/conversations/${roomId}/media/${mediaId}`);
check();
- const key = `${roomId}:${mediaId}`;
+ const isImage = media.type.startsWith('image/') && supportedMime(media.type);
+ const checkDownload = () => { check(); if (isImage) assertCacheCurrent(); };
+ checkDownload();
+ const persistent = isImage ? chatMediaPersistentCache(endpoint, roomId, mediaId, checkDownload) : null;
+ const key = JSON.stringify([roomId, mediaId, isImage ? generation : null]);
if (!cache.has(key)) cache.set(key, (async () => {
const size = Number(media.size);
if (!Number.isSafeInteger(size) || size <= 0 || size > MAX_SIZE) throw new Error('INVALID_MEDIA_SIZE');
const root = ownerScopedUserDataPath();
- const db = getDbClient().drizzle;
+ const base = { id: mediaId, name: media.name, mimeType: media.type, size };
+ if (isImage) {
+ const hit = await persistent?.get(size);
+ checkDownload();
+ if (hit && hit.bytes.length === size)
+ return { ...base, mimeType: hit.mimeType, category: 'image' as const, url: hit.url, path: null };
+ }
const response = await net.fetch(signedUrl(media.url), { redirect: 'error', signal: AbortSignal.timeout(60000) });
// The status tells a file that is gone (4xx: never comes) from a failing server (retried).
if (!response.ok || !response.body) throw Object.assign(new Error('MEDIA_DOWNLOAD_FAILED'), response.ok ? {} : { status: response.status });
@@ -123,7 +148,7 @@ export function createChatMedia(api: ChatApi, current: () => boolean, log?: { wa
const reader = response.body.getReader();
try {
for (;;) {
- const next = await reader.read(); check();
+ const next = await reader.read(); checkDownload();
if (next.done) break;
received += next.value.byteLength;
if (received > size) throw new Error('INVALID_MEDIA_SIZE');
@@ -132,11 +157,12 @@ export function createChatMedia(api: ChatApi, current: () => boolean, log?: { wa
} finally { await reader.cancel(); }
if (received !== size) throw new Error('INVALID_MEDIA_SIZE');
const buffer = Buffer.concat(chunks);
- const base = { id: mediaId, name: media.name, mimeType: media.type, size };
- check();
- if (media.type.startsWith('image/') && supportedMime(media.type)) {
- const stored = await ingestMedia({ buffer, mimeType: media.type, isCache: true, refs: [], assertStillValid: check }, db);
- return { ...base, category: 'image' as const, url: stored.url, path: null };
+ checkDownload();
+ if (isImage) {
+ const stored = await persistent?.put(buffer, media.type);
+ checkDownload();
+ if (!stored) throw new Error('INVALID_MEDIA_IMAGE');
+ return { ...base, mimeType: stored.mimeType, category: 'image' as const, url: stored.url, path: null };
}
// Non-media files use the existing account/group attachment directory.
const dir = path.join(botGroupAttachmentsPath(root, roomId), mediaId);
@@ -144,8 +170,13 @@ export function createChatMedia(api: ChatApi, current: () => boolean, log?: { wa
const file = path.join(dir, safeAttachmentFileName(media.name));
await fs.writeFile(file, buffer); check();
return { ...base, category: 'file' as const, url: null, path: file };
- })().catch(error => { cache.delete(key); throw error; }));
- return cache.get(key)!;
+ })().then(value => {
+ if (isImage) cache.delete(key);
+ return value;
+ }, error => { cache.delete(key); throw error; }));
+ const result = await cache.get(key)!;
+ checkDownload();
+ return result;
},
};
}
diff --git a/docs/dev-rules/media-storage-and-protocols.md b/docs/dev-rules/media-storage-and-protocols.md
index 5860701f0a..a6d84c8312 100644
--- a/docs/dev-rules/media-storage-and-protocols.md
+++ b/docs/dev-rules/media-storage-and-protocols.md
@@ -85,7 +85,12 @@
- 桌面聊天的不可变远程图片通过 `device-link/remoteMediaPersistentCache.ts` 复用媒体总仓,
`remote-media` 索引包含账号、来源设备和原始地址。先查内存、再查磁盘,缓存命中不取件;
重启和断连不清理,不增加自动到期或容量淘汰。设置中的「远程图片缓存」复用账本扫描和
- 条件删除,只清该命名空间独占的缓存,保护草稿、已保存附件及其他集成共享字节。
+ 条件删除:独占缓存可删字节,共享缓存只解除 `remote-media` 索引,保留其他引用及字节;
+ 预检对共享项计 0 可释放字节,清理后其余缓存可继续按原规则回收。草稿与已 pin 附件仍跳过。
+ 清理开始与结束(含失败)都使图片缓存代次失效,防止清理期间启动的读取保留旧投影,
+ 或在清理返回后重建刚删除的索引;清理全程拒绝图片缓存读写,重叠清理全部结束才恢复,
+ 避免较早删掉的图片在遍历后续项时重新入仓。因清理失效的图片读取在下一代次立即重试,
+ 不沿用普通下载失败的 30 秒退避;普通附件不受图片代次影响。
普通路径不按 URL 永久缓存:继续由主机复核版本,完整中转下载也可写入已有版本缓存;
控制端发送图片时,同一缓存还保留实际上传的字节(含压缩结果),按其最终内容地址登记,
覆盖直传与 OSS、文件选择与内存图片。不能用压缩前的本机原图代替历史消息中的压缩版本。
@@ -94,6 +99,19 @@
写入新账号或重建用户刚清理的缓存。复用现有 `remote-media` 清理入口,不增设目录或 TTL。
音视频仍走流式播放。旧主机不返回版本时不把可修改文件登记为长期有效。
回归见 `remoteMediaPersistentCache.test.ts`、`remoteMediaProtocol.test.ts` 与 `storageIpc.test.ts`。
+- Chat Server 群聊、话题和伙伴私聊的图片也使用上述 `remote-media` 长期副本与清理入口,
+ 索引按账号模式、账号 ID、服务器端点、房间 ID、media ID 隔离;签名 URL 会过期,不能当作
+ 持久身份。`chatServerMedia.ts` 每次取件先按原合同校验媒体访问权限,再核对本地大小及内容
+ 指纹;命中只读本地,缺失或损坏才下载并修复。重启不丢索引,发送成功时留存实际上传的
+ 图片字节;留存失败不把已完成上传变成发送失败。下载及发送留存均捕获账号和清理代次,
+ 迟到结果不得重建已清理索引,群聊内存中的图片投影也随清理代次失效;普通文件的成功
+ 结果和在途下载不受图片清理影响,仍保留服务内复用,避免话题每轮重新下载 PDF 等附件。
+ 通用缓存瘦身在候选扫描与条件删除两处排除任何带 `remote-media` 引用的字节,专用清理仍
+ 保护草稿、pin 和其他业务引用,共享缓存只解除本命名空间索引。旧版下载只有无引用 blob、
+ 没有 media ID 索引,首次重新访问可能还需下载一次以建立映射;不猜测文件对应关系,
+ 也不批量清理旧文件。
+ 复用现有表及协议,不新增 migration;回归见 `remoteMediaPersistentCache.test.ts` 的
+ Chat Server 重启、发送留存、隔离、损坏恢复及清理用例,以及 `chatServerGroupOpen.test.ts`。
- 零引用入仓只适用于已有草稿或生成结果提交链路。新增零引用窗口时,必须证明内容在消息
落库前不会被回收,并用测试覆盖成功、失败、取消和重试。
- 所有官方壁纸原图均为 OSS/CDN 按需分发(2026-10-17 用户裁决,将三款场景的已有下载链路
diff --git a/docs/product-rules/bot-group-chat.md b/docs/product-rules/bot-group-chat.md
index 9bfef142c2..6624365f48 100644
--- a/docs/product-rules/bot-group-chat.md
+++ b/docs/product-rules/bot-group-chat.md
@@ -911,6 +911,8 @@ Desktop `chatServer.thread → threadDetail` 复用同一房间的 snapshot/
Desktop 关闭后重开话题,先复用同账号运行代次、同房间 `historyRevision` 的窗口,再后台同步;窗口缓存最多 20 个话题、每个 500 条回复,持久记录仍由上述投影管理。刷新用 `messageUpdates` 修正已展开旧回复;本机表情操作优先复用该修正,未返回目标消息时在同一合并刷新内顺序补读其旧页,不能让最新页刷新把表情回包误判为过期。旧页晚于最新刷新返回时,只补入新加载回复的失败状态,保留已显示源消息的较新状态,并把新增源 ID 带入合并后的补读以核对当前状态。历史代次变化、撤权或主群重置时丢弃旧窗口及在途分页,不能混入旧正文;根消息返回 `MESSAGE_NOT_FOUND` 只清除当前话题窗口,保留群持久投影和其他话题,不能与房间级 `CONVERSATION_NOT_FOUND` 或成员撤权混同。首次打开或缺页仍请求服务器,重启/重连先补增量;执行失败状态和必要的已读写入继续请求服务器,因此缓存命中不等于完全零 HTTP。
+Desktop 群聊、话题与伙伴私聊的服务器图片共用持久附件索引:账号+服务器+房间+media ID 对应媒体总仓中的内容指纹,签名下载 URL 不作为缓存身份。媒体接口仍用于确认访问权限,已下载且校验有效的图片直接复用本地文件,重启不重下;发送完成时同时留存上传字节。图片归现有「远程图片缓存」独立管理,不按普通缓存的容量或年龄淘汰,通用缓存瘦身也不删除;用户通过专门入口清理后,下一次查看才重新下载。缺失或损坏的文件按需修复,账号切换及清理会阻止旧下载重新挂索引。旧版没有持久映射的图片首次访问可能还需下载一次。实现与清理边界见 [媒体存储规则](../dev-rules/media-storage-and-protocols.md) 和 `chatServerMedia.ts`;Mobile 直连仍使用服务器签名 URL,不把本条 Desktop 落盘能力当成手机已实现。
+
主干已为 Desktop 增加持久 outbox 与发送 journal,为 Mobile 接入 durable outbox;保存的展示窗口与增量投影分工见 [本地接入合同](../dev-rules/chat-server-local-test.md)。本轮手机增加独立的持久消息投影,复用原列表缓存、读位和 endpoint 规则;直连服务器和经电脑 Remote Resource 仍是同一房间的两种访问路径。
伙伴私聊以 canonical 主 Session 为本机界面入口;在线远端通过 device-link,离线远端读取服务器房间。主力电脑镜像用户可见消息及最终回复,按 `sourceId` 导入去重;重复 sourceId 返回既有记录,不会自动把旧记录更新为新内容。Desktop 云端转录首次可能多页回读后按消息时间排序,之后从持久覆盖范围读取并按 cursor 修正。手机私聊保留按需翻页,刷新时修正已展示旧页。显式修改的反向镜像仍待协议设计。