⭐️⭐️⭐️ Open-source, self-hosted alternative to Linear and Plane. Project management and issue tracking where teams and AI agents work side by side to plan and ship products.
TypeScript agenticagileai-agentsbug-reportingbug-trackerganttissue-trackerjirajira-alternativekanbanlinearlinear-alternativeopen-sourceproduct-managementproject-managementroadmapself-hostedsprint-planningteam-collaborationtrello-alternative
fix(worker): reject a failed attachment download (#471)
Thomas Gallaway4fa48da0f5d4
diff --git a/apps/worker/AGENTS.md b/apps/worker/AGENTS.mdindex a889c50c..84ecb2b6 100644--- a/apps/worker/AGENTS.md+++ b/apps/worker/AGENTS.md@@ -65,8 +65,10 @@ running to completion in one.`putObject`/`attachmentObjectKey`) is checked before the upload and oncemore inside the same advisory lock (`lockAttachmentStorage`) an interactiveupload takes, so the two never both pass a check that only one of them can- actually fit. A rejected attachment (`AttachmentRejectedError`) is logged- and skipped, not treated as a failed tick.+ actually fit. A download that answers anything but 2xx (an error page, a+ redirect) is refused before its body is read (`attachment-download.ts`). A+ rejected attachment (`AttachmentRejectedError`) is logged and skipped, not+ treated as a failed tick.## Invariants@@ -90,7 +92,8 @@ running to completion in one.- **Pure logic stays dependency-free.** `backoff.ts`, `signature.ts`, and`isRetryableStatus` import nothing from `@repo/db`, so unit tests run without adatabase. Keep DB access in `store.ts`. Same split for imports: `canonical.ts`,- `reader.ts`, `plane-adapter.ts`, and `cross-reference.ts` import nothing from+ `reader.ts`, `plane-adapter.ts`, `cross-reference.ts`, and+ `attachment-download.ts` import nothing from`@repo/db` (its state-category normalization, markdown conversion,cursor/rate-limit, and cross-reference matching logic are unit-testeddirectly); `@repo/db` access stays in `import-store.ts`.diff --git a/apps/worker/src/__tests__/unit/attachment-download.test.ts b/apps/worker/src/__tests__/unit/attachment-download.test.tsnew file mode 100644index 00000000..3d6e38d1--- /dev/null+++ b/apps/worker/src/__tests__/unit/attachment-download.test.ts@@ -0,0 +1,53 @@+import { describe, it, expect } from 'bun:test';+import type { PinnedRequestInit } from '@repo/net';+import {+ downloadAttachment,+ AttachmentRejectedError,+ type AttachmentFetch,+} from '../../attachment-download';++const FILE_URL = 'https://files.example.test/report.pdf';++function respondWith(response: Response): AttachmentFetch {+ return async () => response;+}++describe('downloadAttachment', () => {+ it('returns the body of a 2xx response', async () => {+ const bytes = await downloadAttachment(FILE_URL, 1024, respondWith(new Response('file bytes')));+ expect(bytes.toString()).toBe('file bytes');+ });++ it.each([401, 403, 404, 500, 503])(+ 'rejects an HTTP %d response instead of returning its body as the file',+ async (status) => {+ const error = await downloadAttachment(+ FILE_URL,+ 1024,+ respondWith(new Response('<Error>AccessDenied</Error>', { status })),+ ).catch((e: unknown) => e);+ expect(error).toBeInstanceOf(AttachmentRejectedError);+ expect((error as Error).message).toContain(`HTTP ${status}`);+ },+ );++ it('rejects a redirect, which pinnedFetch returns without following', async () => {+ const redirect = new Response(null, {+ status: 302,+ headers: { location: 'https://elsewhere.example.test/report.pdf' },+ });+ await expect(downloadAttachment(FILE_URL, 1024, respondWith(redirect))).rejects.toBeInstanceOf(+ AttachmentRejectedError,+ );+ });++ it('downloads with the byte limit and a timeout', async () => {+ let seen: { url: string; init: PinnedRequestInit | undefined } | null = null;+ const fetch: AttachmentFetch = async (url, init) => {+ seen = { url, init };+ return new Response('x');+ };+ await downloadAttachment(FILE_URL, 2048, fetch);+ expect(seen!).toEqual({ url: FILE_URL, init: { timeoutMs: 30_000, maxBytes: 2048 } });+ });+});diff --git a/apps/worker/src/attachment-download.ts b/apps/worker/src/attachment-download.tsnew file mode 100644index 00000000..da1659f9--- /dev/null+++ b/apps/worker/src/attachment-download.ts@@ -0,0 +1,24 @@+import { pinnedFetch } from '@repo/net';++// An attachment that cannot be stored: over this instance's size, type or quota+// limits, or a download that did not return the file. The Attachments phase logs+// and skips it instead of failing the tick.+export class AttachmentRejectedError extends Error {}++export type AttachmentFetch = typeof pinnedFetch;++const DOWNLOAD_TIMEOUT_MS = 30_000;++// pinnedFetch returns any status as-is, a redirect included, so an error page+// must be refused here or it would be stored as the file.+export async function downloadAttachment(+ url: string,+ maxBytes: number,+ fetch: AttachmentFetch = pinnedFetch,+): Promise<Buffer> {+ const res = await fetch(url, { timeoutMs: DOWNLOAD_TIMEOUT_MS, maxBytes });+ if (!res.ok) {+ throw new AttachmentRejectedError(`the download answered HTTP ${res.status}`);+ }+ return Buffer.from(await res.arrayBuffer());+}diff --git a/apps/worker/src/import-store.ts b/apps/worker/src/import-store.tsindex f7a26ae0..e3a86eec 100644--- a/apps/worker/src/import-store.ts+++ b/apps/worker/src/import-store.ts@@ -30,6 +30,9 @@ import {} from '@repo/storage';import type { CanonicalState, CanonicalLabel, CanonicalCycle } from './canonical';import type { PlaneCredential } from './plane-adapter';+import { AttachmentRejectedError } from './attachment-download';++export { AttachmentRejectedError };// All @repo/db access for the Plane import: claiming due import_job rows (the// same FOR UPDATE SKIP LOCKED + lease pattern as store.ts uses for@@ -699,12 +702,6 @@ export async function findProjectMemberUserId(return rows[0]?.userId ?? null;}-// Thrown for an attachment that simply does not fit this instance's own-// configured limits (file size, mime type, or project quota) — the Attachments-// phase catches this specifically and moves on to the next attachment, rather-// than treating it as a failed tick the way a network or database error is.-export class AttachmentRejectedError extends Error {}-export interface NewLocalAttachment {projectId: number;issueId: number;diff --git a/apps/worker/src/import-worker.ts b/apps/worker/src/import-worker.tsindex 1b97db24..900e0dd6 100644--- a/apps/worker/src/import-worker.ts+++ b/apps/worker/src/import-worker.ts@@ -1,4 +1,3 @@-import { pinnedFetch } from '@repo/net';import { getStorageSettings, mimeAllowed, MB } from '@repo/db';import { startPollLoop, type WorkerHandle } from './poll-loop';import { intEnv } from './env';@@ -7,6 +6,7 @@ import { PlaneReader, PlaneRateLimitedError } from './plane-adapter';import type { SourceReader } from './reader';import type { CanonicalComment, CanonicalStateCategory } from './canonical';import { extractCrossReferences, applyCrossReferenceReplacements } from './cross-reference';+import { downloadAttachment } from './attachment-download';import {claimDueImportJobs,decryptImportCredential,@@ -480,11 +480,7 @@ async function runAttachments(job: ClaimedImportJob, reader: SourceReader): Prom);}const url = await reader.resolveAttachmentDownloadUrl(record.sourceId, attachment.sourceId);- const res = await pinnedFetch(url, {- timeoutMs: 30_000,- maxBytes: limits.maxAttachmentMb * MB,- });- const bytes = Buffer.from(await res.arrayBuffer());+ const bytes = await downloadAttachment(url, limits.maxAttachmentMb * MB);await createLocalAttachmentAndRecord(job.id, attachment.sourceId, {projectId: job.projectId,issueId: record.localId,diff --git a/docs/dev/plane-import.md b/docs/dev/plane-import.mdindex 7a1678be..2e8a8f35 100644--- a/docs/dev/plane-import.md+++ b/docs/dev/plane-import.md@@ -92,9 +92,12 @@ and mime type are checked against the instance's own upload settings (`getStoragchecked once before the upload (so an already-over-quota file is never written to theobject store at all) and once more inside the same advisory lock(`lockAttachmentStorage`) an interactive upload takes, immediately before the insert — the-two never both pass a check the project can only actually fit one of. A rejected attachment-(`AttachmentRejectedError`, `import-store.ts`) is logged and skipped, not treated as a-failed tick that retries. Re-running an import reuses an existing attachment by exact+two never both pass a check the project can only actually fit one of. A download that+answers anything but 2xx is refused before its body is read (`downloadAttachment`,+`attachment-download.ts`): `pinnedFetch` returns error statuses and redirects as-is, and+their body is not the file. A rejected attachment (`AttachmentRejectedError`) is logged and+skipped, not treated as a failed tick that retries. Re-running an import reuses an existing+attachment by exactfilename on the same issue, the same reuse-by-content-match philosophy every other entityhere already has.diff --git a/docs/importing-from-plane.md b/docs/importing-from-plane.mdindex 72f519a7..03e688cd 100644--- a/docs/importing-from-plane.md+++ b/docs/importing-from-plane.md@@ -45,7 +45,8 @@ of them affect data you may not notice is missing until later.- **Attachment files**, downloaded and attached to the same issue they were on in Plane.A file is skipped, not imported, when it is larger than this instance's own uploadlimit or of a file type this instance doesn't accept — the same limits an ordinary- attachment upload on this instance is held to. A skipped file is only noted in the+ attachment upload on this instance is held to — or when Plane's file storage answers+ the download with an error instead of the file. A skipped file is only noted in theserver log, not shown anywhere in this screen today. Re-running an import reuses afile already attached with the same name on the same issue, rather than attaching ita second time.