Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
feat: export portable FableLoom packages (#13107)
Adam Eivy7983aadaa6d0
diff --git a/docs/README.md b/docs/README.mdindex 30290d6a22..77bffc9ef3 100644--- a/docs/README.md+++ b/docs/README.md@@ -82,7 +82,7 @@ Identity & self: [digital-twin](./features/digital-twin.md) · [identity-system]Knowledge: [brain-system](./features/brain-system.md) · [untrusted messages and GitHub automation](./features/messages-security.md) · [scope adherence](./features/scope-adherence.md)-Create: [writers-room](./features/writers-room.md) · [creative-director-video](./features/creative-director-video.md) · [apple-video-options](./features/apple-video-options.md) · [qwen-image-21](./features/qwen-image-21.md) · [fableloom](./features/fableloom.md) · [FableLoom runtime v1](./features/fableloom-runtime-v1.md) · [Eidoverse Worlds integration](./features/eidoverse.md) · [Eidoverse music-video renderer](./EIDOVERSE_VIDEO.md) · [OpenWorld historical reference](./features/openworld.md) · [sprite-export-contract](./features/sprite-export-contract.md) · [video-text-encoders](./features/video-text-encoders.md) · [video-speed-profiles](./features/video-speed-profiles.md) · [video-render-batches](./features/video-render-batches.md) · [video-upscale](./features/video-upscale.md)+Create: [writers-room](./features/writers-room.md) · [creative-director-video](./features/creative-director-video.md) · [apple-video-options](./features/apple-video-options.md) · [qwen-image-21](./features/qwen-image-21.md) · [fableloom](./features/fableloom.md) · [FableLoom runtime v1](./features/fableloom-runtime-v1.md) · [FableLoom export v1](./features/fableloom-export-v1.md) · [Eidoverse Worlds integration](./features/eidoverse.md) · [Eidoverse music-video renderer](./EIDOVERSE_VIDEO.md) · [OpenWorld historical reference](./features/openworld.md) · [sprite-export-contract](./features/sprite-export-contract.md) · [video-text-encoders](./features/video-text-encoders.md) · [video-speed-profiles](./features/video-speed-profiles.md) · [video-render-batches](./features/video-render-batches.md) · [video-upscale](./features/video-upscale.md)Comms & voice: [beeper](./features/beeper.md) · [messages-browser-send](./features/messages-browser-send.md) · [openclaw-operator-chat](./features/openclaw-operator-chat.md) ([pre-build audit](./research/2026-03-31-openclaw-operator-chat-audit.md)) · [stacker-news](./features/stacker-news.md) · [voice](./features/voice.md) ([Jev feasibility](./research/2026-09-30-jev-voice-feasibility.md))diff --git a/docs/STORAGE.md b/docs/STORAGE.mdindex 78138c951b..501867035b 100644--- a/docs/STORAGE.md+++ b/docs/STORAGE.md@@ -330,6 +330,16 @@ goal data together; interrupted intents remain retryable after restore.## `ephemeral-file` — queues / uploads / transient state+- FableLoom portable exports — `data/cache/fableloom-exports/<jobId>/` holds+ atomic job status, temporary media snapshots and the derived ZIP. The authored+ loom remains db-primary; these jobs are regenerable, keyed downloads with no+ search, sync cursor or relational authority. Jobs expire after 24 hours, retain+ at most four artifacts (256 MiB per ZIP, 64 MiB per source/prepared asset), and+ prepare one package at a time. A restart makes interrupted jobs visibly failed;+ it never resumes generation or publishes partial work. They never federate,+ need no seed/migration, and the existing anchored `/cache/` backup exclusion+ applies. See [FableLoom export v1](./features/fableloom-export-v1.md).+**Definition.** Regenerable, short-lived runtime state. Losing it costs at most an in-flight job or a cache rebuild — never durable user data. It should never be the only home for anything the user expects to persist.**When to use.** Upload staging, in-flight job queues, caches, and scratch state. If a record must survive a reinstall or be queryable across records, it is **not** `ephemeral-file` — promote it.diff --git a/docs/features/fableloom-export-v1.md b/docs/features/fableloom-export-v1.mdnew file mode 100644index 0000000000..76e7375877--- /dev/null+++ b/docs/features/fableloom-export-v1.md@@ -0,0 +1,87 @@+# FableLoom portable export v1++FableLoom exports authored stories and existing rendered media for an author's+own external host. PortOS remains private. Exports never upload, deploy, expose+PortOS, fetch a media URL, call a provider, or generate new creative media.++## API++An operator starts a job with `POST /api/fableloom/:id/export`:++```json+{"episodeIds":["example-episode"],"allowDraft":false,"mediaProfile":"source"}+```++Omit `episodeIds` for all episodes. Selections are unique and retain authored+episode order. Unknown options, wrong types and unknown episode IDs are refused.+The response is `202 {"jobId":"<uuid>"}`. Read+`GET /api/fableloom/:id/export/:jobId` for status, progress, validation findings,+expiry and a size report. A completed job supplies a relative `downloadUrl` for+`GET /api/fableloom/:id/export/:jobId/download`; downloads require operator+authority, including valid delegated agent sessions. Jobs cannot be read or+downloaded through a different loom. Instance authentication retains its existing+optional policy; no export endpoint is always-public.++This core version explicitly refuses `includePlayer:true`,+`includeCharacters:true`, `includeVoice:true` and media profiles other than+`"source"`. Their implementation belongs to the linked export slices.++## Gates and approvals++The job snapshots the authored record, selected episodes, video history and+linked universe. It reuses graph, outline and production validation. All+reachable scenes need a still and an entry clip. Decision scenes also need+holds; legacy entry/video assets may supply their hold. Attached exit clips+must exist; an absent exit means a direct cut under runtime v1.++A still needs its existing storyboard approval. Video needs the loom's current+delivery approval, which record mutations invalidate. Each asset also needs+locked visual-conditioning provenance. Missing provenance, unapproved, draft+and degraded assets fail by default. `allowDraft:true` permits those approval+conditions and records explicit `draftFlags` per media use. It does not relax+graph, outline, production, missing media, encoding, occupancy, path or size+failures. Enabled live holds require verified speech-safe occupancy; intervals+cannot exceed the actual clip duration.++Media reads accept basenames only, resolve inside the appropriate gallery/video+root, refuse escaping symlinks and snapshot bounded regular-file bytes. No+history ID is reconstructed as a filename. Input changes during a read fail+the job. Preparation decodes stills to metadata-free PNG and remuxes H.264+yuv420p / optional AAC MP4 without re-encoding picture/audio samples. Other codecs,+animated stills, resizing, bitrate profiles and loudness normalization are not+supported yet. Embedded container metadata, H.264 user-data SEI, chapters and extra tracks are+removed; authored source files remain unchanged.++## Package schema 1.0.0++- `manifest.json`: export schema, playback runtime and PortOS versions; explicit+ authored metadata, ordered episodes, reachable graphs and playback roles.+ Media uses only content hashes and relative `media/<sha256>.<ext>` references,+ with `durationMs`, `loopable` and projected audio occupancy.+- `provenance.json`: per-use content hash, provider/model labels, parameter hash,+ approval and conditioning status, and draft flags.+- `media/`: deduplicated, metadata-free media.+- `runtime-v1.md` and `README.md`: the versioned playback and hosting contracts.++All JSON object keys and ZIP member order are stable. ZIP timestamps are fixed.+The same source bytes and authored snapshot, with the same PortOS/media-tool+versions, produce identical manifest, media hashes and ZIP bytes. There are no+job IDs, timestamps, instance/peer IDs, local paths, credentials, provider+configuration, adapters, LoRAs, source prompts, sessions or transcripts in the+package. Only authored scene/choice text and the documented projections leave+the authoring record.++## Lifecycle and failures++Jobs and ZIPs are regenerable `ephemeral-file` cache entries. They expire after+24 hours; starting another export evicts expired entries and then the oldest+terminal entries to keep at most four jobs. Four queued/running jobs return+`EXPORT_QUEUE_FULL`. Each asset is capped at 64 MiB and each ZIP at 256 MiB.+Only one job prepares media at a time. Graceful maintenance tracks it through+final status persistence and scratch cleanup.++A failed job exposes an actionable code/message and its preflight findings,+never a raw filesystem/encoder error or source path. It has no download URL.+An interrupted job is failed on the first read after restart; it is not replayed.+An expired job returns `410 EXPORT_EXPIRED`; an unavailable/failed/not-yet-ready+artifact cannot be downloaded. Exporting again is the recovery path.diff --git a/server/lib/README.md b/server/lib/README.mdindex a58c31cf1c..917faa879b 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -61,6 +61,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `renderModeLadder.js` | `resolveRenderPins` and `resolveRenderTargetPins` share backend-scoped model candidate inheritance across browser and server. Image-gen mode resolution ladder (#6815), a dependency-free leaf beside `generationModes.js`/`renderTargets.js` so the client imports it directly instead of hand-copying it: `isModeUsable(settings, mode)` (queueable + enabled gate, no per-provider spec lookup), `pickUsableMode(settings, candidates)` (first usable candidate, falling through disabled pins to the cloud-then-local tail), `renderTargetDefaults(settings, target)` (the normalized `settings.renderDefaults[target]` pin), and `imageModeCandidates(settings, target, record)` (the shared `[recordPin, targetPin, installDefault]` candidate order, declared once so a UI's own "what will Auto resolve to" display can't drift from the resolver). `services/imageGen/cloudProviderConfig.js` re-exports the first three for its existing callers. || `fableLoomBoard.js` | FableLoom Board view: `fableLoomBoardPath` walks one path from an episode's start scene (first choice at each fork unless a picked transition in `via`) into storyboard beats of panels with each fork's choices, stopping at an ending, a loop or a dead end; `fableLoomBoardPick` swaps a fork's pick. Browser-safe. || `fableLoomLimits.js` | FableLoom record caps (`LOOM_LIMITS`) — the single source of truth shared by the route schemas and the record sanitizer. A leaf module, below both. |+| `fableLoomExport.js` | Portable package schema/limits, graph/outline/production preflight, required media roles and explicit path-free playback/occupancy projections. || `cameraMovements.js` | Shared camera-movement vocabulary (#10589): `CAMERA_MOVEMENTS` (stable ids with `family`, `energy`, `snap`/`still` flags, start/end framing and the four-part movement/speed/framing/end prose), `normalizeCameraMovement`/`cameraMovementId`/`cameraMovementFromText`, `shotCameraPrompt(camera)` (the four-part i2v camera block for a structured shot camera) and `cameraMovementCatalogForPrompt`. Used by FableLoom, Music Video planning/review and the layered template's `cameraRig.js`. || `fableLoomCameraMovements.js` | FableLoom names for the shared `cameraMovements.js` catalog (generated cut direction and the manual scene editor selector); persisted ids are unchanged. || `fableLoomPlayback.js` | FableLoom runtime version, node playback modes, audio occupancy and `evaluateFableLoomLiveInput`: shared live-input admission for connected decision holds, offscreen presence and safe audio, with legacy playback defaults. |diff --git a/server/lib/fableLoomExport.js b/server/lib/fableLoomExport.jsnew file mode 100644index 0000000000..26f14021d6--- /dev/null+++ b/server/lib/fableLoomExport.js@@ -0,0 +1,127 @@+import { analyzeEpisodeGraph, computeGraphLayers } from './fableLoomGraph.js';+import { analyzeSeriesStoryOutlines } from './fableLoomOutline.js';+import {+ asFableLoomPlaybackMode, FABLELOOM_RUNTIME_VERSION,+ inspectEpisodeProductionReadiness, resolveFableLoomProtagonistPresence,+ sanitizeInteractionWindow, validateAudioOccupancy,+} from './fableLoomPlayback.js';++export const FABLELOOM_EXPORT_SCHEMA_VERSION = '1.0.0';+export const FABLELOOM_EXPORT_LIMITS = Object.freeze({+ maxJobs: 4,+ maxAgeMs: 24 * 60 * 60 * 1000,+ maxAssetBytes: 64 * 1024 * 1024,+ maxPackageBytes: 256 * 1024 * 1024,+});++/** Only reachable scenes need media; structural and outline errors still block export. */+export function inspectFableLoomExport(loom, episodes, universe) {+ const selected = new Set(episodes.map(e => e.id));+ const findings = analyzeSeriesStoryOutlines(loom).issues+ .filter(issue => selected.has(issue.episodeId))+ .map(issue => ({ ...issue, gate: 'outline' }));+ if (selected.size !== episodes.length || episodes.some(e => !/^[A-Za-z0-9][A-Za-z0-9._-]{0,79}$/.test(e.id))) {+ findings.push({ gate: 'graph', code: 'INVALID_EXPORT_EPISODES', severity: 'error',+ message: 'Episode identifiers must be unique portable identifiers.' });+ }+ for (const episode of episodes) {+ const graph = analyzeEpisodeGraph(episode, {+ participationMode: loom.participationMode,+ requireAudienceIntroduction: episode.id === loom.episodes[0]?.id,+ });+ findings.push(...graph.issues.map(issue => ({ ...issue, gate: 'graph', episodeId: episode.id })));+ const ids = new Set();+ for (const node of episode.nodes) {+ const transitionIds = new Set();+ for (const id of [node.id, ...node.transitions.map(t => t.id)]) {+ if (typeof id !== 'string' || !/^[A-Za-z0-9][A-Za-z0-9._-]{0,199}$/.test(id)) {+ findings.push({ gate: 'graph', code: 'INVALID_EXPORT_ID', severity: 'error', episodeId: episode.id,+ message: 'Scene and transition identifiers must be portable identifiers.' });+ }+ }+ if (ids.has(node.id) || node.transitions.some(t => {+ const duplicate = transitionIds.has(t.id);+ transitionIds.add(t.id);+ return duplicate;+ })) {+ findings.push({ gate: 'graph', code: 'DUPLICATE_EXPORT_ID', severity: 'error', episodeId: episode.id,+ message: 'Scene identifiers and each scene’s transition identifiers must be unique.' });+ }+ ids.add(node.id);+ }+ const reachable = computeGraphLayers(episode).depthById;+ const ready = inspectEpisodeProductionReadiness({+ ...episode, nodes: episode.nodes.filter(n => reachable.has(n.id)),+ }, { loom, universe });+ for (const [nodeId, result] of Object.entries(ready.nodeResults)) {+ findings.push(...result.findings.map(issue => ({ ...issue, gate: 'production', episodeId: episode.id, nodeId })));+ }+ }+ return findings;+}++/** Explicit source references, never filenames reconstructed from a history id. */+export function fableLoomExportAssetRequests(node) {+ const assets = node.playbackAssets;+ const entryId = assets?.entryVideoHistoryId || node.videoHistoryId || null;+ const holds = node.isEnding || asFableLoomPlaybackMode(node.playbackMode) === 'cut'+ ? [] : (assets?.holdLoopVideoHistoryIds?.length ? assets.holdLoopVideoHistoryIds : [entryId]);+ return [+ { role: 'still', kind: 'image', sourceId: node.image },+ { role: 'entry', kind: 'video', sourceId: entryId },+ ...holds.map((sourceId, index) => ({ role: 'hold', kind: 'video', sourceId, index })),+ ...node.transitions.filter(t => assets?.exitByTransition?.[t.id])+ .map(t => ({ role: 'exit', kind: 'video', sourceId: assets.exitByTransition[t.id], transitionId: t.id })),+ ];+}++/** Machine-local configuration and provider prompts are deliberately absent. */+export function projectFableLoomExportNode(node, loom, assets) {+ const window = sanitizeInteractionWindow(node.interactionWindow);+ return {+ id: node.id, title: node.title, prose: node.prose,+ playbackMode: asFableLoomPlaybackMode(node.playbackMode),+ audienceConnection: node.audienceConnection,+ protagonistPresence: resolveFableLoomProtagonistPresence(node, loom) || 'offscreen',+ challengePhase: node.challengePhase ?? null,+ isEnding: node.isEnding === true, endingLabel: node.endingLabel || '',+ transitions: node.transitions.map(t => ({+ id: t.id, intent: t.intent, triggers: t.triggers, description: t.description, targetNodeId: t.targetNodeId,+ })),+ interactionWindow: window && {+ enabled: window.enabled, protagonistPresence: window.protagonistPresence,+ audioTarget: window.audioTarget, ambientDuckDb: window.ambientDuckDb, holdLoopRotation: window.holdLoopRotation,+ },+ playbackAssets: {+ still: assets.find(a => a.role === 'still')?.playback ?? null,+ entry: assets.find(a => a.role === 'entry')?.playback ?? null,+ holds: assets.filter(a => a.role === 'hold').map(a => a.playback),+ exits: Object.fromEntries(assets.filter(a => a.role === 'exit').map(a => [a.transitionId, a.playback])),+ },+ };+}++export function projectFableLoomExportManifest(loom, episodes, portosVersion) {+ return {+ schemaVersion: FABLELOOM_EXPORT_SCHEMA_VERSION, runtimeVersion: FABLELOOM_RUNTIME_VERSION, portosVersion,+ loom: {+ name: loom.name, logline: loom.logline, participationMode: loom.participationMode,+ audienceCommunicationMedium: loom.audienceCommunicationMedium, format: loom.format, episodes,+ },+ };+}++/** Occupancy keys refer to packaged bytes, and never to a local history record. */+export function fableLoomExportOccupancy(node, sourceId) {+ const raw = node.playbackAssets?.audioOccupancy?.[sourceId];+ if (!raw) return null;+ const occupancy = validateAudioOccupancy(raw);+ const intervals = list => list.map(({ startMs, endMs, blocking }) => ({+ startMs, endMs, ...(blocking === true ? { blocking: true } : {}),+ }));+ return {+ durationMs: occupancy.durationMs, characterDialogue: intervals(occupancy.characterDialogue),+ music: intervals(occupancy.music), effects: intervals(occupancy.effects),+ clipping: occupancy.clipping, safeForLiveVoice: occupancy.safeForLiveVoice,+ };+}diff --git a/server/lib/fableLoomValidation.js b/server/lib/fableLoomValidation.jsindex b9dac5b2e3..dee8da1c2e 100644--- a/server/lib/fableLoomValidation.js+++ b/server/lib/fableLoomValidation.js@@ -46,6 +46,17 @@ const title = z.string().max(LOOM_LIMITS.EPISODE_TITLE_MAX);const synopsis = z.string().max(LOOM_LIMITS.SYNOPSIS_MAX);const nodeIdStr = z.string().min(1).max(80);const format = z.enum(LOOM_FORMATS);++export const exportOptionsSchema = z.object({+ episodeIds: z.array(z.string().min(1).max(80)).min(1).max(LOOM_LIMITS.EPISODES_MAX)+ .refine(ids => new Set(ids).size === ids.length, 'Episode selections must be unique.').optional(),+ includePlayer: z.boolean().default(false),+ includeCharacters: z.boolean().default(false),+ includeVoice: z.boolean().default(false),+ allowDraft: z.boolean().default(false),+ mediaProfile: z.string().min(1).max(40).default('source'),+}).strict();+// The loom's pinned play routing is the shared per-record LLM route pin// (`server/lib/llmRoutePin.js`), nullable as a whole so the UI can clear it// outright. Its `effort` is the shared ladder enum rather than a free string —diff --git a/server/lib/hostControlRoutes.js b/server/lib/hostControlRoutes.jsindex d39429a496..bc55c366af 100644--- a/server/lib/hostControlRoutes.js+++ b/server/lib/hostControlRoutes.js@@ -1014,6 +1014,10 @@ export const HOST_CONTROL_ROUTES = Object.freeze([// Autopilots and support requests that queue CoS agents.'POST /api/pipeline/series/:id/autopilot/start','POST /api/fableloom/:id/editorial/autopilot/start',+ // Export preparation decodes authored media; download releases the resulting+ // private package. Both preserve authenticated delegated agent authority.+ 'POST /api/fableloom/:id/export',+ 'GET /api/fableloom/:id/export/:jobId/download','POST /api/image-video/models/support-request',// FableLoom authoring and production (#10668) can dispatch CLI/TUI agents,diff --git a/server/lib/index.js b/server/lib/index.jsindex 87a8635ae9..0dbd896eac 100644--- a/server/lib/index.js+++ b/server/lib/index.js@@ -132,6 +132,7 @@ export * as editorial from './editorial/index.js';export * from './familySettingsMap.js';export * from './fableLoomBoard.js';export * from './fableLoomGraph.js';+export * from './fableLoomExport.js';export * from './cameraMovements.js';export * from './fableLoomCameraMovements.js';export * from './fableLoomPlayback.js';diff --git a/server/routes/fableLoom.js b/server/routes/fableLoom.jsindex b577237d9c..bab6c722f6 100644--- a/server/routes/fableLoom.js+++ b/server/routes/fableLoom.js@@ -45,6 +45,7 @@ import {editorialAutopilotStartSchema,editorialRemediateSchema,playthroughReviewSchema,+ exportOptionsSchema,} from '../lib/fableLoomValidation.js';import { analyzeEpisodeGraph } from '../lib/fableLoomGraph.js';import { analyzeSeriesStoryOutlines } from '../lib/fableLoomOutline.js';@@ -130,6 +131,28 @@ router.get('/:id', asyncHandler(async (req, res) => {res.json(loom);}));+router.post('/:id/export', asyncHandler(async (req, res) => {+ const input = validateRequest(exportOptionsSchema, req.body ?? {});+ const { startFableLoomExport } = await import('../services/fableLoom/exports.js');+ res.status(202).json(await startFableLoomExport(req.params.id, input));+}));++router.get('/:id/export/:jobId', asyncHandler(async (req, res) => {+ const { getFableLoomExport } = await import('../services/fableLoom/exports.js');+ res.json(await getFableLoomExport(req.params.id, req.params.jobId));+}));++router.get('/:id/export/:jobId/download', asyncHandler(async (req, res) => {+ let artifact;+ let closed = false;+ res.once('close', () => { closed = true; artifact?.release(); });+ const { getFableLoomExportDownload } = await import('../services/fableLoom/exports.js');+ if (closed) return;+ artifact = await getFableLoomExportDownload(req.params.id, req.params.jobId);+ if (closed) { artifact.release(); return; }+ res.download(artifact.path, artifact.filename);+}));+router.patch('/:id', asyncHandler(async (req, res) => {const patch = validateRequest(loomPatchSchema, req.body);res.json(await updateLoom(req.params.id, patch));diff --git a/server/routes/hostControlFableLoom.test.js b/server/routes/hostControlFableLoom.test.jsindex 2d414f940f..8e252ff8c9 100644--- a/server/routes/hostControlFableLoom.test.js+++ b/server/routes/hostControlFableLoom.test.js@@ -86,7 +86,12 @@ vi.mock('../services/fableLoom/shotRuns.js', () => ({getEpisodeShotRun: vi.fn(async () => ({ id: 'example-record' })),cancelEpisodeShotRun: vi.fn(async () => ({ id: 'example-record' })),}));+vi.mock('../services/fableLoom/exports.js', () => ({+ startFableLoomExport: vi.fn(async () => ({ jobId: 'example-export' })),+ getFableLoomExportDownload: vi.fn(),+}));import * as shotRuns from '../services/fableLoom/shotRuns.js';+import * as exports from '../services/fableLoom/exports.js';import * as fableLoom from '../services/fableLoom/index.js';import { authGate, hostControlRouteGate, hostControlBodyGate } from '../services/authGate.js';import { createSession } from '../services/auth.js';@@ -98,6 +103,7 @@ const episode = `${loom}/episodes/example-episode`;const node = `${episode}/nodes/example-node`;const picker = { providerId: 'example-cli', model: 'example-model' };const operations = [+ [`${loom}/export`, {}, exports.startFableLoomExport, 202],[`${loom}/plan/generate`, picker, fableLoom.generateSeriesPlan],[`${loom}/plan/review`, picker, fableLoom.reviewSeriesPlan],[`${loom}/plan/feedback`, { ...picker, feedback: 'Example feedback' }, fableLoom.feedbackSeriesPlan],@@ -135,7 +141,7 @@ const call = (app, [path, body], headers = {}, method = 'post') => {return pending.send(body);};const expectNoEffects = () => {- for (const effect of [...Object.values(fableLoom), ...Object.values(shotRuns)]) expect(effect).not.toHaveBeenCalled();+ for (const effect of [...Object.values(fableLoom), ...Object.values(shotRuns), ...Object.values(exports)]) expect(effect).not.toHaveBeenCalled();};beforeEach(() => {@@ -147,6 +153,11 @@ beforeEach(() => {});describe('FableLoom agent workflow authority (#10668)', () => {+ it('protects private package downloads before reading an artifact', async () => {+ const result = await call(appFor(), [`${loom}/export/example-export/download`], {}, 'get');+ expect([result.status, result.body.code]).toEqual([403, 'HOST_CONTROL_FORBIDDEN']);+ expect(exports.getFableLoomExportDownload).not.toHaveBeenCalled();+ });it('refuses remote authoring and production before effects for CLI/TUI/API-first and Express spelling variants', async () => {const app = appFor();for (const [path, body] of operations) {diff --git a/server/services/fableLoom/README.md b/server/services/fableLoom/README.mdindex c1f2a97106..0b191bdd95 100644--- a/server/services/fableLoom/README.md+++ b/server/services/fableLoom/README.md@@ -18,6 +18,8 @@ intent to a transition and moves them through the graph until an ending.| `production.js` | Episodic production orchestration: batch planning, DAG generation, cancellable batch runs, and user-triggered episodic continuity review (#5384). || `falVideoAutomation.js` | User-triggered, serialized Playwright automation over the persistent PortOS CDP browser: uploads a scene still + full shot prompt to fal.ai H3 Max, downloads the finished MP4 into shared video history, and attaches it to the originating scene. || `store.js` | PostgreSQL/file backend facade (`fableloom_stories`; collectionStore escape hatch for tests). |+| `exports.js` | Immutable authored snapshots, readiness/approval gates, deterministic portable ZIP packages and bounded persistent export jobs. |+| `exportMedia.js` | Root-confined bounded media reads, metadata-free PNG stills and H.264/AAC MP4 stream copies; no network or generation. || `db.js` | PostgreSQL leaf I/O. |Pure graph analysis (validation, BFS layering, prompt rendering) lives indiff --git a/server/services/fableLoom/exportMedia.js b/server/services/fableLoom/exportMedia.jsnew file mode 100644index 0000000000..2e65b869c3--- /dev/null+++ b/server/services/fableLoom/exportMedia.js@@ -0,0 +1,98 @@+import { constants } from 'node:fs';+import { open, realpath, stat } from 'node:fs/promises';+import { join } from 'node:path';+import { promisify } from 'node:util';+import sharp from 'sharp';+import { execFile } from '../../lib/childProcess.js';+import { safeChildProcessOptions } from '../../lib/processEnv.js';+import { findFfmpeg, findFfprobe, runFfmpegProcess } from '../../lib/ffmpeg.js';+import { atomicWrite } from '../../lib/fileCore.js';+import { isPathInsideDir } from '../../lib/pathSafety.js';+import { FABLELOOM_EXPORT_LIMITS } from '../../lib/fableLoomExport.js';+import { ServerError } from '../../lib/errorHandler.js';++const execFileAsync = promisify(execFile);+const refusal = (code, message) => new ServerError(message, { status: 422, code });++/** Snapshot bounded bytes from a regular file in one approved media root. */+export async function readFableLoomExportSource(root, filename) {+ if (typeof filename !== 'string' || !/^[A-Za-z0-9][A-Za-z0-9._-]*\.(png|jpg|jpeg|webp|mp4)$/i.test(filename)) {+ throw refusal('EXPORT_MEDIA_PATH', 'Media must be a local gallery or video basename.');+ }+ const base = await realpath(root);+ const path = await realpath(join(base, filename));+ if (!isPathInsideDir(base, path)) throw refusal('EXPORT_MEDIA_PATH', 'Media resolves outside its approved root.');+ const handle = await open(path, constants.O_RDONLY | (constants.O_NOFOLLOW || 0));+ try {+ const before = await handle.stat();+ if (!before.isFile() || before.size <= 0 || before.size > FABLELOOM_EXPORT_LIMITS.maxAssetBytes) {+ throw refusal('EXPORT_MEDIA_SIZE', 'Media is empty, not a regular file, or exceeds the 64 MiB asset limit.');+ }+ const buffer = Buffer.alloc(before.size + 1);+ let offset = 0;+ while (offset < buffer.length) {+ const { bytesRead } = await handle.read(buffer, offset, buffer.length - offset, offset);+ if (!bytesRead) break;+ offset += bytesRead;+ }+ const after = await handle.stat();+ if (offset !== before.size || before.size !== after.size+ || before.mtimeMs !== after.mtimeMs || before.ctimeMs !== after.ctimeMs) {+ throw refusal('EXPORT_MEDIA_CHANGED', 'Media changed while taking the export snapshot. Start a new export.');+ }+ return buffer.subarray(0, offset);+ } finally {+ await handle.close();+ }+}++async function probe(path) {+ const bin = await findFfprobe();+ if (!bin) throw refusal('EXPORT_FFMPEG_REQUIRED', 'Install ffmpeg and ffprobe before exporting video.');+ const { stdout } = await execFileAsync(bin, [+ '-v', 'error', '-protocol_whitelist', 'file', '-f', 'mov', '-i', path,+ '-show_entries', 'stream=codec_type,codec_name,pix_fmt:format=duration', '-of', 'json',+ ], safeChildProcessOptions({ timeout: 10_000, killSignal: 'SIGKILL', maxBuffer: 64 * 1024 }));+ const result = JSON.parse(stdout);+ const video = result.streams?.filter(s => s.codec_type === 'video') || [];+ const audio = result.streams?.filter(s => s.codec_type === 'audio') || [];+ if (video.length !== 1 || video[0].codec_name !== 'h264' || video[0].pix_fmt !== 'yuv420p'+ || audio.length > 1 || audio.some(s => s.codec_name !== 'aac')) {+ throw refusal('EXPORT_ENCODING_UNSUPPORTED', 'Source profile requires one H.264 yuv420p video stream and optional AAC audio. Transcode profiles are not available yet.');+ }+ const durationMs = Math.round(Number(result.format?.duration) * 1000);+ if (!Number.isSafeInteger(durationMs) || durationMs <= 0) {+ throw refusal('EXPORT_MEDIA_INVALID', 'Video duration could not be verified.');+ }+ return durationMs;+}++/** Remove metadata without generating media or changing video/audio encodes. */+export async function prepareFableLoomExportMedia(bytes, kind, directory) {+ if (kind === 'image') {+ const image = sharp(bytes, { limitInputPixels: 16 * 1024 * 1024 });+ const metadata = await image.metadata();+ if (!['png', 'jpeg', 'webp'].includes(metadata.format) || (metadata.pages || 1) !== 1) {+ throw refusal('EXPORT_ENCODING_UNSUPPORTED', 'Stills must be non-animated PNG, JPEG or WebP images.');+ }+ return { bytes: await image.rotate().png({ compressionLevel: 9 }).toBuffer(), extension: 'png', durationMs: 0 };+ }+ const input = join(directory, 'input.mp4');+ const output = join(directory, 'clean.mp4');+ await atomicWrite(input, bytes);+ await probe(input);+ const bin = await findFfmpeg();+ if (!bin) throw refusal('EXPORT_FFMPEG_REQUIRED', 'Install ffmpeg and ffprobe before exporting video.');+ const result = await runFfmpegProcess({ bin, timeoutMs: 60_000, args: [+ '-nostdin', '-y', '-v', 'error', '-protocol_whitelist', 'file', '-f', 'mov', '-i', input,+ '-map', '0:v:0', '-map', '0:a:0?', '-map_metadata', '-1', '-map_metadata:s', '-1',+ '-map_chapters', '-1', '-c', 'copy', '-bsf:v', 'filter_units=remove_types=6',+ '-fflags', '+bitexact', '-movflags', '+faststart', output,+ ] });+ if (!result.ok) throw refusal('EXPORT_MEDIA_INVALID', 'Video metadata removal failed. Check the source clip and try again.');+ const outputInfo = await stat(output);+ if (outputInfo.size > FABLELOOM_EXPORT_LIMITS.maxAssetBytes) {+ throw refusal('EXPORT_MEDIA_SIZE', 'Prepared media exceeds the 64 MiB asset limit.');+ }+ return { bytes: await readFableLoomExportSource(directory, 'clean.mp4'), extension: 'mp4', durationMs: await probe(output) };+}diff --git a/server/services/fableLoom/exports.js b/server/services/fableLoom/exports.jsnew file mode 100644index 0000000000..94e9efd2b9--- /dev/null+++ b/server/services/fableLoom/exports.js@@ -0,0 +1,360 @@+import { createHash, randomUUID } from 'node:crypto';+import { readFile, readdir, realpath, stat } from 'node:fs/promises';+import { join } from 'node:path';+import { PATHS } from '../../lib/paths.js';+import { atomicWrite, ensureDir, rmGuarded } from '../../lib/fileCore.js';+import { readJSONFile } from '../../lib/jsonIo.js';+import { canonicalStringify } from '../../lib/objects.js';+import { isPathInsideDir } from '../../lib/pathSafety.js';+import { createZip } from '../../lib/zipWriter.js';+import { maintenance } from '../../lib/maintenanceAdmission.js';+import { ServerError } from '../../lib/errorHandler.js';+import { exportOptionsSchema } from '../../lib/fableLoomValidation.js';+import { computeGraphLayers } from '../../lib/fableLoomGraph.js';+import {+ FABLELOOM_EXPORT_LIMITS, FABLELOOM_EXPORT_SCHEMA_VERSION,+ fableLoomExportAssetRequests, fableLoomExportOccupancy, inspectFableLoomExport,+ projectFableLoomExportManifest, projectFableLoomExportNode,+} from '../../lib/fableLoomExport.js';+import { getUniverse } from '../universeBuilder.js';+import { loadHistory } from '../videoGen/history.js';+import { getLoom } from './records.js';+import { prepareFableLoomExportMedia, readFableLoomExportSource } from './exportMedia.js';++const uuid = /^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$/;+const active = new Map();+const downloads = new Map();+let admissionTail = Promise.resolve();+let workerTail = Promise.resolve();+const root = () => join(PATHS.data, 'cache', 'fableloom-exports');+const jobDirectory = id => join(root(), id);+const jobFile = id => join(jobDirectory(id), 'job.json');+const hash = bytes => createHash('sha256').update(bytes).digest('hex');+const json = value => canonicalStringify(value) + '\n';+const error = (code, message, status = 422) => new ServerError(message, { status, code });+const missing = () => error('NOT_FOUND', 'Export job not found for this loom.', 404);+const save = job => atomicWrite(jobFile(job.jobId), job);+function serializeAdmission(action) {+ const pending = admissionTail.then(action);+ admissionTail = pending.catch(() => {});+ return pending;+}+const portableLabel = value => typeof value === 'string'+ && /^[A-Za-z0-9][A-Za-z0-9._/@:-]{0,199}$/.test(value)+ && !/^(file:|[A-Za-z]:)/i.test(value) && !value.includes('://') ? value : null;++function publicJob(job) {+ return {+ jobId: job.jobId, status: job.status, progress: job.progress, findings: job.findings,+ expiresAt: new Date(job.createdAt + FABLELOOM_EXPORT_LIMITS.maxAgeMs).toISOString(),+ ...(job.error ? { error: job.error } : {}),+ ...(job.sizeReport ? { sizeReport: job.sizeReport } : {}),+ ...(job.status === 'completed' ? {+ downloadUrl: `/api/fableloom/${encodeURIComponent(job.loomId)}/export/${job.jobId}/download`,+ } : {}),+ };+}++async function readJob(id, loomId = null) {+ if (!uuid.test(id)) throw missing();+ const job = await readJSONFile(jobFile(id), null, { strict: true, logError: false }).catch(() => {+ throw error('EXPORT_STATE_INVALID', 'Export state is unreadable or from an unsupported version.', 503);+ });+ if (!job) return null;+ if (loomId !== null && job.loomId !== loomId) throw missing();+ if (job.version !== 1 || job.jobId !== id || typeof job.loomId !== 'string'+ || !Number.isFinite(job.createdAt) || !Array.isArray(job.findings)+ || !['queued', 'validating', 'packing', 'completed', 'failed'].includes(job.status)) {+ throw error('EXPORT_STATE_INVALID', 'Export state is unreadable or from an unsupported version.', 503);+ }+ if (!active.has(id) && !['completed', 'failed'].includes(job.status)) {+ await rmGuarded(join(jobDirectory(id), 'work'), { recursive: true, force: true });+ job.status = 'failed';+ job.error = { code: 'EXPORT_INTERRUPTED', message: 'The server restarted during export. Start a new export.' };+ await save(job);+ }+ return job;+}++async function prune() {+ await ensureDir(root());+ const entries = await readdir(root(), { withFileTypes: true });+ const retained = [];+ for (const entry of entries) {+ if (!entry.isDirectory() || !uuid.test(entry.name) || active.has(entry.name)) continue;+ const job = await readJob(entry.name);+ if (!job) throw error('EXPORT_STATE_INVALID', 'An incomplete export directory needs recovery.', 503);+ if (Date.now() - job.createdAt >= FABLELOOM_EXPORT_LIMITS.maxAgeMs && !downloads.has(entry.name)) {+ await rmGuarded(jobDirectory(entry.name), { recursive: true, force: true });+ } else retained.push(job);+ }+ retained.sort((a, b) => a.createdAt - b.createdAt || a.jobId.localeCompare(b.jobId));+ while (retained.length + active.size >= FABLELOOM_EXPORT_LIMITS.maxJobs && retained.length) {+ const index = retained.findIndex(job => !downloads.has(job.jobId));+ if (index < 0) break;+ const [oldest] = retained.splice(index, 1);+ await rmGuarded(jobDirectory(oldest.jobId), { recursive: true, force: true });+ }+ if (active.size + retained.length >= FABLELOOM_EXPORT_LIMITS.maxJobs) {+ throw error('EXPORT_QUEUE_FULL', 'Four exports are queued, running or being downloaded. Wait for one to finish.', 429);+ }+}++function assetProvenance(loom, node, request) {+ const assets = node.playbackAssets;+ const perAsset = assets?.visualConditioningByAsset || {};+ // Rendering a video replaces the node's latest manifest. Image attachment+ // preserves exactly the current still's manifest in the per-asset map.+ const imageConditioning = node.visualConditioning?.capability?.kind === 'image'+ ? node.visualConditioning : Object.values(perAsset).find(m => m?.capability?.kind === 'image');+ const typedVideo = perAsset[request.sourceId]?.capability?.kind === 'video'+ ? perAsset[request.sourceId] : null;+ const hasTypedPlayback = assets?.entryVideoHistoryId || assets?.holdLoopVideoHistoryIds?.length+ || Object.keys(assets?.exitByTransition || {}).length;+ const legacyVideo = !hasTypedPlayback && node.videoHistoryId === request.sourceId+ && node.visualConditioning?.capability?.kind === 'video' ? node.visualConditioning : null;+ const conditioning = request.kind === 'image' ? imageConditioning : typedVideo || legacyVideo;+ const approved = request.kind === 'image' ? node.visualCanon?.storyboardImageApproved === true+ : typeof loom.productionStatus?.deliveryApprovedAt === 'string'+ && Number.isFinite(Date.parse(loom.productionStatus.deliveryApprovedAt));+ const conditioningStatus = ['locked', 'draft', 'degraded'].includes(conditioning?.status)+ ? conditioning.status : 'unknown';+ const draftFlags = [+ ...(!approved ? ['unapproved'] : []),+ ...(conditioningStatus !== 'locked' ? [conditioningStatus] : []),+ ];+ return {+ approved, conditioningStatus, draftFlags,+ provider: portableLabel(conditioning?.render?.provider || conditioning?.capability?.backend),+ model: portableLabel(conditioning?.render?.modelId || conditioning?.capability?.modelId),+ parametersHash: hash(json(conditioning?.render?.parameters || {})),+ };+}++const packageReadme = `# FableLoom portable package++Package schema: 1.0.0. Playback runtime: 1.0.0 (see runtime-v1.md).++Serve these files on your own static host. PortOS is an authoring tool and is+never the public server, relay, or live resolver for this package.++manifest.json contains the ordered episodes and reachable scene graphs. Media+references name only media/<sha256>.<extension>, with durationMs and loopable.+Entry clips auto-advance cut scenes; decision holds repeat until a valid choice.+Exit clips are optional and keyed by transition id; absent exits cut directly.+Occupancy intervals are milliseconds relative to that packaged clip.++provenance.json records each media use, provider/model, parameter hash, approval+state and draftFlags. An allowDraft export does not certify production approval.++The source media profile strips metadata, encodes stills as PNG and stream-copies+H.264 yuv420p / optional AAC into MP4. It does not resize, normalize loudness or+transcode unsupported codecs. The player, character and voice opt-ins are not+available in this package version. Your host owns free-text resolution, TTS and+transport, following runtime-v1.md; no PortOS URL or credential is needed.+`;++async function build(job, snapshot, episodes, history, universe) {+ job.status = 'validating';+ job.findings = inspectFableLoomExport(snapshot, episodes, universe);+ if (job.findings.some(f => f.severity === 'error')) {+ throw error('EXPORT_VALIDATION_FAILED', 'Resolve the graph, outline and production findings before exporting.');+ }+ const requests = episodes.flatMap(episode => {+ const reachable = computeGraphLayers(episode).depthById;+ return episode.nodes.filter(n => reachable.has(n.id)).flatMap(node => (+ fableLoomExportAssetRequests(node).map(request => ({ ...request, episode, node }))+ ));+ });+ for (const request of requests) {+ const provenance = assetProvenance(snapshot, request.node, request);+ if (!request.sourceId) job.findings.push({+ gate: 'media', severity: 'error', code: 'EXPORT_MEDIA_MISSING',+ episodeId: request.episode.id, nodeId: request.node.id,+ message: `A required ${request.role} asset is missing.`,+ });+ if (provenance.draftFlags.length) job.findings.push({+ gate: 'approval', severity: job.options.allowDraft ? 'warning' : 'error',+ code: 'EXPORT_ASSET_DRAFT', episodeId: request.episode.id, nodeId: request.node.id,+ message: `The ${request.role} asset is unapproved or has draft, degraded or unknown provenance.`,+ });+ }+ if (job.findings.some(f => f.severity === 'error')) {+ throw error('EXPORT_VALIDATION_FAILED', 'Resolve missing media or approval findings, or explicitly allow draft assets.');+ }+ job.status = 'packing';+ job.progress = { completed: 0, total: requests.length };+ await save(job);+ const directory = join(jobDirectory(job.jobId), 'work');+ await ensureDir(directory);+ const cache = new Map();+ const media = new Map();+ const nodeAssets = new Map();+ const provenance = [];+ let mediaBytes = 0;+ for (const request of requests) {+ const key = `${request.kind}:${request.sourceId}`;+ let prepared = cache.get(key);+ if (!prepared) {+ const filename = request.kind === 'image' ? request.sourceId : history.find(h => h.id === request.sourceId)?.filename;+ if (!filename) throw error('EXPORT_MEDIA_MISSING', 'A referenced video history item is missing.');+ const bytes = await readFableLoomExportSource(request.kind === 'image' ? PATHS.images : PATHS.videos, filename);+ prepared = await prepareFableLoomExportMedia(bytes, request.kind, directory);+ if (prepared.bytes.length > FABLELOOM_EXPORT_LIMITS.maxAssetBytes) {+ throw error('EXPORT_MEDIA_SIZE', 'Prepared media exceeds the 64 MiB asset limit.');+ }+ prepared.sha256 = hash(prepared.bytes);+ prepared.file = `media/${prepared.sha256}.${prepared.extension}`;+ if (!media.has(prepared.file)) {+ mediaBytes += prepared.bytes.length;+ if (mediaBytes > FABLELOOM_EXPORT_LIMITS.maxPackageBytes) {+ throw error('EXPORT_PACKAGE_SIZE', 'Media exceeds the 256 MiB package limit. Export fewer episodes.');+ }+ media.set(prepared.file, prepared.bytes);+ }+ cache.set(key, prepared);+ }+ const occupancy = request.kind === 'video' ? fableLoomExportOccupancy(request.node, request.sourceId) : null;+ if (request.node.interactionWindow?.enabled && request.role === 'hold'+ && (!occupancy || !occupancy.safeForLiveVoice)) {+ throw error('EXPORT_OCCUPANCY_UNSAFE', 'Live holds require a verified speech-safe audio occupancy manifest.');+ }+ if (occupancy && (occupancy.durationMs > prepared.durationMs+ || ['characterDialogue', 'music', 'effects'].some(key => occupancy[key].some(i => i.endMs > prepared.durationMs)))) {+ throw error('EXPORT_OCCUPANCY_INVALID', 'An audio occupancy interval exceeds its clip duration.');+ }+ const binding = {+ ...request,+ playback: { file: prepared.file, sha256: prepared.sha256, durationMs: prepared.durationMs,+ loopable: request.role === 'hold', ...(occupancy ? { audioOccupancy: occupancy } : {}) },+ };+ const nodeKey = `${request.episode.id}:${request.node.id}`;+ if (!nodeAssets.has(nodeKey)) nodeAssets.set(nodeKey, []);+ nodeAssets.get(nodeKey).push(binding);+ provenance.push({+ episodeId: request.episode.id, nodeId: request.node.id, role: request.role,+ ...(request.transitionId ? { transitionId: request.transitionId } : {}),+ ...(request.index !== undefined ? { index: request.index } : {}),+ file: prepared.file, sha256: prepared.sha256, ...assetProvenance(snapshot, request.node, request),+ });+ job.progress.completed++;+ await save(job);+ }+ const portos = JSON.parse(await readFile(new URL('../../../package.json', import.meta.url), 'utf8'));+ const manifest = projectFableLoomExportManifest(snapshot, episodes.map(episode => ({+ id: episode.id, number: episode.number, title: episode.title, synopsis: episode.synopsis, startNodeId: episode.startNodeId,+ nodes: episode.nodes.filter(n => nodeAssets.has(`${episode.id}:${n.id}`))+ .map(node => projectFableLoomExportNode(node, snapshot, nodeAssets.get(`${episode.id}:${node.id}`))),+ })), portos.version);+ const entries = [+ { name: 'manifest.json', data: json(manifest) },+ { name: 'provenance.json', data: json({ schemaVersion: FABLELOOM_EXPORT_SCHEMA_VERSION, assets: provenance }) },+ { name: 'README.md', data: packageReadme },+ { name: 'runtime-v1.md', data: await readFile(new URL('../../../docs/features/fableloom-runtime-v1.md', import.meta.url)) },+ ...[...media].map(([name, data]) => ({ name, data })),+ ].sort((a, b) => a.name < b.name ? -1 : a.name > b.name ? 1 : 0);+ const totalBytes = entries.reduce((n, e) => n + Buffer.byteLength(e.data), 0);+ if (totalBytes > FABLELOOM_EXPORT_LIMITS.maxPackageBytes) {+ throw error('EXPORT_PACKAGE_SIZE', 'Package exceeds the 256 MiB limit. Export fewer episodes.');+ }+ const archive = createZip(entries);+ if (archive.length > FABLELOOM_EXPORT_LIMITS.maxPackageBytes) {+ throw error('EXPORT_PACKAGE_SIZE', 'ZIP exceeds the 256 MiB limit. Export fewer episodes.');+ }+ await atomicWrite(join(jobDirectory(job.jobId), 'package.zip'), archive);+ job.filename = `fableloom-${hash(json(manifest)).slice(0, 16)}-v1.zip`;+ job.sizeReport = { mediaBytes, totalBytes, archiveBytes: archive.length, mediaFiles: media.size };+ job.status = 'completed';+}++async function run(job, context, permit) {+ let settled = false;+ try {+ try {+ await build(job, context.snapshot, context.episodes, context.history, context.universe);+ } catch (failure) {+ job.status = 'failed';+ job.error = failure.code?.startsWith('EXPORT_')+ ? { code: failure.code, message: failure.message }+ : { code: 'EXPORT_MEDIA_INVALID', message: 'A source media file is missing, unreadable or invalid. Check its attached media and start a new export.' };+ }+ await rmGuarded(join(jobDirectory(job.jobId), 'work'), { recursive: true, force: true });+ await save(job);+ settled = true;+ } finally {+ active.delete(job.jobId);+ if (settled) await permit.finish();+ else permit.markUnsettled();+ }+}++export async function startFableLoomExport(loomId, input = {}) {+ const options = exportOptionsSchema.parse(input);+ if (options.includePlayer || options.includeCharacters || options.includeVoice || options.mediaProfile !== 'source') {+ throw error('EXPORT_OPTION_UNSUPPORTED', 'This exporter supports the source profile without player, character or voice packages.', 400);+ }+ const jobId = randomUUID();+ const permit = maintenance.admit('media', jobId);+ try {+ const context = await serializeAdmission(async () => {+ const loom = await getLoom(loomId);+ if (!loom) throw error('NOT_FOUND', 'Loom not found.', 404);+ const ids = new Set(options.episodeIds ?? loom.episodes.map(e => e.id));+ if (!ids.size || loom.episodes.filter(e => ids.has(e.id)).length !== ids.size) {+ throw error('EXPORT_EPISODES_INVALID', 'Select existing episodes from this loom.', 400);+ }+ await prune();+ const snapshot = structuredClone(loom);+ const episodes = snapshot.episodes.filter(e => ids.has(e.id));+ const history = structuredClone(await loadHistory());+ if (!Array.isArray(history)) throw error('EXPORT_MEDIA_INVALID', 'Video history is unreadable.');+ const universe = snapshot.universeId ? structuredClone(await getUniverse(snapshot.universeId)) : null;+ const job = { version: 1, jobId, loomId, createdAt: Date.now(), status: 'queued', options,+ progress: { completed: 0, total: 0 }, findings: [] };+ await save(job);+ active.set(jobId, job);+ return { job, snapshot, episodes, history, universe };+ });+ const pending = workerTail.then(() => permit.run(() => run(context.job, context, permit)));+ workerTail = pending.catch(() => { console.error('❌ FableLoom export persistence failed; maintenance ownership is retained.'); });+ return { jobId };+ } catch (failure) {+ await permit.finish();+ throw failure;+ }+}++export async function getFableLoomExport(loomId, jobId) {+ if (!await getLoom(loomId)) throw missing();+ const job = await readJob(jobId, loomId);+ if (!job || job.loomId !== loomId) throw missing();+ if (Date.now() - job.createdAt >= FABLELOOM_EXPORT_LIMITS.maxAgeMs) {+ throw error('EXPORT_EXPIRED', 'This export has expired. Start a new export.', 410);+ }+ return publicJob(job);+}++export async function getFableLoomExportDownload(loomId, jobId) {+ return serializeAdmission(async () => {+ const job = await getFableLoomExport(loomId, jobId);+ if (job.status !== 'completed') throw error('EXPORT_NOT_READY', 'The export has not completed successfully.', 409);+ const path = await realpath(join(jobDirectory(jobId), 'package.zip'));+ const base = await realpath(jobDirectory(jobId));+ const info = await stat(path);+ if (!isPathInsideDir(base, path) || !info.isFile() || info.size !== job.sizeReport.archiveBytes) {+ throw error('EXPORT_ARTIFACT_INVALID', 'The export artifact is unavailable. Start a new export.', 409);+ }+ const stored = await readJob(jobId);+ downloads.set(jobId, (downloads.get(jobId) || 0) + 1);+ let released = false;+ return { path, filename: stored.filename, release: () => {+ if (released) return;+ released = true;+ const remaining = downloads.get(jobId) - 1;+ if (remaining) downloads.set(jobId, remaining);+ else downloads.delete(jobId);+ } };+ });+}++export const __testing = { drain: async () => { await admissionTail; await workerTail; } };diff --git a/server/services/fableLoom/exports.test.js b/server/services/fableLoom/exports.test.jsnew file mode 100644index 0000000000..d81b1bd320--- /dev/null+++ b/server/services/fableLoom/exports.test.js@@ -0,0 +1,313 @@+import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it as vitestIt, vi } from 'vitest';+import { createHash } from 'node:crypto';+import { mkdir, readFile, rm, stat, symlink, truncate, writeFile } from 'node:fs/promises';+import { join } from 'node:path';+import express from 'express';+import sharp from 'sharp';+import { cleanupTempDataRoots, lazyTempDataRoot, makePathsProxy, ownTestBodies } from '../../lib/mockPathsDataRoot.js';+import { request } from '../../lib/testHelper.js';+import { errorMiddleware } from '../../lib/errorHandler.js';+import { readZipArchive } from '../../lib/zipArchive.js';+import { findFfmpeg, findFfprobe, runFfmpegProcess } from '../../lib/ffmpeg.js';+import { FABLELOOM_EXPORT_LIMITS } from '../../lib/fableLoomExport.js';++vi.mock('../../lib/paths.js', async original => makePathsProxy(await original(), {+ dataRoot: () => lazyTempDataRoot('portos-fableloom-export-'),+}));+vi.mock('./records.js', () => ({ getLoom: vi.fn() }));+vi.mock('../videoGen/history.js', () => ({ loadHistory: vi.fn() }));+vi.mock('../universeBuilder.js', () => ({ getUniverse: vi.fn() }));+vi.mock('./index.js', () => ({}));+vi.mock('./shotRuns.js', () => ({}));++import { PATHS } from '../../lib/paths.js';+import { getLoom } from './records.js';+import { loadHistory } from '../videoGen/history.js';+import { getUniverse } from '../universeBuilder.js';+import * as media from './exportMedia.js';+import {+ __testing, startFableLoomExport, getFableLoomExport, getFableLoomExportDownload,+} from './exports.js';+import routes from '../../routes/fableLoom.js';++const owned = ownTestBodies(vitestIt);+const it = owned.it;+const bin = await findFfmpeg();+const probe = await findFfprobe();+const canPrepareVideo = Boolean(bin && probe);+const videoIt = canPrepareVideo ? it : vitestIt.skip;+const privateMarker = 'EXAMPLE_PRIVATE_SENTINEL';+const sourcePath = '/example-private/source';+let sourceImage;+let sourceVideo;+let loom;+const jobIds = [];+const hash = bytes => createHash('sha256').update(bytes).digest('hex');+const recordFile = id => join(PATHS.data, 'cache/fableloom-exports', id, 'job.json');+const app = () => express().use(express.json()).use('/api/fableloom', routes).use(errorMiddleware);+const conditioning = kind => ({+ version: 1, status: 'locked', capability: { kind, backend: 'example-provider', modelId: 'Example/Model' },+ render: { provider: 'example-provider', modelId: 'Example/Model', parameters: { seed: 42, secret: privateMarker } },+ adapters: [{ filename: sourcePath }], assets: [{ filename: sourcePath }],+ compiledPrompt: privateMarker, instanceId: privateMarker,+});+function episode(id, number = 1) {+ const nodes = [+ { id: id + '-start', title: 'At the gate', prose: 'A traveller reaches a quiet gate.',+ playbackMode: 'decision', audienceConnection: 'connected', protagonistPresence: 'offscreen',+ isEnding: false, endingLabel: '', image: 'still.png',+ visualCanon: { storyboardImageApproved: true }, visualConditioning: conditioning('image'),+ videoHistoryId: 'entry-record',+ playbackAssets: { entryVideoHistoryId: 'entry-record', holdLoopVideoHistoryIds: ['hold-record'],+ exitByTransition: { choose: 'exit-record' },+ visualConditioningByAsset: Object.fromEntries(['entry-record', 'hold-record', 'exit-record'].map(key => [key, conditioning('video')])),+ audioOccupancy: { 'hold-record': { durationMs: 1000, effects: [+ { startMs: 0, endMs: 500, blocking: false, name: privateMarker },+ ] } } },+ interactionWindow: { enabled: true, protagonistCharacterId: 'example-character', protagonistPresence: 'offscreen' },+ transitions: [+ { id: 'choose', intent: 'Open', triggers: ['open'], description: 'Open the gate.', targetNodeId: id + '-end' },+ { id: 'leave', intent: 'Leave', triggers: ['leave'], description: 'Take another path.', targetNodeId: id + '-end' },+ ], sessions: [privateMarker], apiKey: privateMarker },+ { id: id + '-end', title: 'Home', prose: 'The traveller finds a home.',+ playbackMode: 'cut', audienceConnection: 'connected', protagonistPresence: 'offscreen',+ isEnding: true, endingLabel: 'Home', image: 'still.png', videoHistoryId: 'entry-record',+ visualCanon: { storyboardImageApproved: true }, visualConditioning: conditioning('image'),+ playbackAssets: { entryVideoHistoryId: 'entry-record', visualConditioningByAsset: { 'entry-record': conditioning('video') } },+ transitions: [] },+ ];+ return { id, number, title: 'Example episode', synopsis: 'A journey.', startNodeId: nodes[0].id, nodes,+ storyOutline: { startKey: nodes[0].id, validation: { status: 'valid' },+ scenes: nodes.map(node => ({ ...node, key: node.id, summary: node.title,+ transitions: node.transitions.map(t => ({ ...t, targetKey: t.targetNodeId })) })) } };+}++async function begin(options = {}) {+ const result = await startFableLoomExport(loom.id, options);+ jobIds.push(result.jobId);+ return result.jobId;+}+async function terminal(id) {+ let result;+ await vi.waitFor(async () => {+ result = await getFableLoomExport(loom.id, id);+ expect(['completed', 'failed']).toContain(result.status);+ }, { timeout: 10_000, interval: 20 });+ return result;+}+async function archive(id) {+ const result = await terminal(id);+ expect(result, JSON.stringify(result)).toMatchObject({ status: 'completed' });+ const artifact = await getFableLoomExportDownload(loom.id, id);+ try {+ const bytes = await readFile(artifact.path);+ return { bytes, files: new Map(readZipArchive(bytes).map(e => [e.name, e.read()])) };+ } finally { artifact.release(); }+}++beforeAll(async () => {+ await mkdir(PATHS.images, { recursive: true });+ await mkdir(PATHS.videos, { recursive: true });+ sourceImage = await sharp({ create: { width: 32, height: 32, channels: 3, background: '#345678' } })+ .withMetadata({ exif: { IFD0: { ImageDescription: privateMarker + sourcePath } } }).png().toBuffer();+ await writeFile(join(PATHS.images, 'still.png'), sourceImage);+ if (canPrepareVideo) {+ const output = join(PATHS.videos, 'independent-name.mp4');+ const encoded = await runFfmpegProcess({ bin, timeoutMs: 10_000, args: [+ '-nostdin', '-y', '-v', 'error', '-f', 'lavfi', '-i', 'color=c=blue:s=32x32:r=10:d=1',+ '-f', 'lavfi', '-i', 'anullsrc=r=44100:cl=mono', '-t', '1',+ '-c:v', 'libx264', '-c:a', 'aac', '-pix_fmt', 'yuv420p',+ '-bsf:v', 'h264_metadata=sei_user_data=0123456789abcdef0123456789abcdef+' + privateMarker,+ '-metadata', 'comment=' + privateMarker + sourcePath, output,+ ] });+ expect(encoded).toEqual({ ok: true });+ sourceVideo = await readFile(output);+ expect(sourceVideo.indexOf(privateMarker)).not.toBe(sourceVideo.lastIndexOf(privateMarker));+ }+});+beforeEach(async () => {+ await __testing.drain();+ vi.restoreAllMocks();+ loom = { id: 'example-loom', name: 'The quiet gate', logline: 'An unexpected journey.',+ participationMode: 'protagonist', audienceCommunicationMedium: 'Walkie-talkie', format: 'prose',+ universeId: 'example-universe', productionStatus: { deliveryApprovedAt: '2026-01-01T00:00:00.000Z' },+ episodes: [episode('episode-one'), episode('episode-two', 2)],+ sessionTranscripts: privateMarker, peerId: privateMarker, providerConfig: { apiKey: privateMarker } };+ getLoom.mockImplementation(async id => id === loom.id || id === 'another-loom' ? loom : null);+ getUniverse.mockResolvedValue({ characters: [{ id: 'example-character' }] });+ loadHistory.mockResolvedValue(['entry-record', 'hold-record', 'exit-record'].map(id => ({+ id, filename: 'independent-name.mp4', absolutePath: sourcePath, prompt: privateMarker, apiKey: privateMarker,+ })));+ jobIds.length = 0;+ await rm(join(PATHS.data, 'cache/fableloom-exports'), { recursive: true, force: true });+});+afterEach(async () => { await __testing.drain(); vi.restoreAllMocks(); });+afterAll(async () => { await owned.drain(); await __testing.drain(); cleanupTempDataRoots(); });++describe('portable FableLoom export boundary', () => {+ videoIt('exports selected episodes deterministically, strips private data and embedded tags, and serves the ZIP', async () => {+ const started = await request(app()).post('/api/fableloom/example-loom/export').send({ episodeIds: ['episode-two'] });+ expect(started.status).toBe(202);+ jobIds.push(started.body.jobId);+ const first = await archive(started.body.jobId);+ const second = await archive(await begin({ episodeIds: ['episode-two'] }));+ expect(second.bytes.equals(first.bytes)).toBe(true);+ const manifest = JSON.parse(first.files.get('manifest.json'));+ expect(manifest).toMatchObject({ schemaVersion: '1.0.0', runtimeVersion: '1.0.0',+ loom: { name: 'The quiet gate', episodes: [{ id: 'episode-two' }] } });+ expect(manifest.loom.episodes).toHaveLength(1);+ const [start, end] = manifest.loom.episodes[0].nodes;+ expect(start.playbackAssets.holds[0]).toMatchObject({ loopable: true, durationMs: 1000,+ audioOccupancy: { safeForLiveVoice: true } });+ expect(start.playbackAssets.exits.choose).toMatchObject({ loopable: false, durationMs: 1000 });+ expect(end).toMatchObject({ isEnding: true, endingLabel: 'Home' });+ expect(first.files.has('runtime-v1.md')).toBe(true);+ const mediaFiles = [...first.files].filter(([name]) => name.startsWith('media/'));+ expect(mediaFiles).toHaveLength(2);+ for (const [name, bytes] of mediaFiles) expect(name).toContain(hash(bytes));+ for (const bytes of [first.bytes, ...first.files.values()]) {+ expect(bytes.includes(Buffer.from(privateMarker))).toBe(false);+ expect(bytes.includes(Buffer.from(sourcePath))).toBe(false);+ }+ expect(await readFile(join(PATHS.images, 'still.png'))).toEqual(sourceImage);+ expect(await readFile(join(PATHS.videos, 'independent-name.mp4'))).toEqual(sourceVideo);+ const status = await request(app()).get('/api/fableloom/example-loom/export/' + started.body.jobId);+ expect(status.body).toMatchObject({ status: 'completed', sizeReport: { mediaFiles: 2, archiveBytes: first.bytes.length } });+ const download = await request(app()).get(status.body.downloadUrl);+ expect(download.status).toBe(200);+ expect(download.headers['content-type']).toMatch(/zip/);+ expect(download.headers['content-disposition']).toMatch(/fableloom-[a-f0-9]{16}-v1.zip/);+ const wrongLoom = await request(app()).get('/api/fableloom/another-loom/export/' + started.body.jobId);+ expect(wrongLoom.status).toBe(404);+ await expect(getFableLoomExportDownload('another-loom', started.body.jobId)).rejects.toMatchObject({ status: 404 });+ });++ videoIt('refuses unapproved media by default and records draft flags only after explicit opt-in', async () => {+ const node = loom.episodes[0].nodes[0];+ node.visualCanon.storyboardImageApproved = false;+ node.playbackAssets.visualConditioningByAsset['asset-' + node.id + '-still'] = node.visualConditioning;+ node.visualConditioning = conditioning('video');+ node.visualConditioning.render.provider = 'example-video-provider';+ const refused = await terminal(await begin());+ expect(refused).toMatchObject({ status: 'failed', error: { code: 'EXPORT_VALIDATION_FAILED' } });+ expect(refused.findings).toContainEqual(expect.objectContaining({ gate: 'approval', severity: 'error' }));+ expect(refused).not.toHaveProperty('downloadUrl');+ const draft = await archive(await begin({ allowDraft: true }));+ const provenance = JSON.parse(draft.files.get('provenance.json'));+ expect(provenance.assets).toContainEqual(expect.objectContaining({+ role: 'still', approved: false, draftFlags: ['unapproved'], parametersHash: expect.stringMatching(/^[a-f0-9]{64}$/),+ }));+ expect(provenance.assets.every(a => a.provider === 'example-provider')).toBe(true);+ });++ it('keeps structural and required-media failures blocking under allowDraft', async () => {+ loom.episodes[0].nodes[0].transitions[0].targetNodeId = 'missing-node';+ const graph = await terminal(await begin({ allowDraft: true }));+ expect(graph).toMatchObject({ status: 'failed', error: { code: 'EXPORT_VALIDATION_FAILED' } });+ expect(graph.findings).toContainEqual(expect.objectContaining({ gate: 'graph', code: 'DANGLING_TRANSITION' }));+ loom.episodes = [episode('episode-one')];+ loom.episodes[0].nodes[0].image = null;+ const missing = await terminal(await begin({ allowDraft: true }));+ expect(missing.findings).toContainEqual(expect.objectContaining({ gate: 'media', code: 'EXPORT_MEDIA_MISSING', severity: 'error' }));+ await expect(getFableLoomExportDownload(loom.id, jobIds.at(-1))).rejects.toMatchObject({ code: 'EXPORT_NOT_READY' });+ });++ videoIt('uses an immutable record snapshot and refuses unsafe or missing live occupancy', async () => {+ const id = await begin({ episodeIds: ['episode-one'] });+ loom.name = 'Changed after admission';+ const output = await archive(id);+ expect(JSON.parse(output.files.get('manifest.json')).loom.name).toBe('The quiet gate');+ delete loom.episodes[0].nodes[0].playbackAssets.audioOccupancy['hold-record'];+ const unsafe = await terminal(await begin({ episodeIds: ['episode-one'], allowDraft: true }));+ expect(unsafe).toMatchObject({ status: 'failed', error: { code: 'EXPORT_OCCUPANCY_UNSAFE' } });+ });++ it('refuses unsupported and malformed options before creating jobs', async () => {+ for (const input of [{ includePlayer: true }, { includeCharacters: true }, { includeVoice: true },+ { mediaProfile: 'webm' }, { mediaProfile: { width: 100 } }, { allowDraft: 'yes' },+ { episodeIds: [] }, { episodeIds: ['episode-one', 'episode-one'] }, { destination: sourcePath }]) {+ const response = await request(app()).post('/api/fableloom/example-loom/export').send(input);+ expect(response.status, JSON.stringify(input)).toBe(400);+ }+ await expect(startFableLoomExport(loom.id, { episodeIds: ['unknown-episode'] })).rejects.toMatchObject({ code: 'EXPORT_EPISODES_INVALID' });+ });++ it('never follows a remote, traversing or escaping symlink source and keeps failures path-free', async () => {+ for (const bad of ['https://example.com/movie.mp4', '../private.mp4', sourcePath + '.mp4']) {+ loadHistory.mockResolvedValue([{ id: 'entry-record', filename: bad }]);+ const result = await terminal(await begin({ episodeIds: ['episode-one'] }));+ expect(result).toMatchObject({ status: 'failed', error: { code: 'EXPORT_MEDIA_PATH' } });+ expect(JSON.stringify(result)).not.toContain(bad);+ }+ const outside = join(PATHS.data, 'outside.png');+ await writeFile(outside, sourceImage);+ await symlink(outside, join(PATHS.images, 'escape.png'));+ loom.episodes[0].nodes[0].image = 'escape.png';+ const result = await terminal(await begin({ episodeIds: ['episode-one'], allowDraft: true }));+ expect(result).toMatchObject({ status: 'failed', error: { code: 'EXPORT_MEDIA_PATH' } });+ await rm(join(PATHS.images, 'escape.png'));+ });++ it('persists actionable interruption and expiry states without treating malformed state as missing', async () => {+ loom.episodes[0].nodes[0].image = null;+ const id = await begin();+ await terminal(id);+ await __testing.drain();+ const stored = JSON.parse(await readFile(recordFile(id), 'utf8'));+ stored.status = 'packing';+ await writeFile(recordFile(id), JSON.stringify(stored));+ await expect(getFableLoomExport('another-loom', id)).rejects.toMatchObject({ status: 404 });+ expect(JSON.parse(await readFile(recordFile(id), 'utf8')).status).toBe('packing');+ expect(await getFableLoomExport(loom.id, id)).toMatchObject({ status: 'failed', error: { code: 'EXPORT_INTERRUPTED' } });+ stored.status = 'failed';+ stored.createdAt = Date.now() - FABLELOOM_EXPORT_LIMITS.maxAgeMs - 1;+ await writeFile(recordFile(id), JSON.stringify(stored));+ await expect(getFableLoomExport(loom.id, id)).rejects.toMatchObject({ code: 'EXPORT_EXPIRED', status: 410 });+ await begin();+ await expect(stat(recordFile(id))).rejects.toMatchObject({ code: 'ENOENT' });+ const last = jobIds.at(-1);+ await terminal(last);+ await writeFile(recordFile(last), '{invalid-json');+ await expect(getFableLoomExport(loom.id, last)).rejects.toThrow();+ await expect(getFableLoomExport(loom.id, '../../secret')).rejects.toMatchObject({ status: 404 });+ });++ videoIt('bounds the live queue and retains a leased download through terminal-job eviction', async () => {+ const prepare = media.prepareFableLoomExportMedia;+ const gate = Promise.withResolvers();+ vi.spyOn(media, 'prepareFableLoomExportMedia').mockImplementation(async (...args) => {+ await gate.promise;+ return prepare(...args);+ });+ const ids = [];+ try {+ ids.push(...await Promise.all(Array.from({ length: 4 }, () => begin({ episodeIds: ['episode-one'] }))));+ await expect(begin()).rejects.toMatchObject({ code: 'EXPORT_QUEUE_FULL', status: 429 });+ } finally { gate.resolve(); await __testing.drain(); }+ for (const [index, id] of ids.entries()) {+ expect(await getFableLoomExport(loom.id, id)).toMatchObject({ status: 'completed' });+ const stored = JSON.parse(await readFile(recordFile(id), 'utf8'));+ stored.createdAt = Date.now() - (ids.length - index) * 1000;+ await writeFile(recordFile(id), JSON.stringify(stored));+ }+ const lease = await getFableLoomExportDownload(loom.id, ids[0]);+ try {+ loom.episodes[0].nodes[0].image = null;+ await terminal(await begin());+ expect((await readFile(lease.path)).subarray(0, 2).toString()).toBe('PK');+ await expect(getFableLoomExport(loom.id, ids[1])).rejects.toMatchObject({ status: 404 });+ } finally { lease.release(); lease.release(); }+ await terminal(await begin());+ await expect(getFableLoomExport(loom.id, ids[0])).rejects.toMatchObject({ status: 404 });+ });++ it('rejects an oversized source before decoding or allocating its bytes', async () => {+ const filename = join(PATHS.images, 'oversized.png');+ await writeFile(filename, '');+ await truncate(filename, FABLELOOM_EXPORT_LIMITS.maxAssetBytes + 1);+ loom.episodes[0].nodes[0].image = 'oversized.png';+ const result = await terminal(await begin({ episodeIds: ['episode-one'] }));+ expect(result).toMatchObject({ status: 'failed', error: { code: 'EXPORT_MEDIA_SIZE' } });+ await rm(filename);+ });+});diff --git a/server/services/fableLoom/index.js b/server/services/fableLoom/index.jsindex 77735d93c2..dd7b3dc027 100644--- a/server/services/fableLoom/index.js+++ b/server/services/fableLoom/index.js@@ -127,3 +127,4 @@ export {startFalVideoAutomation,} from './falVideoAutomation.js';export { applyEpisodeShots, runEpisodeShotAutopilot } from './shots.js';+export { startFableLoomExport, getFableLoomExport, getFableLoomExportDownload } from './exports.js';