Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: bind Runway reference tags to canon identities (#13105)
Adam Eivy7284acafdad7
diff --git a/server/lib/README.md b/server/lib/README.mdindex f3fa707383..4896acd150 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -365,6 +365,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `reactorStartingFrame.js` | Fits a reactor.inc `starting_frame` to the fast-h3 session canvas before it is uploaded. `readImageSize(path)` → `{width,height}` or `null` (never a guessed dimension), and `prepareReactorStartingFrame(sourceImagePath, requestedAspect, outputPath)` → `{ aspect, canvas, framePath, fittedPath }`: an absent `requestedAspect` DERIVES the canvas from the frame's own shape (`nearestReactorAspect`), then centre cover-crops the frame to that canvas's exact pixel size so reactor's own "fitted to the session canvas" step is a no-op. `fittedPath` is the temp file the caller deletes; `null` means the original was uploaded unchanged. A frame sharp cannot decode (or a resize that fails) passes through untouched on the default canvas rather than failing a render PortOS merely could not measure — reactor decodes more containers than sharp does. || `reactorVideoClip.js` | The reactor.inc `fast-h3` clip contract, dependency-free so the route schema, the FableLoom prompt compiler and the VideoGen picker all read one copy — `REACTOR_MAX_PROMPT_LENGTH` (800; the API rejects a longer prompt rather than truncating it), `REACTOR_MIN_CLIP_SECONDS` / `REACTOR_MAX_CLIP_SECONDS` (5.167–14.375, i.e. 124–345 frames at the `REACTOR_CLIP_FPS` 24 it delivers), `REACTOR_MAX_CLIP_ID_LENGTH`, and the picker list `REACTOR_CLIP_LENGTHS` + `REACTOR_DEFAULT_CLIP_LENGTH` + `reactorClipLengthLabel()` (the two exact endpoints plus every whole second between them, since the accepted range is continuous but starts and ends off a round number). Also the CANVAS contract `set_canvas` accepts: `REACTOR_CANVASES` (`16:9` 1344×768, `4:3` 1024×768, `1:1` 768×768, `9:16` 768×1344 — every one a 768px short edge, so the aspect string is the whole choice), `REACTOR_ASPECTS`, `REACTOR_DEFAULT_ASPECT`, `reactorCanvas(aspect)` and `nearestReactorAspect(width, height)` (closest canvas in log-ratio space; anything unmeasurable answers the default rather than deriving one). Mirrored to `client/src/lib/reactorVideoClip.js`, parity pinned by `reactorVideoClip.test.js`. || `runwayVideoModels.js` | Curated Runway VIDEO model catalog for the developer API's `/v1/image_to_video` and (Gen-4.5) `/v1/text_to_video` — `RUNWAY_VIDEO_MODELS` (Gen-4.5 default, Gen-4 Turbo, Kling 3.0 Standard / Pro, Kling 2.5 Turbo Pro, MiniMax H3 Max, WAN 3.0) with per-model prompt caps, duration rules, output sizes (`ratio` or `resolution`), end-frame support and list prices verified on `RUNWAY_VIDEO_PRICING_VERIFIED_AT`; `getRunwayVideoModel()`, `coerceRunwayVideoSeconds()` (round UP to a covering length), `resolveRunwayVideoResolution()`, `runwayVideoLengths()`, `buildRunwayVideoRequest()` (the endpoint and exact body, text-only when no frame and the model has a `textToVideo` rule, `runwayModelTakesText()`; refuses an uncurated model, a missing frame or an over-cap prompt, never cuts it), `runwayVideoClipQuote()` / `estimateRunwayVideoCostUsd()`, `describeRunwayVideoRate()`. Dependency-free; the client re-exports it. |+| `runwayImageModels.js` | Native Gen-4 image model schemas, supported canvases, prices, hosted disclosure, request validation and stable binding-derived reference tags; shared with browser model pickers. || `kimi.js` | Moonshot AI Kimi Code (`kimi`) provider helpers — id constants (`KIMI_CLI_ID`/`KIMI_TUI_ID`), `isKimiCommand`/`isKimiCliProvider`/`isKimiTuiProvider` predicates, `ensureKimiHeadlessArgs` (model flag only — non-interactive mode is implicit in `--prompt`, and kimi has no `--print`/`--afk` and refuses approval flags alongside `--prompt`) / `ensureKimiTuiArgs` (`--yolo`) argv builders (model selection uses the `KIMI_CONFIGURED_DEFAULT` sentinel in `providerModels.js` so PortOS omits `--model` like Grok/Antigravity), and `prepareKimiPrompt` (delivers the prompt as the `--prompt <value>` argv, not stdin). || `runners.js` | Image-runner family constants (`RUNNER_FAMILIES`, `loraCompatKey`), video LoRA families, MiniMax H3 runtime predicates, and the shared `AUDIO_TO_VIDEO_RUNTIMES` / `isAudioToVideoRuntime` capability gate (LTX plus H3 Ref2VA). Mirrored to `client/src/lib/runnerFamilies.js`. || `codexAccount.js` | The ChatGPT-subscription half of the Codex providers (#5589), pure. The fixed `codex app-server` invocation, the `account/*` method + notification names, `CODEX_ACCOUNT_STATUS` (`runtime-missing` / `unknown` / `signed-out` / `login-pending` / `ready` / `quota-exhausted` / `reauth-required`) and `CODEX_ERROR_CODES`, `isCodexSubscriptionProvider` (re-exported from `providerModels.js`; keyed on the `codex` COMMAND, so a renamed clone still resolves), the `normalizeCodexAccount` / `normalizeCodexRateLimits` / `normalizeCodexLoginStart` payload normalizers, `redactCodexPayload` (every log/error/response path runs through it, so "PortOS never writes down a Codex token" is one function rather than a rule per call site), and `deriveCodexAccountStatus`. Sentinels are load-bearing: `runtimeInstalled: null` = NOT PROBED, `accountFetched: false` ≠ signed out, `rateLimits: null` (failed fetch) ≠ a fetched window with nothing to report. The process half is `services/codexAppServer.js`. |diff --git a/server/services/fableLoom/visualConditioning.js b/server/services/fableLoom/visualConditioning.jsindex 8aa4f5df4a..89bfc9979a 100644--- a/server/services/fableLoom/visualConditioning.js+++ b/server/services/fableLoom/visualConditioning.js@@ -401,7 +401,14 @@ export async function compileFableLoomVisualRequest({universeId: universe.id,capability,bindings: manifestBindings(bindings),- assets: allocation.accepted.map(({ path: _path, ...asset }) => asset),+ assets: allocation.accepted.map(({ path: _path, ...asset }) => {+ if (capability.backend !== 'runway') return asset;+ const entry = asset.role.startsWith('character')+ ? bindings.boundCharacters.find(({ character }) => character.id === asset.bindingId)?.character+ : asset.role === 'environment' ? bindings.boundPlace+ : bindings.boundObjects.find((object) => object.id === asset.bindingId);+ return { ...asset, ...(entry?.name ? { label: entry.name } : {}) };+ }),adapters,omitted,warnings,diff --git a/server/services/fableLoom/visualConditioning.test.js b/server/services/fableLoom/visualConditioning.test.jsindex 4eb8cbd25a..9b0f72e4d5 100644--- a/server/services/fableLoom/visualConditioning.test.js+++ b/server/services/fableLoom/visualConditioning.test.js@@ -81,6 +81,7 @@ describe('FableLoom visual conditioning compiler', () => {authoredPrompt: 'Two rivals', ...deps(loom),});expect(result.visualConditioning.status).toBe('locked');+ expect(result.visualConditioning.assets.filter((asset) => asset.role === 'character-neutral').map((asset) => asset.label)).toEqual(['Aria', 'Bex']);expect(result.referenceImagePaths).toEqual(['/approved/opening.png', '/approved/aria-neutral.png', '/approved/bex-neutral.png']);});diff --git a/server/services/imageGen/runway.js b/server/services/imageGen/runway.jsindex ecad6fdaa1..42bd5a6b4d 100644--- a/server/services/imageGen/runway.js+++ b/server/services/imageGen/runway.js@@ -22,7 +22,7 @@ import { getSettings } from '../settings.js';import {awaitRunwayTask, cancelRunwayTask, createRunwayTaskEntry, resolveRunwayApiKey, submitRunwayTask,} from '../runwayApi.js';-import { IMAGE_GEN_MODE } from './modes.js';+import { IMAGE_GEN_MODE, describeFidelity } from './modes.js';import { resolveInputImages } from './inputImages.js';import { cloudPromptRequired } from './cloudProviderConfig.js';import { rejectDegenerateFrame } from './frameGuard.js';@@ -74,12 +74,17 @@ export async function checkConnection({ model = RUNWAY_IMAGE_DEFAULT_MODEL } = {return { connected: Boolean(apiKey), mode: MODE, model, ...(apiKey ? {} : { reason: noKeyReason }) };}-export function runwayImageSentPrompt({ prompt = '', negativePrompt, inputImages, visualConditioning, promptAdditions = [] }) {+export function runwayImageSentPrompt({ prompt = '', negativePrompt, inputImages, initImageStrength, visualConditioning, promptAdditions = [] }) {const references = runwayReferenceTags(inputImages.paths.map((path) => {const asset = visualConditioning?.assets?.find((item) => item.filename === basename(path));return { filename: basename(path), ...(asset || {}), path };}));- const roles = references.map((ref) => `@${ref.tag}: ${ref.role || (ref.path === inputImages.initPath ? 'source image to edit' : 'visual reference')}${ref.bindingId ? ` (${ref.bindingId})` : ''}.`).join('\n');+ const roles = references.map((ref) => {+ const role = ref.path === inputImages.initPath+ ? `source image to edit — ${describeFidelity(initImageStrength)}`+ : ref.role || 'visual reference';+ return `@${ref.tag}: ${role}${ref.label ? ` for ${ref.label}` : ''}.`;+ }).join('\n');const added = [promptAddition('reference-tags', 'Tagged Runway reference images', roles), promptAddition('avoid', 'Negative prompt, sent as "Avoid:"', negativePrompt ? `Avoid: ${negativePrompt}` : '')];const fullPrompt = [prompt, ...added.map((item) => item?.text).filter(Boolean)].join('\n\n');return { fullPrompt, references, record: sentPromptRecord({ sentPrompt: fullPrompt, carried: carriedPromptAdditions({ promptAdditions }), added }) };@@ -88,8 +93,10 @@ export function runwayImageSentPrompt({ prompt = '', negativePrompt, inputImagesasync function encodeInputImage(path) {const bytes = await readFile(path);const mime = detectImageFormat(bytes)?.mime;- if (!mime || bytes.length > 5 * 1024 * 1024) throw new ServerError('Runway references must be images no larger than 5 MB', { status: 400, code: 'RUNWAY_IMAGE_INPUT_INVALID' });- return `data:${mime};base64,${bytes.toString('base64')}`;+ const uri = mime ? `data:${mime};base64,${bytes.toString('base64')}` : null;+ // Runway's 5 MB ceiling applies to the encoded URI, including base64 overhead.+ if (!uri || Buffer.byteLength(uri) > 5_000_000) throw new ServerError('Runway reference data URIs must be images no larger than 5 MB after base64 encoding', { status: 400, code: 'RUNWAY_IMAGE_INPUT_INVALID' });+ return uri;}export const generateImage = (params) => preparations.run(params, generateImagePrepared);@@ -97,7 +104,7 @@ export const generateImage = (params) => preparations.run(params, generateImagePasync function generateImagePrepared({promptAdditions = [],model, resolution = '720p', ratio, prompt = '', negativePrompt, width, height, seed,- initImagePath, referenceImagePaths = [],+ initImagePath, initImageStrength, referenceImagePaths = [],visualConditioning = null,jobId: providedJobId = null,cleanC2PA = false,@@ -129,7 +136,7 @@ async function generateImagePrepared({const filename = `${jobId}.png`;const outputPath = join(PATHS.images, filename);if (inputImages.paths.length !== (initImagePath ? 1 : 0) + referenceImagePaths.length) throw new ServerError('Runway reference inputs are unavailable or exceed the three-image limit', { status: 400, code: 'RUNWAY_IMAGE_INPUT_INVALID' });- const { fullPrompt, references, record: sentRecord } = runwayImageSentPrompt({ prompt, negativePrompt, inputImages, visualConditioning, promptAdditions });+ const { fullPrompt, references, record: sentRecord } = runwayImageSentPrompt({ prompt, negativePrompt, inputImages, initImageStrength, visualConditioning, promptAdditions });const request = buildRunwayImageRequest({ modelId: family.id, prompt: fullPrompt, resolution, ratio, width, height, seed, referenceImages: references.map((ref) => ({ uri: '(validated input)', tag: ref.tag })) });const meta = {diff --git a/server/services/imageGen/runway.test.js b/server/services/imageGen/runway.test.jsindex 7d3c4b4456..e021db7e8b 100644--- a/server/services/imageGen/runway.test.js+++ b/server/services/imageGen/runway.test.js@@ -55,10 +55,12 @@ describe('Runway image workflow', () => {it('sends both bound character anchors with distinct stable tags and records basenames rather than paths', async () => {const paths = ['hero.png', 'rival.png'].map((name) => join(PATHS.images, name));await Promise.all(paths.map((path) => writeFile(path, output)));- const visualConditioning = { assets: [{ filename: 'hero.png', bindingId: 'char-a', role: 'character-neutral' }, { filename: 'rival.png', bindingId: 'char-b', role: 'character-neutral' }] };+ const visualConditioning = { assets: [{ filename: 'hero.png', bindingId: 'char-a', role: 'character-neutral', label: 'Hero' }, { filename: 'rival.png', bindingId: 'char-b', role: 'character-neutral', label: 'Rival' }] };const job = await runway.generateImage({ prompt: 'Two rivals', model: 'gen4_image_turbo', referenceImagePaths: paths, visualConditioning });expect(await terminal(job.jobId)).toMatchObject({ type: 'completed' });const refs = calls[0].body.referenceImages;+ expect(calls[0].body.promptText).toContain(`@${refs[0].tag}: character-neutral for Hero.`);+ expect(calls[0].body.promptText).toContain(`@${refs[1].tag}: character-neutral for Rival.`);expect(refs).toHaveLength(2);expect(refs[0].tag).not.toBe(refs[1].tag);for (const ref of refs) { expect(ref.uri).toMatch(/^data:image\/jpeg;base64,/); expect(calls[0].body.promptText).toContain(`@${ref.tag}`); }@@ -66,6 +68,13 @@ describe('Runway image workflow', () => {expect(sidecar.referenceTags.map((r) => r.filename)).toEqual(['hero.png', 'rival.png']);expect(JSON.stringify(sidecar)).not.toContain(root);});+ it('refuses an oversized encoded reference before any paid submission', async () => {+ const path = join(PATHS.images, 'large.png');+ await writeFile(path, Buffer.concat([output, Buffer.alloc(4_000_000)]));+ const job = await runway.generateImage({ prompt: 'A garden', referenceImagePaths: [path] });+ expect(await terminal(job.jobId)).toMatchObject({ type: 'failed', error: expect.stringContaining('after base64 encoding') });+ expect(calls).toEqual([]);+ });it('fails a moderated task once with its verbatim reason and permits the next asset', async () => {nextTask = { status: 'FAILED', failureCode: 'SAFETY.INPUT', failure: 'Input rejected by Runway moderation.' };const job = await runway.generateImage({ prompt: 'A garden' });