Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

Show exactly what each image and video render sent to the model

Claudeaedf87b6d203
diff --git a/AGENTS.md b/AGENTS.md
index 5503c3db88..f3de8e1f9d 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -101,7 +101,7 @@ Because both are off by default, **never treat "the password is set" as an avail
**Pattern for background pre-generation (e.g. caches):** boot-time init loads only what's on disk (zero LLM calls); the bulk/cold fill runs only from an explicit user-triggered endpoint behind a UI prompt that names the provider/model and lets the user change it or decline; incremental top-ups after the user has engaged (replenishing one item they consumed) may run silently. Reference: `server/services/meatspacePostDrillCache.js` (`initDrillCache` / `requestCacheFill`) and `client/src/components/meatspace/post/WordplayTrainer.jsx` (`CacheFillConsentModal`).
-**Never modify a model's input out of the user's sight.** What a user types or picks for a model (an LLM, image, video or music engine) is sent as written. Do not regex-rewrite, strip, truncate, or append or prepend text to it on the way out. Anything PortOS itself adds (a style or bible, a guard clause, a negative prompt, a reference) must appear where the user composes the request, so they can see the exact text and inputs that go out before they send them (the music video shot card's "Sent to the model" block, `client/src/components/musicVideo/SceneRenderInputs.jsx`, is the reference). When an input breaks a model's limit, refuse with the reason and let the user fix it; never quietly change it. Generated prompts are fixed at the source, in the instructions to the model that writes them, not by a cleanup pass before the send.
+**Never modify a model's input out of the user's sight.** What a user types or picks for a model (an LLM, image, video or music engine) is sent as written: no regex rewrite, strip or truncation on the way out. The editable text box holds only the user's own words; static shared material PortOS adds (a universe or project style, a production bible, a mood-board style, shot actions, preset negatives, an agent wrapper, a policy clause) never gets copied into it. Instead each addition is listed beside the box as a labeled chip whose text the user can open, and a "Show full prompt" view shows the exact final text and inputs, with Copy, built by the same server code that sends it. A render with no compose screen (autopilot, batch) records its exact sent prompt and additions so it can be inspected afterwards. When an input breaks a model's limit, show the count and refuse; never cut it. Generated prompts are fixed at the source, in the instructions to the model that writes them, not by a cleanup pass before the send. Plan and inventory: #11317.
## Architecture
diff --git a/client/src/components/media/MediaLightbox.jsx b/client/src/components/media/MediaLightbox.jsx
index cb83cca44b..27cfdcef22 100644
--- a/client/src/components/media/MediaLightbox.jsx
+++ b/client/src/components/media/MediaLightbox.jsx
@@ -20,6 +20,7 @@ import { copyToClipboard } from '../../lib/clipboard';
import { IMAGE_GEN_MODE } from '../../lib/imageGenBackends';
import { formatDateTime, formatDateNumeric, formatDurationMs } from '../../utils/formatters';
import AttributionList from './AttributionList';
+import SentPromptPanel from './SentPromptPanel';
// Intentionally NOT migrated to <ui/Modal> or <components/Drawer>. The
// prev/next buttons sit as viewport-edge siblings of the card (not children
@@ -734,6 +735,15 @@ function SettingsPane({
</div>
) : null}
+ {!item.compact && (
+ <SentPromptPanel
+ prompt={item.prompt === '(no prompt)' ? '' : item.prompt}
+ sentPrompt={item.sentPrompt}
+ sentNegativePrompt={item.sentNegativePrompt}
+ additions={item.promptAdditions}
+ />
+ )}
+
{onAnnotationChange && (
<div>
<div className="flex items-center justify-between mb-1">
diff --git a/client/src/components/media/SentPromptPanel.jsx b/client/src/components/media/SentPromptPanel.jsx
new file mode 100644
index 0000000000..992b9fbf26
--- /dev/null
+++ b/client/src/components/media/SentPromptPanel.jsx
@@ -0,0 +1,68 @@
+import { useState } from 'react';
+import { Copy } from 'lucide-react';
+import { copyToClipboard } from '../../lib/clipboard';
+
+/**
+ * What a model actually received, beside the prompt the user wrote (#11317):
+ * one chip per block PortOS added (tap to read it), and a "Show full prompt"
+ * view of the exact final text with Copy. Renders nothing for a record made
+ * before sent prompts were recorded.
+ */
+export default function SentPromptPanel({ prompt = '', sentPrompt = null, sentNegativePrompt = null, additions = [] }) {
+ const [openChip, setOpenChip] = useState(null);
+ const [showFull, setShowFull] = useState(false);
+ if (typeof sentPrompt !== 'string') return null;
+
+ const asWritten = !additions.length && !sentNegativePrompt && sentPrompt.trim() === (prompt || '').trim();
+ if (asWritten) return <p className="text-[11px] text-gray-500">Sent to the model exactly as written.</p>;
+
+ const fullText = sentNegativePrompt ? `${sentPrompt}\n\nNegative prompt: ${sentNegativePrompt}` : sentPrompt;
+ const opened = additions.find((addition) => addition.id === openChip);
+
+ return (
+ <div className="space-y-2">
+ {additions.length > 0 && (
+ <div>
+ <span className="block mb-1 text-gray-500 uppercase tracking-wide">Added to the prompt</span>
+ <div className="flex flex-wrap gap-1">
+ {additions.map((addition) => (
+ <button
+ key={addition.id}
+ type="button"
+ aria-pressed={openChip === addition.id}
+ onClick={() => setOpenChip(openChip === addition.id ? null : addition.id)}
+ className={`min-h-[32px] px-2 py-1 rounded-full border text-[11px] ${openChip === addition.id
+ ? 'border-port-accent text-white bg-port-accent/20'
+ : 'border-port-border text-gray-300 hover:text-white'}`}
+ >
+ + {addition.label}
+ </button>
+ ))}
+ </div>
+ {opened && <p className="mt-1 p-2 rounded bg-port-bg border border-port-border text-gray-300 whitespace-pre-wrap break-words">{opened.text}</p>}
+ </div>
+ )}
+ <div className="flex items-center justify-between">
+ <button
+ type="button"
+ aria-expanded={showFull}
+ onClick={() => setShowFull(!showFull)}
+ className="min-h-[44px] px-2 py-1 rounded text-port-accent hover:text-white"
+ >
+ {showFull ? 'Hide full prompt' : 'Show full prompt'}
+ </button>
+ {showFull && (
+ <button
+ type="button"
+ onClick={() => copyToClipboard(fullText, 'Full prompt copied')}
+ className="min-h-[44px] min-w-[44px] inline-flex items-center justify-center gap-1 px-2 py-1 rounded text-gray-400 hover:text-white hover:bg-port-border/50"
+ aria-label="Copy full prompt"
+ >
+ <Copy className="w-3 h-3" /> Copy
+ </button>
+ )}
+ </div>
+ {showFull && <pre className="p-2 rounded bg-port-bg border border-port-border text-gray-200 whitespace-pre-wrap break-words font-mono text-[11px]">{fullText}</pre>}
+ </div>
+ );
+}
diff --git a/client/src/components/media/SentPromptPanel.test.jsx b/client/src/components/media/SentPromptPanel.test.jsx
new file mode 100644
index 0000000000..f9d18fff6b
--- /dev/null
+++ b/client/src/components/media/SentPromptPanel.test.jsx
@@ -0,0 +1,32 @@
+import { describe, expect, it, vi } from 'vitest';
+import { fireEvent, render, screen } from '@testing-library/react';
+import SentPromptPanel from './SentPromptPanel';
+
+const copyToClipboard = vi.fn();
+vi.mock('../../lib/clipboard', () => ({ copyToClipboard: (...args) => copyToClipboard(...args) }));
+
+describe('SentPromptPanel', () => {
+ it('lists each addition as a chip and shows the exact sent text with Copy', () => {
+ render(
+ <SentPromptPanel
+ prompt="a red kite"
+ sentPrompt="muted earth tones. a red kite"
+ sentNegativePrompt="blur"
+ additions={[{ id: 'style-1', label: 'Universe style', text: 'muted earth tones' }]}
+ />,
+ );
+ fireEvent.click(screen.getByRole('button', { name: '+ Universe style' }));
+ expect(screen.getByText('muted earth tones')).toBeTruthy();
+
+ fireEvent.click(screen.getByRole('button', { name: 'Show full prompt' }));
+ fireEvent.click(screen.getByRole('button', { name: 'Copy full prompt' }));
+ expect(copyToClipboard).toHaveBeenCalledWith('muted earth tones. a red kite\n\nNegative prompt: blur', 'Full prompt copied');
+ });
+
+ it('says a prompt went out as written, and stays out of the way for older records', () => {
+ const { rerender, container } = render(<SentPromptPanel prompt="a red kite" sentPrompt="a red kite" />);
+ expect(screen.getByText('Sent to the model exactly as written.')).toBeTruthy();
+ rerender(<SentPromptPanel prompt="a red kite" />);
+ expect(container.textContent).toBe('');
+ });
+});
diff --git a/client/src/components/media/normalize.js b/client/src/components/media/normalize.js
index c564a04ade..76dbcf8880 100644
--- a/client/src/components/media/normalize.js
+++ b/client/src/components/media/normalize.js
@@ -39,6 +39,7 @@ export function normalizeImage(i) {
downloadUrl: originalUrl,
prompt: i.prompt || i.metadata?.prompt || '(no prompt)',
negativePrompt: i.negativePrompt || i.negative_prompt || null,
+ ...sentPromptFields(i),
modelId,
width: i.width,
height: i.height,
@@ -137,6 +138,7 @@ export function normalizeVideo(v) {
downloadUrl: `/data/videos/${v.filename}`,
prompt: v.prompt || '(no prompt)',
negativePrompt: v.negativePrompt || v.negative_prompt || null,
+ ...sentPromptFields(v),
modelId: v.modelId,
width: v.width,
height: v.height,
@@ -157,6 +159,16 @@ export function normalizeVideo(v) {
};
}
+// The exact text the model received and what PortOS added around the prompt
+// (#11317). Absent on records made before they were recorded.
+function sentPromptFields(raw) {
+ return {
+ sentPrompt: typeof raw.sentPrompt === 'string' ? raw.sentPrompt : null,
+ sentNegativePrompt: typeof raw.sentNegativePrompt === 'string' ? raw.sentNegativePrompt : null,
+ promptAdditions: Array.isArray(raw.promptAdditions) ? raw.promptAdditions.filter((item) => item?.label && typeof item.text === 'string') : [],
+ };
+}
+
// Resolve `loraFilenames` from a raw sidecar with four fallbacks:
// 1. raw.loraFilenames — new gen records, already basenames
// 2. raw.lora_filenames — same shape, snake_case writer
diff --git a/client/src/hooks/useImageGenForm.js b/client/src/hooks/useImageGenForm.js
index 439f726b03..0d06e7dc41 100644
--- a/client/src/hooks/useImageGenForm.js
+++ b/client/src/hooks/useImageGenForm.js
@@ -2,7 +2,7 @@ import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { getNavPageForPath } from '../../../server/lib/navManifest.js';
import { appendTriggerWords } from '../lib/loraTriggers';
import { composeStyledPrompt } from '../lib/composeStyledPrompt';
-import { universeStylePreset } from '../lib/universeStylePreset';
+import { universeStylePreset, withUniverseLabel } from '../lib/universeStylePreset';
import { RUNNER_FAMILIES, loraCompatKey, loraCompatKeysMatch, loraFamilyOf, usesDiffusersRunner } from '../lib/runnerFamilies';
import {
FAL_IMAGE_DEFAULT_MODEL,
@@ -483,7 +483,7 @@ export function useImageGenForm({ searchParams, setSearchParams, backend }) {
cloudModelId: isFalMode ? (falModel || savedFalModel || FAL_IMAGE_DEFAULT_MODEL) : null,
});
const activeStylePresets = useMemo(() => [
- selectedUniverse ? universeStylePreset(selectedUniverse) : null,
+ selectedUniverse ? withUniverseLabel(universeStylePreset(selectedUniverse)) : null,
stylePreset,
].filter(Boolean), [selectedUniverse, stylePreset]);
const styledPrompt = useMemo(
diff --git a/client/src/hooks/useImageGenPageRuntime.js b/client/src/hooks/useImageGenPageRuntime.js
index 5d659b4503..b66430df8a 100644
--- a/client/src/hooks/useImageGenPageRuntime.js
+++ b/client/src/hooks/useImageGenPageRuntime.js
@@ -1,6 +1,6 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { useNavigate, useSearchParams } from 'react-router';
-import { composeStyledPrompt } from '../lib/composeStyledPrompt';
+import { composeStyledPrompt, stylePromptAdditions } from '../lib/composeStyledPrompt';
import {
FAL_IMAGE_MODEL_IDS,
IMAGE_GEN_MODE,
@@ -33,6 +33,12 @@ import {
} from '../services/api';
import { musicVideoFilingFields } from '../lib/musicVideoFiling';
+// The style presets folded into the prompt, recorded beside it on the render (#11317).
+const promptAdditionsField = (presets) => {
+ const additions = stylePromptAdditions(presets);
+ return additions.length ? { promptAdditions: additions } : {};
+};
+
const IMAGE_QUEUE_EVENTS = ['media-jobs:changed'];
const STAGE_LABELS = {
@@ -292,7 +298,7 @@ export function useImageGenPageRuntime() {
cleanC2PA: fields.cleanC2PA,
denoise: fields.denoise,
};
- Object.assign(payload, musicVideoFilingFields(searchParams));
+ Object.assign(payload, musicVideoFilingFields(searchParams), promptAdditionsField(derived.activeStylePresets));
const hasInitImage = derived.i2iCapable && images.initImage.source != null;
const hasReferenceImages = derived.populatedRefs.length > 0;
if (hasInitImage || hasReferenceImages) {
@@ -416,7 +422,7 @@ export function useImageGenPageRuntime() {
denoise: fields.denoise,
};
if (fields.seed && Number(fields.seed) >= 0) payload.seed = Number(fields.seed);
- Object.assign(payload, musicVideoFilingFields(searchParams));
+ Object.assign(payload, musicVideoFilingFields(searchParams), promptAdditionsField(derived.activeStylePresets));
const data = await generateImage(payload, { silent: true });
setResult({
...data,
diff --git a/client/src/lib/README.md b/client/src/lib/README.md
index b38f8cac67..5fc938c6f6 100644
--- a/client/src/lib/README.md
+++ b/client/src/lib/README.md
@@ -52,7 +52,7 @@ grep -i "what you want to do" client/src/lib/README.md
| `shotContinuity.js` | The storyboards / episode-video stages' INLINE pre-render continuity warnings (#1468). `findAxisReversals` / `findShotTypeMonotony` are re-exported from `server/lib/editorial/shotContinuity.js`, the same detectors the authoritative `visual.shot-continuity` check runs; `sceneShotWarnings(scene, opts)` is the client-only composer (the server emits findings in a different shape). Pure/DOM-free. |
| `shotGrammar.js` | Storyboard shot-grammar display labels. `SHOT_TYPES` / `SCREEN_DIRECTIONS` are re-exported from `server/lib/shotGrammar.js`, so a hand-set value in the editor always passes `storyboardShotSchema`; `SHOT_TYPE_LABELS` / `SCREEN_DIRECTION_LABELS` are the client-only selects (#1468). |
| `untrustedContentSources.js` | Untrusted-content source display labels. `UNTRUSTED_CONTENT_SOURCES` / `PRIVATE_UNTRUSTED_CONTENT_SOURCES` are re-exported from `server/lib/untrustedContentSources.js`, so the Abuse Guard panel always offers every source the server screens; `UNTRUSTED_CONTENT_SOURCE_LABELS` is the client-only select (#7680). |
-| `universeStylePreset.js` | `universeStylePreset(universe, series)` re-exported from `server/lib/universeVisualStyle.js` — the style preset `composeStyledPrompt` layers on top, shared with the FableLoom visual-canon compiler so a series style override composes identically on both sides (#8442). |
+| `universeStylePreset.js` | `universeStylePreset(universe, series)` re-exported from `server/lib/universeVisualStyle.js`, plus `withUniverseLabel(preset)` (names it "Universe style" for the prompt-addition chips) — the style preset `composeStyledPrompt` layers on top, shared with the FableLoom visual-canon compiler so a series style override composes identically on both sides (#8442). |
| `universeRunTag.js` | `buildUniverseSectionRenderTag(universe, kindKey, entry)` — the durable `universeRun` job tag the canon-render call sites (UniverseCanonSection, NounsStage, Story Builder characters step) pass to `generateImage` so the server auto-files the render into the universe collection AND appends it to the entry's `imageRefs[]` (no client follow-up PATCH). |
| `autopilotMilestones.js` | Series-Autopilot milestone map. `buildAutopilotMilestones(plan, progress, {terminal})` folds the run's projected plan (the `start` frame) and its live progress snapshot (`progress` frames / the status route) into ordered rows with a `MILESTONE_STATUS` each — a furthest-index cursor, so a gate the run revisits can't un-finish the milestones after it; `summarizeAutopilotMilestones(rows)` rolls them into the header meter (a settled-but-unstepped milestone counts as complete); `describeAutopilotVerification(kind, verification)` renders what a gate actually validated — shared with the panel's `frameLabel`, since the milestone row and the activity log render the same telemetry; `isStoppedTerminal(terminal)` is the one definition of "the run stopped mid-plan"; `autopilotMarkerTerminal(status)` translates a persisted `autopilot.status` into the terminal frame type the fold reads, so a map rebuilt from the marker after a reload (#4140) flags the step a paused run stopped on. Also owns `AUTOPILOT_STEP_LABELS` / `autopilotStepLabel(kind)`, the one set of human labels for conductor step kinds (shared by the map and the panel's live status line). Used by `components/pipeline/AutopilotMilestones.jsx` + `AutopilotPanel.jsx`. |
| `beeperMessageBody.js` | Beeper message bodies: `decodeHtmlEntities` (the five predefined entities plus `&nbsp;`, using the shared `server/lib/xmlEntities.js` decoder), `looksLikeHtmlBody`, `parseMessageBody(body)` → blocks of marked spans (`p`/`div`/`br`/`blockquote`/`strong`/`b`/`em`/`i`/`a[href]`, `null` when the body carries none of them) and `messagePreviewText(body)` for the one-line rail row. Some networks deliver HTML (Discord, Matrix) and the mirror stores it verbatim, so the thread and the preview parse it into a data model and render React elements — never `dangerouslySetInnerHTML`; a non-http(s) `href` degrades to plain text. |
diff --git a/client/src/lib/composeStyledPrompt.js b/client/src/lib/composeStyledPrompt.js
index cf276e1e21..40f031dab2 100644
--- a/client/src/lib/composeStyledPrompt.js
+++ b/client/src/lib/composeStyledPrompt.js
@@ -1,10 +1,10 @@
// `composeStyledPrompt` (single preset or an array of presets) is a pure leaf
// re-exported from server/lib/composeStyledPrompt.js so the browser and the
// server compose style presets with identical logic (#8442).
-import { composeStyledPrompt } from '../../../server/lib/composeStyledPrompt.js';
+import { composeStyledPrompt, stylePromptAdditions } from '../../../server/lib/composeStyledPrompt.js';
import { universeStylePreset } from './universeStylePreset';
-export { composeStyledPrompt };
+export { composeStyledPrompt, stylePromptAdditions };
// Build the styled `{ prompt, negativePrompt }` for a single named canon subject
// (character / place / object) layered on the universe's style preset. This is
diff --git a/client/src/lib/universeStylePreset.js b/client/src/lib/universeStylePreset.js
index 3e51acee83..9dd5cc29da 100644
--- a/client/src/lib/universeStylePreset.js
+++ b/client/src/lib/universeStylePreset.js
@@ -5,3 +5,6 @@
// tokens. A drift between the two used to leave a series override applied in
// the preview but repeated/ignored in the actual render (#8442).
export { universeStylePreset } from '../../../server/lib/universeVisualStyle.js';
+
+// The universe preset as the "Universe style" chip beside the prompt box (#11317).
+export const withUniverseLabel = (preset) => (preset ? { ...preset, additionLabel: 'Universe style' } : null);
diff --git a/client/src/lib/videoGenSubmission.js b/client/src/lib/videoGenSubmission.js
index 42507336fe..cc415f158f 100644
--- a/client/src/lib/videoGenSubmission.js
+++ b/client/src/lib/videoGenSubmission.js
@@ -1,5 +1,5 @@
-import { composeStyledPrompt } from './composeStyledPrompt';
-import { universeStylePreset } from './universeStylePreset.js';
+import { composeStyledPrompt, stylePromptAdditions } from './composeStyledPrompt';
+import { universeStylePreset, withUniverseLabel } from './universeStylePreset.js';
import { isLtx2FamilyRuntime } from './runnerFamilies';
import { isDefaultI2vReferenceMode } from './videoReferenceModes';
import { clampImageEdge } from './imageGenResolutions';
@@ -20,7 +20,7 @@ import { isDefaultVideoStreamingMode } from './videoStreamingMode.js';
// The form owns state transitions and validation. This module owns the three
// request contracts the video route accepts: Grok, federated, and local.
const stylePresetsFor = (selectedUniverse, stylePreset) => [
- selectedUniverse ? universeStylePreset(selectedUniverse) : null,
+ selectedUniverse ? withUniverseLabel(universeStylePreset(selectedUniverse)) : null,
stylePreset,
].filter(Boolean);
@@ -38,12 +38,31 @@ export function envelopVideoPrompt(text, {
currentModel, negativePrompt, stylePreset, selectedUniverse, noMusic, disableAudio,
}) {
const composed = composeStyledPrompt(text, negativePrompt, stylePresetsFor(selectedUniverse, stylePreset));
- const effectiveDisableAudio = supportsVideoAudioControls(currentModel) && disableAudio;
- return (supportsVideoAudioPromptControls(currentModel) && noMusic && !effectiveDisableAudio && !/no music/i.test(composed.prompt))
- ? `${composed.prompt}\n\nno music, no soundtrack`
+ return addsNoMusic(composed.prompt, { currentModel, noMusic, disableAudio })
+ ? `${composed.prompt}\n\n${NO_MUSIC_LINE}`
: composed.prompt;
}
+const NO_MUSIC_LINE = 'no music, no soundtrack';
+
+// The "No music" toggle's line, added only on a model that takes audio prompt
+// controls while its audio is on, and only once.
+const addsNoMusic = (composedPrompt, { currentModel, noMusic, disableAudio }) => {
+ const effectiveDisableAudio = supportsVideoAudioControls(currentModel) && disableAudio;
+ return supportsVideoAudioPromptControls(currentModel) && noMusic && !effectiveDisableAudio && !/no music/i.test(composedPrompt);
+};
+
+/** Everything the form adds around the typed prompt, as labeled chips (#11317). */
+export function videoPromptAdditions({ prompt, negativePrompt, stylePreset, selectedUniverse, currentModel, noMusic, disableAudio, isLocal = true }) {
+ const presets = stylePresetsFor(selectedUniverse, stylePreset);
+ const composed = composeStyledPrompt(prompt, negativePrompt, presets);
+ return [
+ ...stylePromptAdditions(presets),
+ ...(isLocal && addsNoMusic(composed.prompt, { currentModel, noMusic, disableAudio })
+ ? [{ id: 'no-music', label: 'No music', text: NO_MUSIC_LINE }] : []),
+ ];
+}
+
// Hosted lanes (Grok, fal, reactor) take only text or image mode, and the
// source pair is meaningful only in image mode. '' is the wire value otherwise.
const hostedImageFields = (mode, sourceImageFile, sourceImageUpload) => {
@@ -296,14 +315,17 @@ export function buildVideoGenSubmission(args) {
});
const shared = { ...args, composed };
- if (isGrok) return grokVideoSubmission(shared);
- if (isFal) return falVideoSubmission(shared);
- if (isReactor) return reactorVideoSubmission(shared);
- if (remoteSubmissionFields) return remoteVideoSubmission(shared);
- return localVideoSubmission(args, {
- composed,
- withEnvelope,
- effectiveDisableAudio: supportsVideoAudioControls(currentModel) && disableAudio,
- beats: liveBeats(chainingActive, chunkPrompts, chunks, withEnvelope),
- });
+ const isLocal = !isGrok && !isFal && !isReactor && !remoteSubmissionFields;
+ const additions = videoPromptAdditions({ ...args, isLocal });
+ const payload = isGrok ? grokVideoSubmission(shared)
+ : isFal ? falVideoSubmission(shared)
+ : isReactor ? reactorVideoSubmission(shared)
+ : remoteSubmissionFields ? remoteVideoSubmission(shared)
+ : localVideoSubmission(args, {
+ composed,
+ withEnvelope,
+ effectiveDisableAudio: supportsVideoAudioControls(currentModel) && disableAudio,
+ beats: liveBeats(chainingActive, chunkPrompts, chunks, withEnvelope),
+ });
+ return additions.length ? { ...payload, promptAdditions: additions } : payload;
}
diff --git a/client/src/services/apiImageVideo.js b/client/src/services/apiImageVideo.js
index 9418617035..e7433533f1 100644
--- a/client/src/services/apiImageVideo.js
+++ b/client/src/services/apiImageVideo.js
@@ -332,6 +332,8 @@ export function buildFormData(fields) {
for (const [k, v] of Object.entries(fields)) {
if (v == null || v === '') continue;
if (v instanceof Blob) fd.append(k, v);
+ // A list of records (promptAdditions) travels as one JSON field the route parses.
+ else if (Array.isArray(v) && v.some((item) => item !== null && typeof item === 'object')) fd.append(k, JSON.stringify(v));
else if (Array.isArray(v)) v.forEach((item) => fd.append(k, String(item)));
else fd.append(k, String(v));
}
diff --git a/server/lib/README.md b/server/lib/README.md
index 00067f6839..9788cc8293 100644
--- a/server/lib/README.md
+++ b/server/lib/README.md
@@ -93,6 +93,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub
| `mindBundleCrypto.js` | Persistent Mind portable-bundle container (#7621) — `sealMindBundle` / `openMindBundle` / `readMindBundleHeader`, scrypt-from-passphrase + AES-256-GCM with the cleartext header as AAD. Passphrase-derived, never `PRIVACY_VAULT_KEY`: that key is machine-bound and a bundle sealed with it could not open on the destination install. Format names live in `mindBundleFormat.js`. |
| `mindBundleFormat.js` | Mind bundle format vocabulary, free of node builtins so the browser reads the same definitions (`client/src/lib/mindBundle.js`) — magic/versions/extension, scrypt parameters, export scopes (`PERSISTENT_MIND_BUNDLE_SCOPES`), import groups and their source scopes (`PERSISTENT_MIND_BUNDLE_GROUPS`, `MIND_BUNDLE_GROUP_SCOPES`, `MIND_BUNDLE_GROUP_CHOICES`), size/passphrase bounds, and the named refusal reasons (`MIND_BUNDLE_REFUSALS`, `mindBundleRefusal`). |
| `mindLocalContextClamp.js` | RAM/GPU safety ceilings for mind-driven local `numCtx` adjustments (CPU-only tiers, free-RAM + PortOS headroom). Pure. |
+| `modelInput.js` | What a model actually received beside what the user wrote (#11317): `promptAddition`, `sentPromptRecord`, `carriedPromptAdditions`, `agentInstructionsAddition`, and the `promptAdditionsSchema` request field. Records `sentPrompt` / `sentNegativePrompt` / `promptAdditions` on every render. |
| `motionSkills.js` | Optional motion-design agent skill packs (HyperFrames, Remotion, Claude Animation) — `MOTION_SKILL_PACKS` catalog shared by `npm run setup:motion` and the launch-video form, `detectMotionSkills()` finds installed skills under `~/.agents/skills` / `~/.claude/skills`. |
| `persistentMindPublic.js` | `publicPersistentMindState(state)` — the sole client/socket projection of durable mind state: exposes lifecycle fields, queue count, and the derived next wake time; drops queued message bodies/active wake payloads, and replaces free-form provider errors with coarse safe reasons. Non-user pause reasons pass through (credential-scrubbed) instead of reading "Paused by user", and `lastError` (coarse last-wake failure: killed/OOM, canceled, …) stays separate from `pauseReason`. |
| `persistentMindThinkingPresets.js` | Also owns bounded self-thinking request schemas, durable request normalization, and fixed switching limits. Saved named temporary thinking presets for the persistent mind (exact provider/model/effort alternates to its one home profile): default-empty durable shape, normalization that drops half-pinned entries, strict API schema, whole-list replace merge, preset lookup, strict accepted-selection schema/normalization, and route equality independent of labels. Storage only — saving or selecting a preset starts no inference. |
@@ -201,7 +202,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub
| `aspectRatio.js` | Describe a pixel canvas as the ratio phrase a prompt can use — `aspectRatioTerms(w, h)` (bounded best rational approximation, so 1096x1536 reads as 5:7 rather than 137:192) and `aspectRatioPhrase(w, h)` → `"5:7 portrait"` / `"16:9 landscape"` / `"1:1 square"`. Use it wherever a prompt has to state a framing the canvas already fixes, so the sentence and the canvas cannot drift. Pure. |
| `audioBeatDsp.js` | Shared beat/tempo DSP core (#9424) used by both `beatGrid.js` and `services/musicVideo/audioAnalysis.js`: `decodeAudioToPcm` (ffmpeg → mono f32 PCM, capped at `MAX_ANALYSIS_SEC`), Hann/FFT helpers, `onsetEnvelope` (spectral flux + RMS energy), `estimateTempo` (full-track autocorrelation over 70-180 BPM with a windowed, clustered fallback for tracks with a variable intro/outro), `fitBeats`, `pickDownbeats`. Exported as the `audioBeatDsp` namespace from the barrel. |
| `audioFingerprint.js` | Loudness fingerprint of a stretch of a song (#9266): `computeRmsEnvelope(path)` decodes a file once to a 100 Hz integer-dBFS RMS envelope on the song's own frame grid, `windowFingerprint(envelope, { startSec, endSec })` stores a window of it (base64 Int8Array, ≤3 KB for 30 s), and `compareWindowFingerprint(fp, envelope)` says whether a newer master still carries that window (Pearson ≥ 0.98 and mean \|ΔdB\| ≤ 1.5 above a −60 dB codec-noise floor). Lets a lip-sync take survive a re-master that changed audio outside its window. `envelopeOffsetSec(reference, candidate)` cross-correlates two envelopes and returns `{ offsetSec, correlation }` (where the reference's sound starts inside the candidate, refined below a frame) or null on a weak match — used to line a padded provider clip up with its source window (#11210). |
-| `composeStyledPrompt.js` | Compose user prompt + negative with an optional style preset, or an array of presets layered in order. Re-exported (unmodified) by `client/src/lib/composeStyledPrompt.js`. |
+| `composeStyledPrompt.js` | Compose user prompt + negative with an optional style preset, or an array of presets layered in order. `stylePromptAdditions(preset)` lists what a compose folds in as labeled additions (#11317). Re-exported (unmodified) by `client/src/lib/composeStyledPrompt.js`. |
| `scriptVideoCompiler.js` | Compiles a scene script + bible into beat-level continuous-video clip specs (duration/frame-grid snapping, byte-stable descriptor injection, chain-length fresh-cut rule). |
| `creativeDirectorPresets.js` | Locked-at-creation aspect ratio + quality presets for the Creative Director. |
| `creativeLatitude.js` | The IP-latitude clause every creative LLM request carries (`withCreativeLatitude`, `CREATIVE_LATITUDE_TOKENS`), plus the one creative-vs-operational table both stamp keys are matched against (`isCreativeStage` for stage names, `isCreativeRunSource` for run `source` tags). |
diff --git a/server/lib/composeStyledPrompt.js b/server/lib/composeStyledPrompt.js
index 57a081f170..60b20a47a1 100644
--- a/server/lib/composeStyledPrompt.js
+++ b/server/lib/composeStyledPrompt.js
@@ -30,3 +30,21 @@ export function composeStyledPrompt(userPrompt, userNegative, preset) {
negativePrompt: [negative, styleNeg].filter(Boolean).join(', '),
};
}
+
+/**
+ * The labeled additions `composeStyledPrompt` folds around the user's text, for
+ * the "added to your prompt" chips and the sent-prompt record (#11317). A preset
+ * names itself with `additionLabel` (the universe style) or its picker `label`.
+ */
+export function stylePromptAdditions(preset) {
+ const presets = (Array.isArray(preset) ? preset : [preset]).filter(Boolean);
+ const additions = presets
+ .filter((item) => (item.prompt || '').trim())
+ .map((item, index) => ({
+ id: `style-${index + 1}`,
+ label: item.additionLabel || (item.label ? `Style preset: ${item.label}` : 'Style'),
+ text: item.prompt.trim(),
+ }));
+ const styleNeg = presets.map((item) => (item.negativePrompt || '').trim()).filter(Boolean).join(', ');
+ return styleNeg ? [...additions, { id: 'style-negatives', label: 'Style negatives', text: styleNeg }] : additions;
+}
diff --git a/server/lib/composeStyledPrompt.test.js b/server/lib/composeStyledPrompt.test.js
index 68c8006a9a..f37ee891a6 100644
--- a/server/lib/composeStyledPrompt.test.js
+++ b/server/lib/composeStyledPrompt.test.js
@@ -1,5 +1,5 @@
import { describe, it, expect } from 'vitest';
-import { composeStyledPrompt } from './composeStyledPrompt.js';
+import { composeStyledPrompt, stylePromptAdditions } from './composeStyledPrompt.js';
describe('composeStyledPrompt', () => {
it('returns the bare prompt + negative when no preset is supplied', () => {
@@ -41,3 +41,16 @@ describe('composeStyledPrompt', () => {
});
});
});
+
+describe('stylePromptAdditions', () => {
+ it('lists each preset the compose folds in, and their negatives once', () => {
+ const universe = { additionLabel: 'Universe style', prompt: 'muted earth tones', negativePrompt: 'neon' };
+ const preset = { label: 'Film noir', prompt: 'cinematic noir', negativePrompt: 'color' };
+ expect(stylePromptAdditions([universe, preset, null])).toEqual([
+ { id: 'style-1', label: 'Universe style', text: 'muted earth tones' },
+ { id: 'style-2', label: 'Style preset: Film noir', text: 'cinematic noir' },
+ { id: 'style-negatives', label: 'Style negatives', text: 'neon, color' },
+ ]);
+ expect(stylePromptAdditions(null)).toEqual([]);
+ });
+});
diff --git a/server/lib/falVideoModels.js b/server/lib/falVideoModels.js
index f310838ea7..7beec72d51 100644
--- a/server/lib/falVideoModels.js
+++ b/server/lib/falVideoModels.js
@@ -403,27 +403,6 @@ export function describeFalVideoRate(modelId, resolution) {
* master soundtrack (Music Video) or silent-by-default clips; a caller asks for
* provider audio explicitly.
*/
-// The last clause, sentence or line boundary at or before `max`, else a hard cut.
-const clipAtBoundary = (text, max) => {
- if (text.length <= max) return text;
- const head = text.slice(0, max);
- const cut = Math.max(head.lastIndexOf('\n'), head.lastIndexOf(', '), head.lastIndexOf('. '), head.lastIndexOf('; '));
- return (cut > max / 2 ? head.slice(0, cut) : head).trimEnd();
-};
-
-/**
- * `text` plus its folded `Avoid:` clause within `max` characters. The Avoid
- * clause keeps up to a quarter of the budget; the authored text is cut at a
- * clause boundary. Unchanged when it already fits.
- */
-function fitFalPrompt(text, avoid, max) {
- const whole = avoid ? `${text}\nAvoid: ${avoid}` : text;
- if (whole.length <= max) return whole;
- const avoidPart = avoid ? `\nAvoid: ${clipAtBoundary(avoid, Math.max(0, Math.floor(max / 4) - 8))}` : '';
- const keptAvoid = avoidPart.trim() === 'Avoid:' ? '' : avoidPart;
- return `${clipAtBoundary(text, max - keptAvoid.length)}${keptAvoid}`;
-}
-
export function buildFalVideoRequest({
modelId, prompt = '', negativePrompt = '', seconds, audioSec, aspectRatio, width, height, resolution,
imageUrl = null, endImageUrl = null, referenceImageUrls = [], audioUrl = null, referenceAudioUrls = [],
@@ -436,17 +415,17 @@ export function buildFalVideoRequest({
const text = typeof prompt === 'string' ? prompt.trim() : '';
const avoid = typeof negativePrompt === 'string' ? negativePrompt.trim() : '';
- let promptClipped = null;
if (model.prompt !== 'none') {
if (model.prompt === 'required' && !text) throw refuse(`${model.label} needs a prompt`);
// No native field: fold the negative prompt into the prompt as an "Avoid:"
// clause — the same fallback grok.js uses for a provider without one.
const folded = model.negativePrompt === 'fold' && avoid ? `${text}\nAvoid: ${avoid}` : text;
- // A prompt over the model's cap is refused by fal only after the job is
- // queued, so it is shortened here: the authored text first, then the Avoid clause.
- const fitted = model.promptMaxChars ? fitFalPrompt(text, model.negativePrompt === 'fold' ? avoid : '', model.promptMaxChars) : folded;
- if (fitted.length < folded.length) promptClipped = { from: folded.length, to: fitted.length, max: model.promptMaxChars };
- if (fitted) body.prompt = fitted;
+ // fal refuses an over-cap prompt only after the job is queued, so it is refused
+ // here, before anything is paid for. It is never shortened (AGENTS.md, #11317).
+ if (model.promptMaxChars && folded.length > model.promptMaxChars) {
+ throw refuse(`The prompt is ${folded.length.toLocaleString('en-US')} characters${folded === text ? '' : ' with its "Avoid:" clause'}; ${model.label} takes ${model.promptMaxChars.toLocaleString('en-US')}. Shorten it to render.`);
+ }
+ if (folded) body.prompt = folded;
if (model.negativePrompt === 'native' && avoid) body.negative_prompt = avoid;
}
@@ -493,7 +472,6 @@ export function buildFalVideoRequest({
return {
body,
model,
- promptClipped,
seconds: coercedSec,
resolution: res,
generateAudio: wantsAudio,
diff --git a/server/lib/falVideoModels.test.js b/server/lib/falVideoModels.test.js
index 46e9975935..982b980564 100644
--- a/server/lib/falVideoModels.test.js
+++ b/server/lib/falVideoModels.test.js
@@ -164,17 +164,15 @@ describe('per-clip quotes — what a shot actually bills', () => {
// Regression: a 2,000+ character prompt reached Hailuo-02, which fal refused with a 422 only after the job was queued.
describe('buildFalVideoRequest — prompt length caps', () => {
- it('cuts a prompt over the model limit at a clause boundary and keeps a short Avoid clause', () => {
- const prompt = Array.from({ length: 300 }, (_, i) => `clause ${i}`).join(', ');
- const { body, promptClipped } = build('fal-ai/minimax/hailuo-02/standard/image-to-video', { prompt, negativePrompt: 'blur' });
- expect(body.prompt.length).toBeLessThanOrEqual(2000);
- expect(body.prompt).toMatch(/clause \d+\nAvoid: blur$/);
- expect(body.prompt.startsWith('clause 0, clause 1,')).toBe(true);
- expect(promptClipped).toEqual({ from: prompt.length + '\nAvoid: blur'.length, to: body.prompt.length, max: 2000 });
+ // AGENTS.md: a model's input is never cut out of sight; over the cap is refused before fal is paid.
+ it('refuses a prompt over the model limit, counting its Avoid clause', () => {
+ const prompt = 'x'.repeat(1995);
+ expect(() => build('fal-ai/minimax/hailuo-02/standard/image-to-video', { prompt, negativePrompt: 'blur' }))
+ .toThrow(/2,007 characters with its "Avoid:" clause; .* takes 2,000/);
});
it('leaves a prompt within the limit, and a model without a known limit, untouched', () => {
- expect(build('fal-ai/minimax/hailuo-02/standard/image-to-video').promptClipped).toBeNull();
+ expect(build('fal-ai/minimax/hailuo-02/standard/image-to-video', { prompt: 'a', negativePrompt: 'blur' }).body.prompt).toBe('a\nAvoid: blur');
const long = 'x '.repeat(1500).trim();
expect(build('fal-ai/veo3.1/fast/image-to-video', { prompt: long }).body.prompt).toBe(long);
});
diff --git a/server/lib/index.js b/server/lib/index.js
index cbde30e831..a5a5402cf4 100644
--- a/server/lib/index.js
+++ b/server/lib/index.js
@@ -220,6 +220,7 @@ export * from './installLogger.js';
export * from './kimi.js';
export * from './mediaModelBuckets.js';
export * from './mediaModels.js';
+export * from './modelInput.js';
export * from './modelInventory.js';
export * from './meetingUrl.js';
export * from './minimaxH3Memory.js';
diff --git a/server/lib/modelInput.js b/server/lib/modelInput.js
new file mode 100644
index 0000000000..d1dae2258d
--- /dev/null
+++ b/server/lib/modelInput.js
@@ -0,0 +1,62 @@
+import { z } from 'zod';
+
+/**
+ * What a model actually received, beside what the user wrote (AGENTS.md "Never
+ * modify a model's input out of the user's sight", #11317).
+ *
+ * The user's text box keeps only their own words. Everything PortOS adds around
+ * them (a style, a bible, a negative folded in as "Avoid:", an agent's tool
+ * instructions, LoRA trigger words) is a labeled addition the UI lists as a chip,
+ * and `sentPrompt` is the exact final text, so a render can be inspected after
+ * the fact even when nothing showed it before the send (autopilot, batch).
+ */
+
+const ADDITION_TEXT_MAX = 20000;
+
+/** One labeled block PortOS adds to a model's input, or null when it adds nothing. */
+export function promptAddition(id, label, text) {
+ const trimmed = typeof text === 'string' ? text.trim() : '';
+ return trimmed ? { id, label, text: trimmed.slice(0, ADDITION_TEXT_MAX) } : null;
+}
+
+/** The additions a request already carries (from the composer that built it). */
+export const carriedPromptAdditions = (params) => (Array.isArray(params?.promptAdditions)
+ ? params.promptAdditions.filter((item) => item && typeof item.text === 'string' && item.text.trim())
+ : []);
+
+/**
+ * The record fields for a render: the exact text sent, the negative prompt when
+ * the engine takes one natively, and every addition (carried ones first, then
+ * the backend's own). An addition with no text is dropped.
+ */
+export function sentPromptRecord({ sentPrompt, sentNegativePrompt = null, carried = [], added = [] }) {
+ const additions = [...carried, ...added].filter(Boolean);
+ return {
+ sentPrompt: typeof sentPrompt === 'string' ? sentPrompt : '',
+ ...(typeof sentNegativePrompt === 'string' && sentNegativePrompt.trim() ? { sentNegativePrompt: sentNegativePrompt.trim() } : {}),
+ ...(additions.length ? { promptAdditions: additions } : {}),
+ };
+}
+
+/**
+ * Request schema for the additions a client composer attaches (style presets, universe
+ * style). A multipart form sends the list as one JSON string field.
+ */
+export const promptAdditionsSchema = z.preprocess((value) => {
+ if (typeof value !== 'string') return value;
+ try { return JSON.parse(value); } catch { return value; }
+}, z.array(z.object({
+ id: z.string().min(1).max(64),
+ label: z.string().min(1).max(120),
+ text: z.string().min(1).max(ADDITION_TEXT_MAX),
+})).max(32).optional());
+
+/**
+ * An agent backend's instructions around the prompt (Codex, Grok, Antigravity), as one
+ * addition: the full message with the prompt itself shown as "[prompt]".
+ */
+export function agentInstructionsAddition(label, fullPrompt, prompt) {
+ const own = typeof prompt === 'string' ? prompt.trim() : '';
+ const text = own ? String(fullPrompt).split(own).join('[prompt]') : String(fullPrompt);
+ return promptAddition('agent-instructions', label, text);
+}
diff --git a/server/lib/modelInput.test.js b/server/lib/modelInput.test.js
new file mode 100644
index 0000000000..85acb46ce1
--- /dev/null
+++ b/server/lib/modelInput.test.js
@@ -0,0 +1,40 @@
+import { describe, expect, it } from 'vitest';
+import { agentInstructionsAddition, promptAdditionsSchema, sentPromptRecord } from './modelInput.js';
+
+describe('promptAdditionsSchema', () => {
+ // A multipart image request (an uploaded reference) carries the list as one JSON string.
+ it('accepts the list as an array or as a multipart JSON string', () => {
+ const list = [{ id: 'style', label: 'Universe style', text: 'muted earth tones' }];
+ expect(promptAdditionsSchema.parse(list)).toEqual(list);
+ expect(promptAdditionsSchema.parse(JSON.stringify(list))).toEqual(list);
+ expect(promptAdditionsSchema.parse(undefined)).toBeUndefined();
+ });
+
+ it('refuses a malformed string rather than dropping it', () => {
+ expect(() => promptAdditionsSchema.parse('not json')).toThrow();
+ expect(() => promptAdditionsSchema.parse([{ id: 'x', label: 'X', text: '' }])).toThrow();
+ });
+});
+
+describe('sentPromptRecord', () => {
+ it('keeps carried additions ahead of the backend ones and drops empty ones', () => {
+ const carried = [{ id: 'style', label: 'Style preset', text: 'film grain' }];
+ const record = sentPromptRecord({
+ sentPrompt: 'a fox, film grain',
+ sentNegativePrompt: ' ',
+ carried,
+ added: [null, { id: 'lora-triggers', label: 'LoRA trigger words', text: 'fxstyle' }],
+ });
+ expect(record).toEqual({
+ sentPrompt: 'a fox, film grain',
+ promptAdditions: [carried[0], { id: 'lora-triggers', label: 'LoRA trigger words', text: 'fxstyle' }],
+ });
+ });
+});
+
+describe('agentInstructionsAddition', () => {
+ it('shows the wrapper with the user prompt marked as [prompt]', () => {
+ const addition = agentInstructionsAddition('Codex image tool instructions', 'Generate an image of: a red kite. Save it as PNG.', 'a red kite');
+ expect(addition.text).toBe('Generate an image of: [prompt]. Save it as PNG.');
+ });
+});
diff --git a/server/routes/imageGen.js b/server/routes/imageGen.js
index cda3be9056..f8affe6c98 100644
--- a/server/routes/imageGen.js
+++ b/server/routes/imageGen.js
@@ -55,6 +55,7 @@ import {
} from '../services/fableLoom/visualConditioning.js';
import { loraCompatKey } from '../lib/runners.js';
import { EFFORT_LEVELS } from '../lib/providerModels.js';
+import { promptAdditionsSchema } from '../lib/modelInput.js';
const router = Router();
@@ -105,6 +106,7 @@ const galleryImageFilenameSchema = (label) => z.string().max(256)
router.get('/style-presets', (_req, res) => res.json(STYLE_PRESETS));
const generateSchema = z.object({
+ promptAdditions: promptAdditionsSchema,
// Empty prompt allowed — i2i / edit / unconditional generation don't require
// one. The multipart FormData builder drops empty-string fields, so an empty
// prompt arrives as `undefined`; default it to '' rather than rejecting.
diff --git a/server/routes/videoGen.js b/server/routes/videoGen.js
index bbaef650c2..310ca6ef5f 100644
--- a/server/routes/videoGen.js
+++ b/server/routes/videoGen.js
@@ -92,6 +92,7 @@ import {
} from '../services/videoGen/runtimeInstaller.js';
import { detectSystemCapabilities, withHardwareCompatibility } from '../lib/systemCapabilities.js';
import { isDisplaySleepEnabled } from '../services/videoGen/displayPower.js';
+import { promptAdditionsSchema } from '../lib/modelInput.js';
const router = Router();
@@ -279,6 +280,7 @@ export const LOCAL_ONLY_VIDEO_PARAMS = Object.freeze({
});
const generateBodySchema = z.object({
+ promptAdditions: promptAdditionsSchema,
// Render backend: the local runtimes (default), the Grok Build CLI's
// image-first image_to_video flow (#2859 phase 2), fal.ai's queue REST API
// (#6213), or reactor.inc's fast-h3 API (#6214). Grok, fal, and reactor all
diff --git a/server/services/imageGen/agy.js b/server/services/imageGen/agy.js
index a977a60460..cdaa66687d 100644
--- a/server/services/imageGen/agy.js
+++ b/server/services/imageGen/agy.js
@@ -49,6 +49,7 @@ import {
import { resolveInputImages } from './inputImages.js';
import { cloudPromptRequired } from './cloudProviderConfig.js';
import { withSpawnCwdEnv } from '../../lib/spawnCwd.js';
+import { agentInstructionsAddition, carriedPromptAdditions, sentPromptRecord } from '../../lib/modelInput.js';
const AGY_TIMEOUT_MS = (() => {
const n = Number(process.env.AGY_IMAGEGEN_TIMEOUT_MS);
@@ -260,6 +261,7 @@ async function resolveAgyDriverModel(model, shippedDefault) {
}
export async function generateImage({
+ promptAdditions = [],
agyPath,
model,
modelIsShippedDefault = false,
@@ -324,6 +326,8 @@ export async function generateImage({
// provenance names the model that actually produced the pixels (#3231).
imageModel: AGY_IMAGEGEN_IMAGE_MODEL,
...(visualConditioning ? { visualConditioning } : {}),
+ ...sentPromptRecord({ sentPrompt: fullPrompt, carried: carriedPromptAdditions({ promptAdditions }),
+ added: [agentInstructionsAddition('Antigravity image tool instructions', fullPrompt, prompt)] }),
createdAt: new Date().toISOString(),
};
// The scratch dir is the last fallible step before the job exists, so a
diff --git a/server/services/imageGen/codex.js b/server/services/imageGen/codex.js
index 4620d8c1fb..63fc47315c 100644
--- a/server/services/imageGen/codex.js
+++ b/server/services/imageGen/codex.js
@@ -55,6 +55,7 @@ import { conformImageAspect } from './aspectConform.js';
import { aspectRatioPhrase } from '../../lib/aspectRatio.js';
import { resolveInputImages } from './inputImages.js';
import { cloudPromptRequired } from './cloudProviderConfig.js';
+import { agentInstructionsAddition, carriedPromptAdditions, sentPromptRecord } from '../../lib/modelInput.js';
// 20 minutes — built-in `image_gen` typically returns in 30–90s, but with the
// parallel codex lane several renders share OpenAI throughput and a single
@@ -183,6 +184,7 @@ export function buildCodexAttachmentPrefix({ initPath, referenceCount, initImage
// of them ride one flag) and the prompt is reshaped so the `$imagegen` skill
// feeds the attachments to `image_gen` as references.
export async function generateImage({
+ promptAdditions = [],
codexPath, model, effort, prompt = '', width, height, negativePrompt,
initImagePath, initImageStrength, referenceImagePaths = [],
visualConditioning = null,
@@ -285,6 +287,8 @@ export async function generateImage({
width: width ? Number(width) : null, height: height ? Number(height) : null,
filename, mode: IMAGE_GEN_MODE.CODEX, model: effectiveModel,
...(visualConditioning ? { visualConditioning } : {}),
+ ...sentPromptRecord({ sentPrompt: fullPrompt, carried: carriedPromptAdditions({ promptAdditions }),
+ added: [agentInstructionsAddition('Codex image tool instructions', fullPrompt, prompt)] }),
createdAt: new Date().toISOString(),
};
const job = { ...meta, clients: [], status: 'running', renderStartedAtMs };
diff --git a/server/services/imageGen/fal.js b/server/services/imageGen/fal.js
index f5e398ea27..941fd28765 100644
--- a/server/services/imageGen/fal.js
+++ b/server/services/imageGen/fal.js
@@ -55,6 +55,7 @@ import { IMAGE_GEN_MODE, describeFidelity, visualReferenceRole } from './modes.j
import { resolveInputImages } from './inputImages.js';
import { cloudPromptRequired } from './cloudProviderConfig.js';
import { rejectDegenerateFrame } from './frameGuard.js';
+import { carriedPromptAdditions, promptAddition, sentPromptRecord } from '../../lib/modelInput.js';
const MODE = IMAGE_GEN_MODE.FAL;
@@ -195,6 +196,7 @@ const firstImageUrl = (result) => {
};
export async function generateImage({
+ promptAdditions = [],
model, prompt = '', negativePrompt, width, height, seed,
initImagePath, initImageStrength, referenceImagePaths = [],
visualConditioning = null,
@@ -257,6 +259,13 @@ export async function generateImage({
modelFamily: family.id,
...(inputImages.paths.length ? { inputImageCount: inputImages.paths.length } : {}),
...(visualConditioning ? { visualConditioning } : {}),
+ // The text runFal sends: the image-role prose plus the negative prompt folded in as "Avoid:".
+ ...sentPromptRecord({
+ sentPrompt: buildFalImageRequest({ modelId: family.textEndpoint, prompt: fullPrompt, negativePrompt }).body.prompt,
+ carried: carriedPromptAdditions({ promptAdditions }),
+ added: [promptAddition('image-roles', 'How to read the attached images', fullPrompt.slice(prompt.trim().length)),
+ promptAddition('avoid', 'Negative prompt, sent as "Avoid:"', negativePrompt)],
+ }),
createdAt: new Date().toISOString(),
};
const job = { ...meta, clients: [], status: 'running', renderStartedAtMs };
diff --git a/server/services/imageGen/grok.js b/server/services/imageGen/grok.js
index 6cc57475ea..3fefc6ad9c 100644
--- a/server/services/imageGen/grok.js
+++ b/server/services/imageGen/grok.js
@@ -56,6 +56,7 @@ import { GROK_ASPECT_RATIOS } from '../../lib/imageGenCapabilities.js';
import { resolveInputImages } from './inputImages.js';
import { cloudPromptRequired } from './cloudProviderConfig.js';
import { withSpawnCwdEnv } from '../../lib/spawnCwd.js';
+import { agentInstructionsAddition, carriedPromptAdditions, sentPromptRecord } from '../../lib/modelInput.js';
// 20 minutes — grok's image_gen typically returns in well under a minute, but
// the agent turn wrapping it (skill load, tool call, file write) has no
@@ -211,6 +212,7 @@ export function buildGrokPrompt({
// `initImageStrength` maps to a fidelity phrase (grok's image_edit has no
// numeric denoise knob, same constraint as codex).
export async function generateImage({
+ promptAdditions = [],
grokPath, aspectRatio, prompt = '', width, height, negativePrompt,
initImagePath, initImageStrength, referenceImagePaths = [],
visualConditioning = null,
@@ -273,6 +275,8 @@ export async function generateImage({
width: width ? Number(width) : null, height: height ? Number(height) : null,
filename, mode: IMAGE_GEN_MODE.GROK,
...(visualConditioning ? { visualConditioning } : {}),
+ ...sentPromptRecord({ sentPrompt: fullPrompt, carried: carriedPromptAdditions({ promptAdditions }),
+ added: [agentInstructionsAddition('Grok image tool instructions', fullPrompt, prompt)] }),
...(effectiveRatio ? { aspectRatio: effectiveRatio } : {}),
createdAt: new Date().toISOString(),
};
diff --git a/server/services/imageGen/local.js b/server/services/imageGen/local.js
index 81bb5787ab..e84d7c7b63 100644
--- a/server/services/imageGen/local.js
+++ b/server/services/imageGen/local.js
@@ -49,6 +49,7 @@ import { weaveLoraTriggers } from '../../lib/loraTriggers.js';
import { provenanceForRender } from '../../lib/assetProvenance.js';
import { createKeyCachedQueue } from '../../lib/createKeyCachedQueue.js';
import { readTriggerWordsByFilename, readLoraLicensesByFilename } from '../loras.js';
+import { carriedPromptAdditions, promptAddition, sentPromptRecord } from '../../lib/modelInput.js';
// Read the registry lazily — callers below hit getImageModels() at request
// time. A prior `IMAGE_MODELS = Object.fromEntries(getImageModels()...)`
@@ -343,6 +344,8 @@ export function buildSidecarMeta({
// License map keyed by LoRA basename, read from sidecars by generateImage
// so this stays pure. Missing entries stamp license: null (unknown).
loraLicenses = null,
+ // The labeled additions the composer put around the user's text (lib/modelInput.js).
+ promptAdditions = [],
initImagePath = null,
initImageStrength = null,
referenceImagePaths = [],
@@ -466,6 +469,10 @@ export function buildSidecarMeta({
meta.renderPrompt = renderPrompt;
meta.addedTriggerWords = addedTriggerWords;
}
+ Object.assign(meta, sentPromptRecord({
+ sentPrompt: renderPrompt, sentNegativePrompt: negativePrompt, carried: carriedPromptAdditions({ promptAdditions }),
+ added: [promptAddition('lora-triggers', 'LoRA trigger words', addedTriggerWords.join(', '))],
+ }));
// Stamp the licenses known at THIS render. A later re-read of the LoRA
// sidecar or model card must not rewrite what was true when the pixels
// were made (#5638). Unknown stays null.
@@ -562,7 +569,7 @@ export function parseImageExecutionMarker(line) {
};
}
-export async function generateImage({ pythonPath, prompt = '', negativePrompt = '', modelId = LOCAL_IMAGEGEN_DEFAULT_MODEL, width = 1024, height = 1024, steps, guidance, seed, quantize = '8', loraFilenames = [], loraPaths = [], loraScales = [], initImagePath = null, initImageStrength = null, referenceImagePaths = [], referenceImageStrengths = [], visualConditioning = null, jobId: providedJobId = null, cleanC2PA = false, denoise = false, regenOf = null, upscaleTo = null, outputTarget = null }) {
+export async function generateImage({ pythonPath, prompt = '', negativePrompt = '', modelId = LOCAL_IMAGEGEN_DEFAULT_MODEL, width = 1024, height = 1024, steps, guidance, seed, quantize = '8', loraFilenames = [], loraPaths = [], loraScales = [], initImagePath = null, initImageStrength = null, referenceImagePaths = [], referenceImageStrengths = [], visualConditioning = null, jobId: providedJobId = null, cleanC2PA = false, denoise = false, regenOf = null, upscaleTo = null, outputTarget = null, promptAdditions = [] }) {
// The ingestion instant `renderTimingFields` measures from — the queue calls
// generateImage the moment it picks this job up. See lib/renderTiming.js.
const renderStartedAtMs = Date.now();
@@ -676,6 +683,7 @@ export async function generateImage({ pythonPath, prompt = '', negativePrompt =
regenOf,
loraTriggerWords,
loraLicenses,
+ promptAdditions,
resolveInputPath: resolveImageInputPath,
loraExists: existsSync,
});
diff --git a/server/services/imageGen/prepareParams.js b/server/services/imageGen/prepareParams.js
index c3075dc175..06207170b8 100644
--- a/server/services/imageGen/prepareParams.js
+++ b/server/services/imageGen/prepareParams.js
@@ -45,6 +45,7 @@ import { getUniverseRenderPin } from '../universeBuilder/crud.js';
import { getImageModels, isFlux2, isEditOnly } from '../../lib/mediaModels.js';
import { hardwareUnavailableReason, isHardwareCompatible } from '../../lib/systemCapabilities.js';
import { usesDiffusersRunner } from '../../lib/runners.js';
+import { promptAddition } from '../../lib/modelInput.js';
// The job tags that name the record owning a render, each mapped to the record
// loader and the render target whose `renderDefaults` pin backs it. A generate
@@ -232,7 +233,10 @@ export async function prepareGenerateParams({ data, files, referenceImageFields
// Named inputs precede uploads. Keep the latter's packed strength positions intact.
if (!referenceUploads.length) namedReferenceFiles.push(...styles);
const look = musicVideoStylePrompt(musicVideoProject, referenceUploads.length ? 0 : styles.length);
- if (look) data.prompt = [data.prompt, look].filter(Boolean).join('. ');
+ if (look) {
+ data.prompt = [data.prompt, look].filter(Boolean).join('. ');
+ data.promptAdditions = [...(data.promptAdditions || []), promptAddition('mood-board-style', 'Mood board style', look)];
+ }
}
const referenceImageCount = namedReferenceFiles.length + referenceUploads.length;
const inputImageCount = (initUpload || data.initImageFile ? 1 : 0) + referenceImageCount;
diff --git a/server/services/videoGen/fal.js b/server/services/videoGen/fal.js
index a1a78aaa18..a13c0a6af2 100644
--- a/server/services/videoGen/fal.js
+++ b/server/services/videoGen/fal.js
@@ -49,6 +49,7 @@ import {
FAL_QUEUE_BASE, awaitFalCompletion, cancelFalRequest, createFalRequestEntry, fileToDataUri,
readCompletedFalResult, resolveFalApiKey, submitFalRequest,
} from '../falQueue.js';
+import { carriedPromptAdditions, promptAddition, sentPromptRecord } from '../../lib/modelInput.js';
// The queue transport (submit / poll / retrieve / cancel) is shared with the
// fal.ai image backend in `services/falQueue.js`. Re-exported so existing
@@ -146,7 +147,7 @@ export async function generateVideo({
sourceImagePath = null, lastImagePath = null, jobId: providedJobId = null,
audioFilePath = null, lipSync = null, shotInstruction = null,
uploadedTempPath = null, uploadedTempPaths = [],
- resolution = null, generateAudio = false, frameAspect = null,
+ resolution = null, generateAudio = false, frameAspect = null, promptAdditions = [],
}) {
await ensureDir(PATHS.videos);
const renderStartedAtMs = Date.now();
@@ -210,11 +211,6 @@ export async function generateVideo({
throw new ServerError(err.message, { status: 400, code: 'VALIDATION_ERROR' });
}
- if (planned?.promptClipped) {
- const { from, to, max } = planned.promptClipped;
- console.warn(`✂️ fal prompt for ${modelId} shortened ${from} → ${to} chars (model limit ${max})`);
- }
-
if (lastImagePath && !planned) {
throw new ServerError('End frames require a fal.ai model with catalogued end-frame support', { status: 400, code: 'VALIDATION_ERROR' });
}
@@ -238,6 +234,14 @@ export async function generateVideo({
// What PortOS expected this render to cost at fal's list price, or null for
// an uncurated model — recorded so spend can be audited per clip.
estimatedCostUsd: planned?.estimatedCostUsd ?? null,
+ // The text fal receives: a negative prompt the model has no field for rides folded in as "Avoid:".
+ ...sentPromptRecord({
+ sentPrompt: planned ? (planned.body.prompt ?? '') : buildRequestBody({ prompt, negativePrompt }).prompt,
+ sentNegativePrompt: planned?.body.negative_prompt ?? null,
+ carried: carriedPromptAdditions({ promptAdditions }),
+ added: [(!planned || planned.model.negativePrompt === 'fold') && planned?.model.prompt !== 'none'
+ ? promptAddition('avoid', 'Negative prompt, sent as "Avoid:"', negativePrompt) : null],
+ }),
filename,
createdAt: new Date().toISOString(),
mode: sourceImagePath ? 'image' : 'text',
diff --git a/server/services/videoGen/generateVideo.js b/server/services/videoGen/generateVideo.js
index cdcbab5a6b..8a36f50ad8 100644
--- a/server/services/videoGen/generateVideo.js
+++ b/server/services/videoGen/generateVideo.js
@@ -78,6 +78,7 @@ import {
} from '../../lib/videoDraftDecoders.js';
import { isDefaultVideoStreamingMode } from '../../lib/videoStreamingMode.js';
import { extendLatentFrames, supportsContextWindow } from '../../lib/videoContinuity.js';
+import { carriedPromptAdditions, promptAddition, sentPromptRecord } from '../../lib/modelInput.js';
// Re-export the extracted runtime + history surface so existing deep imports
// (`from '../videoGen/local.js'`) keep resolving every symbol they used to.
export * from './runtimes.js';
@@ -153,7 +154,7 @@ export const listVideoModels = () => getVideoModels().map(decorateVideoModel);
export const defaultVideoModelId = (capabilities, preferredId) => getDefaultVideoModelId(capabilities, preferredId);
-export async function generateVideo({ pythonPath, prompt, negativePrompt = '', modelId, width = null, height = null, numFrames = null, fps = 24, steps, guidanceScale, seed, batchSize = 1, tiling = 'auto', disableAudio = false, sourceImagePath = null, uploadedTempPath = null, uploadedTempPaths = [], lastImagePath = null, keyframes = null, extendFromVideoPath = null, audioFilePath = null, audioStartSec = null, audioConditioning = null, mode = null, imageStrength = null, i2vReferenceMode = null, loras = null, icReferencePaths = null, icStrength = null, icAttentionStrength = null, icSkipStage2 = false, textEncoderId = null, speedProfileId = null, draftDecode = null, streamingMode = null, visualConditioning = null, hidden = false, keepExtendSource = false, displaySleep = null, jobId: providedJobId = null }) {
+export async function generateVideo({ pythonPath, prompt, negativePrompt = '', modelId, width = null, height = null, numFrames = null, fps = 24, steps, guidanceScale, seed, batchSize = 1, tiling = 'auto', disableAudio = false, sourceImagePath = null, uploadedTempPath = null, uploadedTempPaths = [], lastImagePath = null, keyframes = null, extendFromVideoPath = null, audioFilePath = null, audioStartSec = null, audioConditioning = null, mode = null, imageStrength = null, i2vReferenceMode = null, loras = null, icReferencePaths = null, icStrength = null, icAttentionStrength = null, icSkipStage2 = false, textEncoderId = null, speedProfileId = null, draftDecode = null, streamingMode = null, visualConditioning = null, hidden = false, keepExtendSource = false, displaySleep = null, jobId: providedJobId = null, promptAdditions = [] }) {
const sourceAudioProvenance = audioConditioning == null ? null : federatedSourceAudioSchema.parse(audioConditioning);
uploadedTempPaths = Array.isArray(uploadedTempPaths) ? uploadedTempPaths : [];
if (!prompt?.trim()) throw new ServerError('Prompt is required', { status: 400, code: 'VALIDATION_ERROR' });
@@ -622,6 +623,8 @@ export async function generateVideo({ pythonPath, prompt, negativePrompt = '', m
// was actually rendered. Present only when something was woven, so every
// other history row stays byte-identical to pre-feature renders.
...(addedTriggerWords.length ? { renderPrompt, addedTriggerWords } : {}),
+ ...sentPromptRecord({ sentPrompt: renderPrompt, sentNegativePrompt: negativePrompt, carried: carriedPromptAdditions({ promptAdditions }),
+ added: [promptAddition('lora-triggers', 'LoRA trigger words', addedTriggerWords.join(', '))] }),
...(hidden ? { hidden: true } : {}),
};
// Each render gets one private preview directory. Runners overwrite the
diff --git a/server/services/videoGen/grok.js b/server/services/videoGen/grok.js
index 451354f4d2..c88b44a6b8 100644
--- a/server/services/videoGen/grok.js
+++ b/server/services/videoGen/grok.js
@@ -40,6 +40,7 @@ import { resolveGrokDuration } from '../../lib/grokVideoClip.js';
import { withSpawnCwdEnv } from '../../lib/spawnCwd.js';
import { probeVideoDuration } from '../../lib/ffmpeg.js';
import { maintenance } from '../../lib/maintenanceAdmission.js';
+import { agentInstructionsAddition, carriedPromptAdditions, sentPromptRecord } from '../../lib/modelInput.js';
// 30 minutes — an image-first video turn is two sequential tool calls
// (image_gen then image_to_video render + download), so it runs meaningfully
@@ -123,6 +124,7 @@ export async function generateVideo({
sourceImagePath = null, duration,
uploadedTempPath = null,
jobId: providedJobId = null,
+ promptAdditions = [],
}) {
await ensureDir(PATHS.videos);
@@ -177,6 +179,8 @@ export async function generateVideo({
filename,
createdAt: new Date().toISOString(),
mode: sourceImagePath ? 'image' : 'text',
+ ...sentPromptRecord({ sentPrompt: fullPrompt, carried: carriedPromptAdditions({ promptAdditions }),
+ added: [agentInstructionsAddition('Grok video tool instructions', fullPrompt, prompt)] }),
};
const job = { ...meta, clients: [], status: 'running', renderStartedAtMs };
jobs.set(jobId, job);
diff --git a/server/services/videoGen/submitJob.js b/server/services/videoGen/submitJob.js
index a6117c326d..4370054a0c 100644
--- a/server/services/videoGen/submitJob.js
+++ b/server/services/videoGen/submitJob.js
@@ -387,6 +387,8 @@ const submitValidatedVideoGenJob = async (body, uploads) => {
params.remoteMedia.request = buildFederatedMediaRequest({ kind: 'video', params: { ...params.remoteMedia.request, prompt: params.prompt } });
}
if (params.remoteMedia && !params.musicVideo) delete params.prompt;
+ // The labeled additions the composer put around the user's text ride with the job to its record.
+ if (body.promptAdditions?.length) params.promptAdditions = body.promptAdditions;
return enqueueJob({ kind: 'video', params });
},
);