Skip to content

🎥 Make videos programmatically with React

Choose shared-memory transparency from the output pixel format

Jonny Burgereb673fd51ac7
diff --git a/packages/docs/docs/options/jpeg-quality.mdx b/packages/docs/docs/options/jpeg-quality.mdx
index 1b82bb5096..ae3bf5b373 100644
--- a/packages/docs/docs/options/jpeg-quality.mdx
+++ b/packages/docs/docs/options/jpeg-quality.mdx
@@ -8,7 +8,7 @@ crumb: Options
custom_edit_url: https://github.com/remotion-dev/remotion/edit/main/packages/renderer/src/options/jpeg-quality.tsx
---
-Sets the quality of the generated JPEG images. Must be an integer between 0 and 100. Default: 80.
+Sets the quality of the generated JPEG images. Must be an integer between 0 and 100. Default: 80.<br />This option has no effect when [shared-memory capture](/docs/shared-memory-capture) is active.
## CLI flag
diff --git a/packages/docs/docs/options/video-image-format.mdx b/packages/docs/docs/options/video-image-format.mdx
index 8d91ff2cdf..297ac96185 100644
--- a/packages/docs/docs/options/video-image-format.mdx
+++ b/packages/docs/docs/options/video-image-format.mdx
@@ -8,7 +8,7 @@ crumb: Options
custom_edit_url: https://github.com/remotion-dev/remotion/edit/main/packages/renderer/src/options/video-image-format.tsx
---
-The image format to use when rendering frames for a video. Must be one of "png", "jpeg", "none". Default: `"jpeg"`. JPEG is faster, but does not support transparency.
+The image format to use when rendering frames for a video. Must be one of "png", "jpeg", "none". Default: `"jpeg"`. JPEG is faster, but does not support transparency.<br />When [shared-memory capture](/docs/shared-memory-capture) is active, this option has no effect. The output pixel format determines transparency.
## CLI flag
diff --git a/packages/docs/docs/prores.mdx b/packages/docs/docs/prores.mdx
index f727f5517c..8012c91d19 100644
--- a/packages/docs/docs/prores.mdx
+++ b/packages/docs/docs/prores.mdx
@@ -90,7 +90,7 @@ ProRes does not support [`crf`](/docs/options/crf), and [`videoBitrate`](/docs/o
## Transparent ProRes
To export a transparent overlay or transition for a video editor, use a ProRes profile with alpha support: `"4444"` or `"4444-xq"`.
-The video must be rendered with PNG frames and the `yuva444p10le` pixel format.
+Use the `yuva444p10le` pixel format. [Shared-memory capture](/docs/shared-memory-capture) preserves transparency automatically; screenshot capture requires PNG frames.
```bash title="Terminal"
npx remotion render MyComp out/overlay.mov --codec=prores --prores-profile=4444 --image-format=png --pixel-format=yuva444p10le
diff --git a/packages/docs/docs/renderer/render-media.mdx b/packages/docs/docs/renderer/render-media.mdx
index 4cd2d78aac..da61d4335c 100644
--- a/packages/docs/docs/renderer/render-media.mdx
+++ b/packages/docs/docs/renderer/render-media.mdx
@@ -179,6 +179,8 @@ In which image format the frames should be rendered.
- `png` if you want to [render transparent videos](/docs/transparent-videos/)
- `none` if you are rendering audio
+This option has no effect when [shared-memory capture](/docs/shared-memory-capture) is active. The output [`pixelFormat`](#pixelformat) determines whether transparency is preserved.
+
### `browserExecutable?`<AvailableFrom v="3.0.11" />
A string defining the absolute path on disk of the browser executable that should be used. By default Remotion will try to detect it automatically and download one if none is available. If `puppeteerInstance` is defined, it will take precedence over `browserExecutable`.
@@ -211,7 +213,9 @@ _number_
Sets the quality of the generated JPEG images. Must be an integer between 0 and 100. Default is to leave it up to the browser, [current default is 80](https://github.com/chromium/chromium/blob/99314be8152e688bafbbf9a615536bdbb289ea87/headless/lib/browser/protocol/headless_handler.cc#L32).
-Only applies if `imageFormat` is `'jpeg'`, otherwise this option is invalid.
+For screenshot capture, only applies if `imageFormat` is `'jpeg'`.
+
+This option has no effect when [shared-memory capture](/docs/shared-memory-capture) is active.
### `muted?`<AvailableFrom v="3.2.1" />
diff --git a/packages/docs/docs/shared-memory-capture.mdx b/packages/docs/docs/shared-memory-capture.mdx
index 3bcb36ee7f..7b4f648738 100644
--- a/packages/docs/docs/shared-memory-capture.mdx
+++ b/packages/docs/docs/shared-memory-capture.mdx
@@ -33,16 +33,16 @@ The patch does not apply to previews or [client-side rendering](/docs/client-sid
## Image format and quality
-When shared-memory capture is active, both `imageFormat: "jpeg"` and `imageFormat: "png"` send uncompressed bitmaps to FFmpeg. Switching between them no longer changes the image compression or capture quality.
+When shared-memory capture is active, the [`imageFormat`](/docs/renderer/render-media#imageformat) and [`jpegQuality`](/docs/renderer/render-media#jpegquality) options have no effect. Frames are passed to FFmpeg as uncompressed bitmaps.
-The option still controls the background:
+The output [pixel format](/docs/renderer/render-media#pixelformat) determines the background:
-- `"jpeg"` uses an opaque black background for areas not painted by your composition.
-- `"png"` preserves transparency during capture. Follow the [transparent video guide](/docs/transparent-videos) to choose an output codec and pixel format that preserve transparency in the final video.
+- `"yuva420p"` and `"yuva444p10le"` preserve transparency automatically. Follow the [transparent video guide](/docs/transparent-videos) to choose a codec that supports your pixel format.
+- Other pixel formats composite transparent areas against black.
-The [`jpegQuality`](/docs/renderer/render-media#jpegquality) option has no effect on video frames captured through shared memory. When Remotion falls back to screenshots, the image format and JPEG quality options work as before.
+When Remotion falls back to screenshots, the image format and JPEG quality options work as before. Transparent output requires PNG screenshots.
-In the Studio render dialog, the **Picture** tab shows **Transparent background** instead of **Image Format** and hides **JPEG Quality** when shared-memory capture is available. The transparency setting applies to captured frames; exporting a transparent video also requires a [codec and pixel format that support transparency](/docs/transparent-videos).
+In the Studio render dialog, the Picture tab hides Image Format and JPEG Quality when shared-memory capture is available. Choose a pixel format with alpha support in the Encoding tab to export a transparent video.
Shared-memory capture avoids the extra JPEG compression step. The final video is still compressed according to your [codec and quality settings](/docs/encoding). Overall render speed depends on your composition, encoder and machine.
diff --git a/packages/docs/docs/transparent-videos.mdx b/packages/docs/docs/transparent-videos.mdx
index 9ef655cff3..e7c9cc5e7e 100644
--- a/packages/docs/docs/transparent-videos.mdx
+++ b/packages/docs/docs/transparent-videos.mdx
@@ -29,12 +29,14 @@ If you are lucky enough to visit the website in one of those supported web brows
## Creating a WebM video with Alpha channel
-In order to create a transparent video, you need at least version 1.4 of Remotion. Make sure to not set any background (use the checkerboard button to ensure your video is transparent). In order to render your videos transparent, you need to change 3 settings:
+In order to create a transparent video, you need at least version 1.4 of Remotion. Make sure to not set any background (use the checkerboard button to ensure your video is transparent). Use these settings:
-- Render each frame as PNG.
+- Render each frame as PNG when using screenshot capture.
- Use the VP8 or VP9 codec
- Use the `yuva420p` pixel format.
+With [shared-memory capture](/docs/shared-memory-capture), available from <AvailableFrom v="4.0.534" inline />, the output pixel format enables transparency automatically. You do not need to select PNG. The examples below also work with screenshot capture.
+
If you want to set these options and persist them, add this to your `remotion.config.ts` file (create it if you don't yet have one);
```tsx twoslash
diff --git a/packages/renderer/src/image-format.ts b/packages/renderer/src/image-format.ts
index 72c38822c4..c3a8001ec0 100644
--- a/packages/renderer/src/image-format.ts
+++ b/packages/renderer/src/image-format.ts
@@ -17,11 +17,22 @@ export type ImageFormat =
export const DEFAULT_VIDEO_IMAGE_FORMAT: VideoImageFormat = 'jpeg';
export const DEFAULT_STILL_IMAGE_FORMAT: StillImageFormat = 'png';
+export const validateVideoImageFormat = (
+ videoImageFormat: VideoImageFormat,
+) => {
+ if (!validVideoImageFormats.includes(videoImageFormat)) {
+ throw new TypeError(
+ `Value ${videoImageFormat} is not valid as an image format.`,
+ );
+ }
+};
+
// By returning a value, we improve testability as we can specifically test certain branches
export const validateSelectedPixelFormatAndImageFormatCombination = (
pixelFormat: PixelFormat | undefined,
videoImageFormat: VideoImageFormat,
): 'none' | 'valid' => {
+ validateVideoImageFormat(videoImageFormat);
if (videoImageFormat === 'none') {
return 'none';
}
@@ -30,12 +41,6 @@ export const validateSelectedPixelFormatAndImageFormatCombination = (
return 'valid';
}
- if (!validVideoImageFormats.includes(videoImageFormat)) {
- throw new TypeError(
- `Value ${videoImageFormat} is not valid as an image format.`,
- );
- }
-
if (pixelFormat !== 'yuva420p' && pixelFormat !== 'yuva444p10le') {
return 'valid';
}
diff --git a/packages/renderer/src/options/jpeg-quality.tsx b/packages/renderer/src/options/jpeg-quality.tsx
index f86ee931b7..ecf8724be5 100644
--- a/packages/renderer/src/options/jpeg-quality.tsx
+++ b/packages/renderer/src/options/jpeg-quality.tsx
@@ -26,6 +26,9 @@ export const jpegQualityOption = {
<>
Sets the quality of the generated JPEG images. Must be an integer between
0 and 100. Default: 80.
+ <br />
+ This option has no effect when{' '}
+ <a href="/docs/shared-memory-capture">shared-memory capture</a> is active.
</>
),
ssrName: 'jpegQuality',
diff --git a/packages/renderer/src/options/video-image-format.tsx b/packages/renderer/src/options/video-image-format.tsx
index b39dcb491e..f1d7a82cc3 100644
--- a/packages/renderer/src/options/video-image-format.tsx
+++ b/packages/renderer/src/options/video-image-format.tsx
@@ -17,6 +17,10 @@ export const videoImageFormatOption = {
{validVideoImageFormats.map((f) => `"${f}"`).join(', ')}. Default:{' '}
<code>&quot;jpeg&quot;</code>. JPEG is faster, but does not support
transparency.
+ <br />
+ When <a href="/docs/shared-memory-capture">shared-memory capture</a> is
+ active, this option has no effect. The output pixel format determines
+ transparency.
</>
),
ssrName: 'imageFormat' as const,
diff --git a/packages/renderer/src/remotion-shared-memory.ts b/packages/renderer/src/remotion-shared-memory.ts
index 81cb30b920..47c2a84a2f 100644
--- a/packages/renderer/src/remotion-shared-memory.ts
+++ b/packages/renderer/src/remotion-shared-memory.ts
@@ -4,8 +4,13 @@ import type {
RemotionFramePoolBackend,
} from './browser/devtools-types';
import {isTargetClosedErr} from './browser/flaky-errors';
+import {
+ type StillImageFormat,
+ validateSelectedPixelFormatAndImageFormatCombination,
+} from './image-format';
import type {LogLevel} from './log-level';
import {Log} from './logger';
+import type {PixelFormat} from './pixel-format';
const encoderPadding = 64;
const slotsPerPage = 1;
@@ -82,6 +87,7 @@ export class RemotionSharedMemoryCapture {
#nextFrameId = 1;
#supported: boolean | null = null;
#initialization: Promise<void> | null = null;
+ #pixelFormat: PixelFormat | null = null;
readonly #options: {
width: number;
height: number;
@@ -106,6 +112,28 @@ export class RemotionSharedMemoryCapture {
return this.#supported === true;
}
+ getImageFormat(imageFormat: StillImageFormat): StillImageFormat {
+ if (!this.isSupported() || this.#pixelFormat === null) {
+ if (this.#pixelFormat !== null && imageFormat === 'jpeg') {
+ validateSelectedPixelFormatAndImageFormatCombination(
+ this.#pixelFormat,
+ imageFormat,
+ );
+ }
+
+ return imageFormat;
+ }
+
+ return this.#pixelFormat === 'yuva420p' ||
+ this.#pixelFormat === 'yuva444p10le'
+ ? 'png'
+ : 'jpeg';
+ }
+
+ setPixelFormat(pixelFormat: PixelFormat) {
+ this.#pixelFormat = pixelFormat;
+ }
+
async initializePages(pages: Page[]) {
if (this.#initialization) {
await this.#initialization;
diff --git a/packages/renderer/src/render-media.ts b/packages/renderer/src/render-media.ts
index 0deae3363c..3060a8d58f 100644
--- a/packages/renderer/src/render-media.ts
+++ b/packages/renderer/src/render-media.ts
@@ -34,6 +34,7 @@ import type {VideoImageFormat} from './image-format';
import {
DEFAULT_VIDEO_IMAGE_FORMAT,
validateSelectedPixelFormatAndImageFormatCombination,
+ validateVideoImageFormat,
} from './image-format';
import {isAudioCodec} from './is-audio-codec';
import {DEFAULT_JPEG_QUALITY, validateJpegQuality} from './jpeg-quality';
@@ -444,10 +445,21 @@ const internalRenderMediaRaw = async ({
compositionWithPossibleUnevenDimensions.defaultVideoImageFormat ??
DEFAULT_VIDEO_IMAGE_FORMAT);
- validateSelectedPixelFormatAndImageFormatCombination(
- pixelFormat,
- imageFormat,
- );
+ validateVideoImageFormat(imageFormat);
+ const remotionSharedMemorySupport =
+ !parallelEncoding || disableSharedMemoryCapture
+ ? null
+ : await probeRemotionSharedMemoryFfmpegSupport({
+ binariesDirectory,
+ indent,
+ logLevel,
+ });
+ if (!remotionSharedMemorySupport?.sharedMemory) {
+ validateSelectedPixelFormatAndImageFormatCombination(
+ pixelFormat,
+ imageFormat,
+ );
+ }
const workingDir = fs.mkdtempSync(
path.join(os.tmpdir(), 'react-motion-render'),
@@ -483,14 +495,6 @@ const internalRenderMediaRaw = async ({
);
}
- const remotionSharedMemorySupport =
- preEncodedFileLocation === null || disableSharedMemoryCapture
- ? null
- : await probeRemotionSharedMemoryFfmpegSupport({
- binariesDirectory,
- indent,
- logLevel,
- });
// Chromium creates file-backed frame pools in this private directory when
// POSIX shared memory is unavailable (no /dev/shm, as on AWS Lambda). It is
// only offered when the FFmpeg binary can open such pools via -pool_dir.
@@ -507,6 +511,7 @@ const internalRenderMediaRaw = async ({
backingDirectory: remotionSharedMemoryPoolDirectory,
})
: null;
+ remotionSharedMemory?.setPixelFormat(pixelFormat);
const composition = {
...compositionWithPossibleUnevenDimensions,
diff --git a/packages/renderer/src/take-frame.ts b/packages/renderer/src/take-frame.ts
index 1eb7b1872d..e1cad7a0d6 100644
--- a/packages/renderer/src/take-frame.ts
+++ b/packages/renderer/src/take-frame.ts
@@ -41,10 +41,12 @@ export const takeFrame = async <T extends RemotionSharedMemoryCapture | null>({
return null as TakeFrameResult<T>;
}
+ const captureImageFormat =
+ remotionSharedMemory?.getImageFormat(imageFormat) ?? imageFormat;
if (
- imageFormat === 'png' ||
- imageFormat === 'pdf' ||
- imageFormat === 'webp'
+ captureImageFormat === 'png' ||
+ captureImageFormat === 'pdf' ||
+ captureImageFormat === 'webp'
) {
await puppeteerEvaluateWithCatch({
pageFunction: () => {
@@ -69,9 +71,10 @@ export const takeFrame = async <T extends RemotionSharedMemoryCapture | null>({
const buf = await screenshot({
page: freePage,
- omitBackground: imageFormat === 'png' || imageFormat === 'webp',
+ omitBackground:
+ captureImageFormat === 'png' || captureImageFormat === 'webp',
path: (wantsBuffer ? undefined : output) ?? undefined,
- type: imageFormat,
+ type: captureImageFormat,
jpegQuality,
width,
height,
diff --git a/packages/renderer/src/test/get-chrome-download-url.test.ts b/packages/renderer/src/test/get-chrome-download-url.test.ts
index 8feb6f18b1..1620615a27 100644
--- a/packages/renderer/src/test/get-chrome-download-url.test.ts
+++ b/packages/renderer/src/test/get-chrome-download-url.test.ts
@@ -1,7 +1,9 @@
import {expect, test} from 'bun:test';
import {getExpectedVersion} from '../browser/BrowserFetcher';
import {
+ canUseRemotionMediaBinaries,
getChromeDownloadUrl,
+ isAmazonLinux2023,
TESTED_VERSION,
} from '../browser/get-chrome-download-url';
@@ -62,5 +64,9 @@ test('uses a distinct cache marker for the Remotion Apple Silicon build', () =>
version: null,
chromeMode: 'headless-shell',
}),
- ).toBe(TESTED_VERSION);
+ ).toBe(
+ canUseRemotionMediaBinaries() || isAmazonLinux2023()
+ ? `${TESTED_VERSION}-remotion-v3`
+ : TESTED_VERSION,
+ );
});
diff --git a/packages/studio/src/components/RenderModal/RenderModalPicture.tsx b/packages/studio/src/components/RenderModal/RenderModalPicture.tsx
index 7f3266eb2b..a9d311d5f1 100644
--- a/packages/studio/src/components/RenderModal/RenderModalPicture.tsx
+++ b/packages/studio/src/components/RenderModal/RenderModalPicture.tsx
@@ -1,6 +1,5 @@
import type {StillImageFormat, VideoImageFormat} from '@remotion/renderer';
-import React, {useCallback} from 'react';
-import {Checkbox} from '../Checkbox';
+import React from 'react';
import {VERTICAL_SCROLLBAR_CLASSNAME} from '../Menu/is-menu-item';
import type {SegmentedControlItem} from '../SegmentedControl';
import {SegmentedControl} from '../SegmentedControl';
@@ -21,9 +20,6 @@ export const RenderModalPicture: React.FC<{
readonly setScale: React.Dispatch<React.SetStateAction<number>>;
readonly imageFormatOptions: SegmentedControlItem[];
readonly videoImageFormat: VideoImageFormat;
- readonly setVideoImageFormat: React.Dispatch<
- React.SetStateAction<VideoImageFormat>
- >;
readonly usesSharedMemoryCapture: boolean;
readonly stillImageFormat: StillImageFormat;
readonly setJpegQuality: React.Dispatch<React.SetStateAction<number>>;
@@ -36,7 +32,6 @@ export const RenderModalPicture: React.FC<{
setScale,
imageFormatOptions,
videoImageFormat,
- setVideoImageFormat,
usesSharedMemoryCapture,
setJpegQuality,
jpegQuality,
@@ -44,38 +39,16 @@ export const RenderModalPicture: React.FC<{
compositionWidth,
compositionHeight,
}) => {
- const onTransparentBackgroundChanged = useCallback(
- (event: React.ChangeEvent<HTMLInputElement>) => {
- setVideoImageFormat(event.target.checked ? 'png' : 'jpeg');
- },
- [setVideoImageFormat],
- );
-
return (
<div style={container} className={VERTICAL_SCROLLBAR_CLASSNAME}>
- {renderMode === 'video' ? (
+ {renderMode === 'video' && !usesSharedMemoryCapture ? (
<div style={optionRow}>
- {usesSharedMemoryCapture ? (
- <label style={label} htmlFor="transparent-background">
- Transparent background
- </label>
- ) : (
- <div style={label}>Image Format</div>
- )}
+ <div style={label}>Image Format</div>
<div style={rightRow}>
- {usesSharedMemoryCapture ? (
- <Checkbox
- checked={videoImageFormat === 'png'}
- onChange={onTransparentBackgroundChanged}
- name="transparent-background"
- inputId="transparent-background"
- />
- ) : (
- <SegmentedControl
- items={imageFormatOptions}
- needsWrapping={false}
- />
- )}
+ <SegmentedControl
+ items={imageFormatOptions}
+ needsWrapping={false}
+ />
</div>
</div>
) : null}
@@ -93,7 +66,9 @@ export const RenderModalPicture: React.FC<{
setJpegQuality={setJpegQuality}
/>
) : null}
- {renderMode === 'video' ? <RenderModalHr /> : null}
+ {renderMode === 'video' && !usesSharedMemoryCapture ? (
+ <RenderModalHr />
+ ) : null}
<ScaleSetting
scale={scale}
setScale={setScale}
diff --git a/packages/studio/src/components/RenderModal/ServerRenderModal.tsx b/packages/studio/src/components/RenderModal/ServerRenderModal.tsx
index 9cba48e4fc..ac7315bd91 100644
--- a/packages/studio/src/components/RenderModal/ServerRenderModal.tsx
+++ b/packages/studio/src/components/RenderModal/ServerRenderModal.tsx
@@ -1650,7 +1650,6 @@ const RenderModal: React.FC<
jpegQuality={jpegQuality}
setJpegQuality={setJpegQuality}
videoImageFormat={videoImageFormat}
- setVideoImageFormat={setVideoImageFormat}
usesSharedMemoryCapture={usesSharedMemoryCapture}
stillImageFormat={stillImageFormat}
compositionWidth={resolvedComposition.width}