Skip to content

Free open-source image editor for the browser

Add RAW import with raw-webgpu and absolute white balance (#16)

Rodrigo Belfiore4958db8ac92a
diff --git a/.gitignore b/.gitignore
index 2d1ef52..e913e05 100644
--- a/.gitignore
+++ b/.gitignore
@@ -5,3 +5,5 @@ dist
.env*
playwright-report
test-results
+
+.cache/
diff --git a/API.md b/API.md
index 1c68eac..c364709 100644
--- a/API.md
+++ b/API.md
@@ -47,6 +47,8 @@ Loading calls are queued. XMP import requires a loaded document and is skipped i
`setToneCurve(points)` replaces the tone curve. Each point is `{ x, y }` with coordinates between 0 and 1. Supply at least two points, ordered by `x` with a minimum gap of `1/1024`. The first point must have `x = 0` or `y = 0`; the last must have `x = 1` or `y = 1`. Call `setToneCurve()` to reset it.
+For RAW sources, `setWhiteBalance({ temperature, tint })` sets absolute Kelvin and DNG tint, preserving unspecified values. Temperature accepts 2000–25000 K and tint accepts -150–150, with either range extended to include the file's As Shot value. `setWhiteBalance()` restores the source's initial balance. When camera multipliers are unavailable, that initial balance is the decoder's daylight fallback. The command throws for non-RAW sources or invalid values. Edits enter history immediately; preview development runs asynchronously, and export waits for the captured scene's development. The incremental temperature/tint adjustments remain separate RGB adjustments.
+
`editScene(change)` shallowly merges a partial [Scene](src/lib/editor/scene.ts) into the document as an undoable edit. Supply complete values for nested fields such as `frame`. Prefer `setAdjustments` and `setToneCurve` for their validation.
## History
@@ -81,6 +83,6 @@ Each edit creates an undo step unless a group is open. Preview changes are outsi
## State
-`getState()` returns a detached snapshot containing `file`, `documentId`, `size`, `frame`, `adjustments`, `toneCurve`, `preview`, and `history`. `file` is the filename, and `history` contains `undoCount` and `redoCount`.
+`getState()` returns a detached snapshot containing `file`, `documentId`, `size`, `frame`, `adjustments`, `whiteBalance`, `toneCurve`, `preview`, and `history`. `whiteBalance` contains absolute temperature/tint for RAW sources and is undefined for other sources. `file` is the filename, and `history` contains `undoCount` and `redoCount`.
Without a document, `documentId`, `size`, `frame`, and `preview` are undefined. Adjustments and the tone curve use their defaults, and history counts are zero. Mutating the snapshot does not edit the document.
diff --git a/NOTICE b/NOTICE
new file mode 100644
index 0000000..04d602d
--- /dev/null
+++ b/NOTICE
@@ -0,0 +1,6 @@
+OpenLight uses raw-webgpu, including LibRaw, Adobe DNG SDK and JPEG XL.
+Bundled notices and the pinned native build recipe ship in the installed package:
+node_modules/raw-webgpu/NOTICE
+node_modules/raw-webgpu/THIRD_PARTY_LICENSES.txt
+node_modules/raw-webgpu/native/build.py
+Upstream package: https://github.com/roprgm/raw-webgpu
diff --git a/README.md b/README.md
index 6dc14c9..3f2f376 100644
--- a/README.md
+++ b/README.md
@@ -17,10 +17,13 @@ Image processing runs locally with WebGPU. Built with TypeScript and React.
- Pan and zoom
- Camera Raw XMP import
- TIFF import at 16-bit and floating-point precision, decoded on the GPU
+- Camera RAW and DNG import through LibRaw, with absolute white balance and As Shot reset
- PNG and JPEG export
Requires a WebGPU-capable browser.
+RAW decoding and GPU development use [raw-webgpu](https://github.com/roprgm/raw-webgpu), which documents format support and limitations. OpenLight owns editing, history and preview/export lifetimes.
+
## Development
```sh
@@ -37,3 +40,5 @@ Run `bun run check` to format and lint, `bun run build` to type-check and build,
## License
[MIT](LICENSE)
+
+The bundled RAW decoder includes separately licensed libraries. See [NOTICE](NOTICE).
diff --git a/TODO.md b/TODO.md
new file mode 100644
index 0000000..cfe9ef4
--- /dev/null
+++ b/TODO.md
@@ -0,0 +1,12 @@
+# TODO
+
+## Evaluate replacing tiff-gpu with raw-webgpu
+
+Keep the current TIFF loader until raw-webgpu covers the TIFF formats already supported by OpenLight and its performance is acceptable.
+
+- Add support for bilevel, palette and float64 TIFF in raw-webgpu. These are the three known coverage regressions in the shared 21-fixture corpus. Both loaders currently reject the YCbCr JPEG fixture.
+- Verify reference pixels, ICC color conversion, alpha, orientation, HDR and precision with every TIFF fixture.
+- Compare loading time and memory on the large compressed and uncompressed TIFF files in `public/debug`. The measured raw-webgpu path was slower; shared RAW infrastructure alone is not a reason to accept that regression.
+- Once coverage and performance are accepted, route TIFF through raw-webgpu and remove the superseded loader and dependencies. Keep the integration small and preserve regression coverage.
+
+Verify with `bun run test:tiff` and `bun run benchmark:tiff` in raw-webgpu.
diff --git a/biome.json b/biome.json
index 518ba9e..40d33cd 100644
--- a/biome.json
+++ b/biome.json
@@ -1,7 +1,7 @@
{
"$schema": "./node_modules/@biomejs/biome/configuration_schema.json",
"files": {
- "includes": ["**", "!dist"]
+ "includes": ["**", "!dist", "!.cache", "!.claude"]
},
"css": {
"parser": {
diff --git a/bun.lock b/bun.lock
index daf764a..0cbe4fd 100644
--- a/bun.lock
+++ b/bun.lock
@@ -12,6 +12,7 @@
"class-variance-authority": "^0.7.1",
"cn": "^0.2.4",
"gl-matrix": "^3.4.4",
+ "raw-webgpu": "0.1.0-alpha.1",
"react": "^19.2.8",
"react-dom": "^19.2.8",
"tailwindcss": "^4.3.3",
@@ -272,6 +273,8 @@
"postcss": ["[email protected]", "", { "dependencies": { "nanoid": "^3.3.17", "picocolors": "^1.1.1", "source-map-js": "^1.2.1" } }, "sha512-u82N74LFzG8ca+dD8puPnplTXoGH4fTPpVGuIbt36G3qvNlkvfD0lEAZSxaly3KX8TS/L1A1gsCEmvKmBcVbkQ=="],
+ "raw-webgpu": ["[email protected]", "", {}, "sha512-sDPYERiNCRO6OhyvH62lgD1CBmRon2GrbeJaAt12nCCnVivd5ROzBHTH94o7dAS7hmLScGlsHSk6L35iFZX9hw=="],
+
"react": ["[email protected]", "", {}, "sha512-PWaYA1L/q9u2u7xYQi+Y3L3Yfnie7XyLeaJICV1MGD6LprsBxcAqGjYyr0eY3p+QdsA+x/Irkt4Qif8D63+Sbw=="],
"react-dom": ["[email protected]", "", { "dependencies": { "scheduler": "^0.27.0" }, "peerDependencies": { "react": "^19.2.8" } }, "sha512-rVprimfGBG3DR+Tq0IQG2DT5PxKth1WIGDmj5yPmlzr4YBe7uyE+Du4oVqTDXZSHGGGXRtTJEGSSePyQCMBglQ=="],
diff --git a/package.json b/package.json
index 1759b2c..45bc674 100644
--- a/package.json
+++ b/package.json
@@ -18,6 +18,7 @@
"class-variance-authority": "^0.7.1",
"cn": "^0.2.4",
"gl-matrix": "^3.4.4",
+ "raw-webgpu": "0.1.0-alpha.1",
"react": "^19.2.8",
"react-dom": "^19.2.8",
"tailwindcss": "^4.3.3",
diff --git a/src/app/controls.ts b/src/app/controls.ts
index 7a1ac97..5f11dbf 100644
--- a/src/app/controls.ts
+++ b/src/app/controls.ts
@@ -7,6 +7,7 @@ import { createCameraRawXmpLoader } from "@/app/loaders/camera-raw-xmp";
import { createImageLoader } from "@/app/loaders/image";
import { createLoaderRegistry } from "@/app/loaders/registry";
import type { Workspace } from "@/app/workspace";
+import { setWhiteBalance } from "@/features/white-balance/edits";
import type { Preview } from "@/lib/editor/document";
import { setAdjustments, setToneCurve } from "@/lib/editor/document/edits";
import {
@@ -14,6 +15,7 @@ import {
defaultAdjustments,
type Scene,
} from "@/lib/editor/scene";
+import type { WhiteBalance } from "@/lib/image-source";
import { defaultCurve, type ToneCurve } from "@/lib/tone-curves/curve";
/** Imperative commands bound to an explicit workspace, usable without React. */
@@ -32,6 +34,8 @@ export function createControls(gpu: Gpu, workspace: Workspace) {
importXmp: (file: File) => files.loadFile(xmp, file),
setAdjustments: (change: Partial<Adjustments>) =>
setAdjustments(workspace.getDocument(), change),
+ setWhiteBalance: (change?: Partial<WhiteBalance>) =>
+ setWhiteBalance(workspace.getDocument(), change),
setToneCurve: (curve?: ToneCurve) =>
setToneCurve(workspace.getDocument(), curve),
editScene(change: Partial<Scene>) {
@@ -57,6 +61,7 @@ export function createControls(gpu: Gpu, workspace: Workspace) {
size: scene?.frame.size,
frame: scene?.frame,
adjustments: scene?.adjustments ?? defaultAdjustments,
+ whiteBalance: scene?.whiteBalance,
toneCurve: scene?.toneCurve ?? defaultCurve,
history: document?.history.status.getState() ?? {
undoCount: 0,
diff --git a/src/app/editor/export/export-image.ts b/src/app/editor/export/export-image.ts
index 078d79a..7f72500 100644
--- a/src/app/editor/export/export-image.ts
+++ b/src/app/editor/export/export-image.ts
@@ -25,22 +25,22 @@ export async function exportImage(
throw new Error("JPEG quality must be between 1 and 100.");
}
const scene = document.scene.getState();
- const { image } = document.resources.get(scene.source);
+ const source = document.resources.get(scene.source);
let output: ReturnType<typeof surface> | undefined;
let renderer: ReturnType<typeof createRenderer> | undefined;
try {
const [width, height] = scene.frame.size.map(Math.round);
const canvas = new OffscreenCanvas(width, height);
output = surface(gpu, canvas, { size: [width, height], dpr: 1 });
- renderer = createRenderer(gpu, image);
- renderer.update(scene);
+ renderer = createRenderer(gpu, source);
+ await renderer.update(scene);
const draw = createDisplay(gpu);
const texture = renderer.outputImage();
const destination = output;
frame(gpu, (frame) =>
draw(frame, destination, texture, { view: { zoom: 1, pan: [0, 0] } }),
);
- // Source work is submitted before yielding; the canvas owns the export pixels.
+ // The renderer retains the source until the canvas finishes encoding.
const blob = await canvas.convertToBlob({
type: `image/${options.format}`,
quality: options.format === "jpeg" ? options.quality / 100 : undefined,
diff --git a/src/app/editor/sidebar/adjustment-controls.tsx b/src/app/editor/sidebar/adjustment-controls.tsx
index aadb64b..851dd20 100644
--- a/src/app/editor/sidebar/adjustment-controls.tsx
+++ b/src/app/editor/sidebar/adjustment-controls.tsx
@@ -2,6 +2,7 @@ import type { ComponentProps, ReactNode } from "react";
import { useDocument, useScene } from "@/components/editor/session";
import { Collapsible } from "@/components/ui/collapsible";
import { Slider } from "@/components/ui/slider";
+import { WhiteBalanceControls } from "@/features/white-balance/controls";
import { setAdjustments } from "@/lib/editor/document/edits";
import {
type Adjustments,
@@ -44,6 +45,28 @@ function AdjustmentSlider({ name, ...props }: AdjustmentSliderProps) {
);
}
+function ColorTemperatureControls() {
+ const document = useDocument();
+ const source = useScene((scene) => scene.source);
+ if (document.resources.get(source).raw) {
+ return <WhiteBalanceControls />;
+ }
+ return (
+ <>
+ <AdjustmentSlider
+ name="incrementalTemperature"
+ label="Temp"
+ stops={stops.incrementalTemperature}
+ />
+ <AdjustmentSlider
+ name="incrementalTint"
+ label="Tint"
+ stops={stops.incrementalTint}
+ />
+ </>
+ );
+}
+
export default function AdjustmentControls({ curves }: { curves: ReactNode }) {
return (
<>
@@ -60,16 +83,7 @@ export default function AdjustmentControls({ curves }: { curves: ReactNode }) {
</Collapsible>
<Collapsible title="Color">
<div className="flex flex-col gap-2">
- <AdjustmentSlider
- name="incrementalTemperature"
- label="Temp"
- stops={stops.incrementalTemperature}
- />
- <AdjustmentSlider
- name="incrementalTint"
- label="Tint"
- stops={stops.incrementalTint}
- />
+ <ColorTemperatureControls />
<AdjustmentSlider
name="vibrance"
label="Vibrance"
diff --git a/src/app/loaders/image.ts b/src/app/loaders/image.ts
index 42cb048..293fb24 100644
--- a/src/app/loaders/image.ts
+++ b/src/app/loaders/image.ts
@@ -17,12 +17,13 @@ export function createImageLoader(gpu: Gpu, workspace: Workspace): FileLoader {
throw new Error("loadImage requires a File.");
}
await workspace.open(file.name, async () => {
- const image = await decode(gpu, file);
+ const decoded = await decode(gpu, file);
const resources = createResources();
- const source = resources.add(file, image);
+ const source = resources.add(file, decoded);
return createDocument(
{
- frame: imageFrame(image.size),
+ frame: imageFrame(decoded.image.size),
+ whiteBalance: decoded.raw?.asShot,
source,
adjustments: { ...defaultAdjustments },
toneCurve: defaultCurve,
diff --git a/src/components/editor/pipeline.tsx b/src/components/editor/pipeline.tsx
index b91a2a7..2c61b45 100644
--- a/src/components/editor/pipeline.tsx
+++ b/src/components/editor/pipeline.tsx
@@ -4,6 +4,7 @@ import {
useContext,
useEffect,
useMemo,
+ useState,
} from "react";
import { useGpu } from "vgpu-react";
import { useDocument, useScene } from "@/components/editor/session";
@@ -21,22 +22,50 @@ export function useRenderer() {
return renderer;
}
+function RendererError({ message }: { message: string }) {
+ return (
+ <p
+ role="alert"
+ className="fixed bottom-4 left-4 rounded bg-neutral-900 px-3 py-2 text-red-300 text-sm"
+ >
+ Renderer error: {message}
+ </p>
+ );
+}
+
type RendererProviderProps = { children: ReactNode };
export function RendererProvider({ children }: RendererProviderProps) {
const gpu = useGpu();
const document = useDocument();
const sourceId = useScene((scene) => scene.source);
- const source = document.resources.get(sourceId).image;
+ const source = document.resources.get(sourceId);
const renderer = useMemo(() => createRenderer(gpu, source), [gpu, source]);
+ const [error, setError] = useState<string>();
+
useEffect(() => {
- const render = () => renderer.update(document.scene.getState());
+ let active = true;
+ const render = () => {
+ setError(undefined);
+ renderer.update(document.scene.getState()).catch((error) => {
+ if (active) {
+ setError(String(error));
+ }
+ });
+ };
const unsubscribe = document.scene.subscribe(render);
render();
return () => {
+ active = false;
unsubscribe();
renderer.dispose();
};
}, [renderer, document]);
- return <RendererContext value={renderer}>{children}</RendererContext>;
+
+ return (
+ <RendererContext value={renderer}>
+ {children}
+ {error && <RendererError message={error} />}
+ </RendererContext>
+ );
}
diff --git a/src/features/white-balance/controls.tsx b/src/features/white-balance/controls.tsx
new file mode 100644
index 0000000..05a1576
--- /dev/null
+++ b/src/features/white-balance/controls.tsx
@@ -0,0 +1,41 @@
+import { useDocument, useScene } from "@/components/editor/session";
+import { Slider } from "@/components/ui/slider";
+import { setWhiteBalance, whiteBalanceLimits } from "./edits";
+
+export function WhiteBalanceControls() {
+ const document = useDocument();
+ const source = useScene((scene) => scene.source);
+ const selected = useScene((scene) => scene.whiteBalance);
+ const asShot = document.resources.get(source).raw?.asShot;
+ if (!asShot) {
+ return null;
+ }
+ const balance = selected ?? asShot;
+ const limits = whiteBalanceLimits(asShot);
+ return (
+ <>
+ <button
+ type="button"
+ className="cursor-pointer self-end text-neutral-400 text-xs hover:text-neutral-100"
+ onClick={() => setWhiteBalance(document)}
+ >
+ As Shot
+ </button>
+ <Slider
+ label="Temperature (K)"
+ value={balance.temperature}
+ {...limits.temperature}
+ defaultValue={asShot.temperature}
+ onChange={(temperature) => setWhiteBalance(document, { temperature })}
+ />
+ <Slider
+ label="Tint"
+ value={balance.tint}
+ step={0.1}
+ {...limits.tint}
+ defaultValue={asShot.tint}
+ onChange={(tint) => setWhiteBalance(document, { tint })}
+ />
+ </>
+ );
+}
diff --git a/src/features/white-balance/edits.ts b/src/features/white-balance/edits.ts
new file mode 100644
index 0000000..357ce97
--- /dev/null
+++ b/src/features/white-balance/edits.ts
@@ -0,0 +1,44 @@
+import type { EditorDocument } from "@/lib/editor/document";
+import type { WhiteBalance } from "@/lib/image-source";
+
+export function whiteBalanceLimits(asShot: WhiteBalance) {
+ return {
+ temperature: {
+ min: Math.min(2000, asShot.temperature),
+ max: Math.max(25000, asShot.temperature),
+ },
+ tint: {
+ min: Math.min(-150, asShot.tint),
+ max: Math.max(150, asShot.tint),
+ },
+ };
+}
+
+function validateWhiteBalance(balance: WhiteBalance, asShot: WhiteBalance) {
+ const limits = whiteBalanceLimits(asShot);
+ for (const channel of ["temperature", "tint"] as const) {
+ const value = balance[channel];
+ const { min, max } = limits[channel];
+ if (!Number.isFinite(value) || value < min || value > max) {
+ throw Error("Invalid RAW white balance.");
+ }
+ }
+}
+
+/** Without a change, resets to the camera's as-shot balance. */
+export function setWhiteBalance(
+ document: EditorDocument,
+ change?: Partial<WhiteBalance>,
+) {
+ const scene = document.scene.getState();
+ const asShot = document.resources.get(scene.source).raw?.asShot;
+ if (!asShot) {
+ throw Error("This image does not support RAW white balance.");
+ }
+ let whiteBalance = asShot;
+ if (change) {
+ whiteBalance = { ...(scene.whiteBalance ?? asShot), ...change };
+ validateWhiteBalance(whiteBalance, asShot);
+ }
+ document.edit({ ...scene, whiteBalance });
+}
diff --git a/src/lib/adjustments/index.ts b/src/lib/adjustments/index.ts
index bce8658..64d1274 100644
--- a/src/lib/adjustments/index.ts
+++ b/src/lib/adjustments/index.ts
@@ -36,8 +36,8 @@ export function createAdjustments(gpu: Gpu, source: Target) {
});
return {
output,
- render(frame: Frame, adjustments: Adjustments) {
- frame.pass(output, apply.set({ adjustments }));
+ render(frame: Frame, adjustments: Adjustments, input = source) {
+ frame.pass(output, apply.set({ adjustments, source: input.color }));
},
dispose() {
output.color.dispose();
diff --git a/src/lib/decode/index.ts b/src/lib/decode/index.ts
index 562ca85..c86fbb3 100644
--- a/src/lib/decode/index.ts
+++ b/src/lib/decode/index.ts
@@ -1,4 +1,5 @@
import type { Gpu, Target } from "vgpu";
+import { createImageSource, type ImageSource } from "@/lib/image-source";
import { uploadTiff } from "@/lib/tiff-gpu";
import { decodeHeic } from "./heic";
import linearize from "./linearize";
@@ -30,31 +31,77 @@ export function workerDecoder(
type Format = {
types: string[];
extensions: string[];
- load: () => Promise<Decoder>;
+ decode: (gpu: Gpu, file: File) => Promise<ImageSource>;
};
-const native = async () => createImageBitmap;
-const svg = async () => decodeSvg;
-const heic = async () => decodeHeic;
-const tiff = workerDecoder(() => import("./tiff.worker?worker"));
+/** A format whose decoder yields pixels needing the GPU leg into the working space. */
+function pixelFormat(load: () => Promise<Decoder>) {
+ return async (gpu: Gpu, file: File) => {
+ const decoder = await load();
+ const decoded = await decoder(file);
+ return createImageSource(
+ "chunks" in decoded ? uploadTiff(gpu, decoded) : linearize(gpu, decoded),
+ );
+ };
+}
+
+const native = pixelFormat(async () => createImageBitmap);
+const svg = pixelFormat(async () => decodeSvg);
+const heic = pixelFormat(async () => decodeHeic);
+const tiff = pixelFormat(workerDecoder(() => import("./tiff.worker?worker")));
+const raw = async (gpu: Gpu, file: File) =>
+ (await import("@/lib/raw")).decodeRaw(gpu, file);
const formats: Format[] = [
+ {
+ types: ["image/x-adobe-dng", "image/dng"],
+ extensions: [
+ "dng",
+ "cr2",
+ "cr3",
+ "crw",
+ "nef",
+ "nrw",
+ "arw",
+ "sr2",
+ "srf",
+ "raf",
+ "orf",
+ "rw2",
+ "raw",
+ "rwl",
+ "pef",
+ "ptx",
+ "srw",
+ "3fr",
+ "fff",
+ "iiq",
+ "kdc",
+ "dcr",
+ "mos",
+ "mef",
+ "erf",
+ "mrw",
+ "x3f",
+ ],
+ decode: raw,
+ },
{
types: ["image/tiff", "image/x-tiff"],
extensions: ["tif", "tiff"],
- load: tiff,
+ decode: tiff,
},
- { types: ["image/png"], extensions: ["png"], load: native },
- { types: ["image/jpeg"], extensions: ["jpg", "jpeg"], load: native },
- { types: ["image/gif"], extensions: ["gif"], load: native },
- { types: ["image/webp"], extensions: ["webp"], load: native },
- { types: ["image/avif"], extensions: ["avif"], load: native },
- { types: ["image/bmp"], extensions: ["bmp"], load: native },
- { types: ["image/svg+xml"], extensions: ["svg"], load: svg },
+ { types: ["image/png"], extensions: ["png"], decode: native },
+ { types: ["image/jpeg"], extensions: ["jpg", "jpeg"], decode: native },
+ { types: ["image/gif"], extensions: ["gif"], decode: native },
+ { types: ["image/webp"], extensions: ["webp"], decode: native },
+ { types: ["image/avif"], extensions: ["avif"], decode: native },
+ { types: ["image/bmp"], extensions: ["bmp"], decode: native },
+ { types: ["image/svg+xml"], extensions: ["svg"], decode: svg },
{
types: ["image/heic", "image/heif"],
extensions: ["heic", "heif"],
- load: heic,
+ decode: heic,
},
];
@@ -74,14 +121,13 @@ export const accept = formats
export const canDecode = (file: File) => formatOf(file) !== undefined;
/** Decodes the file into a linear rgba16float target: the format's decoder, then its GPU leg. */
-export default async function decode(gpu: Gpu, file: File): Promise<Target> {
+export default async function decode(
+ gpu: Gpu,
+ file: File,
+): Promise<ImageSource> {
const format = formatOf(file);
if (!format) {
throw new Error(`Unsupported image: ${file.name}`);
}
- const decoder = await format.load();
- const decoded = await decoder(file);
- return "chunks" in decoded
- ? uploadTiff(gpu, decoded)
- : linearize(gpu, decoded);
+ return format.decode(gpu, file);
}
diff --git a/src/lib/editor/document/index.ts b/src/lib/editor/document/index.ts
index afd9704..f4ce874 100644
--- a/src/lib/editor/document/index.ts
+++ b/src/lib/editor/document/index.ts
@@ -17,6 +17,7 @@ function equal(a: Scene, b: Scene) {
a.source === b.source &&
shallow(frameValues(a.frame), frameValues(b.frame)) &&
shallow(a.adjustments, b.adjustments) &&
+ shallow(a.whiteBalance, b.whiteBalance) &&
a.toneCurve.length === b.toneCurve.length &&
a.toneCurve.every((point, i) => shallow(point, b.toneCurve[i]))
);
diff --git a/src/lib/editor/document/resources.ts b/src/lib/editor/document/resources.ts
index 7a8c971..c7fed59 100644
--- a/src/lib/editor/document/resources.ts
+++ b/src/lib/editor/document/resources.ts
@@ -1,16 +1,16 @@
-import type { Target } from "vgpu";
+import type { ImageSource } from "@/lib/image-source";
/** Owns the document's image files and GPU targets, outside scene history. */
export function createResources() {
- const images = new Map<string, { file: File; image: Target }>();
+ const images = new Map<string, { file: File } & ImageSource>();
let disposed = false;
return {
- add(file: File, image: Target) {
+ add(file: File, source: ImageSource) {
if (disposed) {
throw new Error("Document is closed.");
}
const id = crypto.randomUUID();
- images.set(id, { file, image });
+ images.set(id, { file, ...source });
return id;
},
get(id: string) {
@@ -21,17 +21,17 @@ export function createResources() {
return resource;
},
retain(ids: ReadonlySet<string>) {
- for (const [id, { image }] of images) {
+ for (const [id, source] of images) {
if (!ids.has(id)) {
- image.color.dispose();
+ source.dispose();
images.delete(id);
}
}
},
dispose() {
disposed = true;
- for (const { image } of images.values()) {
- image.color.dispose();
+ for (const source of images.values()) {
+ source.dispose();
}
images.clear();
},
diff --git a/src/lib/editor/renderer.ts b/src/lib/editor/renderer.ts
index 496362d..e965764 100644
--- a/src/lib/editor/renderer.ts
+++ b/src/lib/editor/renderer.ts
@@ -1,12 +1,18 @@
-import { frame, type Gpu, type Target } from "vgpu";
+import { frame, type Gpu } from "vgpu";
import { createAdjustments } from "@/lib/adjustments";
import type { Scene } from "@/lib/editor/scene";
import { createImageFrame } from "@/lib/image-frame";
+import type { ImageSource, WhiteBalance } from "@/lib/image-source";
import { createToneCurves } from "@/lib/tone-curves";
import { createUnsharpMask } from "@/lib/unsharp-mask";
+function sameBalance(a: WhiteBalance | undefined, b: WhiteBalance | undefined) {
+ return a?.temperature === b?.temperature && a?.tint === b?.tint;
+}
+
/** Owns scene passes and intermediate textures for one decoded source. */
-export function createRenderer(gpu: Gpu, source: Target) {
+export function createRenderer(gpu: Gpu, resource: ImageSource) {
+ const source = resource.image;
const adjust = createAdjustments(gpu, source);
const adjusted = adjust.output;
const toneCurves = createToneCurves(gpu, adjusted);
@@ -14,12 +20,84 @@ export function createRenderer(gpu: Gpu, source: Target) {
const sharpen = createUnsharpMask(gpu, source);
const transform = createImageFrame(gpu);
+ const release = resource.retain();
+ const raw = resource.raw?.createPass();
let original = source;
let input = adjusted;
let fullImage = adjusted;
const listeners = new Set<() => void>();
let rendered = false;
let output = adjusted;
+ let balance = resource.raw?.asShot;
+ let next: Scene | undefined;
+ let pending: Promise<void> | undefined;
+ let disposed = false;
+ function render(scene: Scene) {
+ frame(gpu, (frame) => {
+ const developed = raw?.render() ?? source;
+ adjust.render(frame, scene.adjustments, developed);
+ const curved = toneCurves.render(frame, scene.toneCurve);
+ const { clarity: amount, sharpening, sharpenRadius } = scene.adjustments;
+ const clarified = clarity.render(frame, curved, amount / 200, 64);
+ fullImage = sharpen.render(
+ frame,
+ clarified,
+ sharpening / 50,
+ sharpenRadius,
+ );
+ [original, input, output] = transform.render(
+ frame,
+ [source, adjusted, fullImage],
+ scene.frame,
+ );
+ });
+ rendered = true;
+ for (const listener of listeners) {
+ listener();
+ }
+ }
+ /** Only calibration crosses the worker; each renderer owns its GPU RAW pass. */
+ async function develop() {
+ while (next && !disposed) {
+ const scene = next;
+ next = undefined;
+ const selected = scene.whiteBalance ?? resource.raw?.asShot;
+ if (raw && selected && !sameBalance(balance, selected)) {
+ await raw.prepare(selected);
+ if (disposed) {
+ return;
+ }
+ balance = selected;
+ }
+ if (!next) {
+ render(scene);
+ }
+ }
+ }
+ async function update(scene: Scene): Promise<void> {
+ if (disposed) {
+ throw Error("Renderer is closed.");
+ }
+ if (!resource.raw) {
+ render(scene);
+ return;
+ }
+ next = scene;
+ pending ??= develop()
+ .catch((error) => {
+ if (!next) {
+ throw error;
+ }
+ })
+ .finally(() => {
+ pending = undefined;
+ if (next && !disposed) {
+ return update(next);
+ }
+ });
+ return pending;
+ }
+
return {
originalImage: () => original,
inputImage: () => input,
@@ -34,41 +112,20 @@ export function createRenderer(gpu: Gpu, source: Target) {
listeners.delete(listener);
};
},
- update(scene: Scene) {
- frame(gpu, (frame) => {
- adjust.render(frame, scene.adjustments);
- const curved = toneCurves.render(frame, scene.toneCurve);
- const {
- clarity: amount,
- sharpening,
- sharpenRadius,
- } = scene.adjustments;
- const clarified = clarity.render(frame, curved, amount / 200, 64);
- fullImage = sharpen.render(
- frame,
- clarified,
- // Slider 150 applies a 3× detail gain; radius remains Gaussian sigma.
- sharpening / 50,
- sharpenRadius,
- );
- [original, input, output] = transform.render(
- frame,
- [source, adjusted, fullImage],
- scene.frame,
- );
- });
- rendered = true;
- for (const listener of listeners) {
- listener();
- }
- },
+ update,
dispose() {
+ if (disposed) {
+ return;
+ }
+ disposed = true;
listeners.clear();
adjust.dispose();
toneCurves.dispose();
clarity.dispose();
sharpen.dispose();
transform.dispose();
+ raw?.dispose();
+ release();
},
};
}
diff --git a/src/lib/editor/scene.ts b/src/lib/editor/scene.ts
index 9c2a6bb..5e18901 100644
--- a/src/lib/editor/scene.ts
+++ b/src/lib/editor/scene.ts
@@ -1,5 +1,6 @@
import type { Adjustments as ToneAdjustments } from "@/lib/adjustments";
import type { ImageFrame } from "@/lib/image-frame/geometry";
+import type { WhiteBalance } from "@/lib/image-source";
import type { ToneCurve } from "@/lib/tone-curves/curve";
export type Adjustments = ToneAdjustments & {
@@ -49,6 +50,7 @@ export const adjustmentMinimums: Partial<Adjustments> = {
export type Scene = {
readonly frame: ImageFrame;
readonly source: string;
+ readonly whiteBalance?: Readonly<WhiteBalance>;
readonly adjustments: Readonly<Adjustments>;
readonly toneCurve: ToneCurve;
};
diff --git a/src/lib/image-source.ts b/src/lib/image-source.ts
new file mode 100644
index 0000000..4be6b9a
--- /dev/null
+++ b/src/lib/image-source.ts
@@ -0,0 +1,44 @@
+import type { Target } from "vgpu";
+
+export type WhiteBalance = { temperature: number; tint: number };
+export type RawDevelopment = {
+ asShot: WhiteBalance;
+ createPass(): {
+ prepare(balance: WhiteBalance): Promise<void>;
+ render(): Target;
+ dispose(): void;
+ };
+ dispose(): void;
+};
+
+/** Document content retained while a preview or export still uses it. */
+export function createImageSource(image: Target, raw?: RawDevelopment) {
+ let references = 1;
+ function release() {
+ let active = true;
+ return () => {
+ if (!active) {
+ return;
+ }
+ active = false;
+ if (--references === 0) {
+ image.color.dispose();
+ raw?.dispose();
+ }
+ };
+ }
+ return {
+ image,
+ raw,
+ retain() {
+ if (!references) {
+ throw Error("Image source is closed.");
+ }
+ references++;
+ return release();
+ },
+ dispose: release(),
+ };
+}
+
+export type ImageSource = ReturnType<typeof createImageSource>;
diff --git a/src/lib/raw/index.ts b/src/lib/raw/index.ts
new file mode 100644
index 0000000..35b2207
--- /dev/null
+++ b/src/lib/raw/index.ts
@@ -0,0 +1,62 @@
+import { createRawDecoder, type RawDecoder } from "raw-webgpu";
+import { type Gpu, type Target, target } from "vgpu";
+import { createImageSource } from "@/lib/image-source";
+
+const decoders = new WeakMap<Gpu, RawDecoder>();
+
+/** Adapts package-owned sensor sources to OpenLight's document and renderer lifetimes. */
+export async function decodeRaw(gpu: Gpu, file: File) {
+ let decoder = decoders.get(gpu);
+ if (!decoder) {
+ decoder = createRawDecoder(gpu.gpu);
+ decoders.set(gpu, decoder);
+ }
+ const source = await decoder.load(file);
+ let original: Target | undefined;
+ try {
+ original = target(gpu, { size: source.size, format: "rgba16float" });
+ const initial = source.createDevelopPass();
+ try {
+ initial.render({
+ destination: original.color.gpu,
+ calibration: source.calibration,
+ });
+ } finally {
+ initial.dispose();
+ }
+ return createImageSource(original, {
+ asShot: source.asShot,
+ createPass() {
+ const output = target(gpu, {
+ size: source.size,
+ format: "rgba16float",
+ });
+ const pass = source.createDevelopPass();
+ let calibration = source.calibration;
+ let dirty = true;
+ return {
+ async prepare(balance) {
+ calibration = await source.calibrate(balance);
+ dirty = true;
+ },
+ render() {
+ if (dirty) {
+ pass.render({ destination: output.color.gpu, calibration });
+ dirty = false;
+ }
+ return output;
+ },
+ dispose() {
+ pass.dispose();
+ output.color.dispose();
+ },
+ };
+ },
+ dispose: source.dispose,
+ });
+ } catch (error) {
+ original?.color.dispose();
+ source.dispose();
+ throw error;
+ }
+}
diff --git a/tests/documents.test.ts b/tests/documents.test.ts
index 46a9d1e..3fb5039 100644
--- a/tests/documents.test.ts
+++ b/tests/documents.test.ts
@@ -6,6 +6,7 @@ import { setAdjustments, setToneCurve } from "@/lib/editor/document/edits";
import { createResources } from "@/lib/editor/document/resources";
import { defaultAdjustments } from "@/lib/editor/scene";
import { imageFrame } from "@/lib/image-frame/geometry";
+import { createImageSource } from "@/lib/image-source";
import { defaultCurve } from "@/lib/tone-curves/curve";
function document() {
@@ -113,7 +114,8 @@ test("documents edit independently without React, retain bounded history, and re
const gpu = await init();
const resources = createResources();
const file = new File(["fixture"], "image.png");
- const add = () => resources.add(file, target(gpu, { size: [2, 2] }));
+ const add = () =>
+ resources.add(file, createImageSource(target(gpu, { size: [2, 2] })));
const source = add();
const doc = createDocument(
{ ...document().scene.getState(), source },
diff --git a/tests/editing.e2e.ts b/tests/editing.e2e.ts
index 8ffbf38..f50aa40 100644
--- a/tests/editing.e2e.ts
+++ b/tests/editing.e2e.ts
@@ -42,6 +42,7 @@ test("edit a photo, inspect the preview and histograms, undo changes, and export
page,
browser,
}) => {
+ test.setTimeout(60_000);
const state = () => page.evaluate(() => window.openlight.getState());
await page.goto("/");
await expect(page.getByRole("button", { name: "Open image" })).toBeVisible();
diff --git a/tests/fixtures/raw/README.md b/tests/fixtures/raw/README.md
new file mode 100644
index 0000000..be95c2d
--- /dev/null
+++ b/tests/fixtures/raw/README.md
@@ -0,0 +1,7 @@
+# RAW fixtures
+
+These two synthetic MIT-licensed fixtures are copied unchanged from [raw-webgpu](https://github.com/roprgm/raw-webgpu/tree/7aa3573e23c1cfd71cb4f7227beff82484a9d1fd/tests/fixtures), which owns their generator and codec tests. They contain no camera photographs.
+
+`bayer.dng` stores RGGB samples; `linear-jxl.dng` stores equivalent linear RGB with JPEG XL compression. Both use an sRGB-like camera profile, samples `[32768, 16384, 8192]`, white level 65535 and orientation 6. Expected output is 96×128 with sRGB pixels approximately `[188, 137, 99]`.
+
+OpenLight tests loading, white-balance edits, preview/export agreement, history and resource lifetime with these files.
diff --git a/tests/fixtures/raw/bayer.dng b/tests/fixtures/raw/bayer.dng
new file mode 100644
index 0000000..59f0ae3
Binary files /dev/null and b/tests/fixtures/raw/bayer.dng differ
diff --git a/tests/fixtures/raw/linear-jxl.dng b/tests/fixtures/raw/linear-jxl.dng
new file mode 100644
index 0000000..18a8d3f
Binary files /dev/null and b/tests/fixtures/raw/linear-jxl.dng differ
diff --git a/tests/loaders.test.ts b/tests/loaders.test.ts
index e2a06ee..706e620 100644
--- a/tests/loaders.test.ts
+++ b/tests/loaders.test.ts
@@ -2,6 +2,7 @@ import { expect, test } from "bun:test";
import { createCameraRawXmpLoader } from "@/app/loaders/camera-raw-xmp";
import { createLoaderRegistry, type FileLoader } from "@/app/loaders/registry";
import { createWorkspace } from "@/app/workspace";
+import { canDecode } from "@/lib/decode";
import { createDocument } from "@/lib/editor/document";
import { setAdjustments } from "@/lib/editor/document/edits";
import { defaultAdjustments } from "@/lib/editor/scene";
@@ -18,6 +19,9 @@ function settings(attributes: string, name = "photo.xmp") {
}
test("file batches preserve ordering, group imports, recover from failures, and discard stale settings", async () => {
+ for (const name of ["mamiya.MEF", "epson.erf", "fuji.raf", "iphone.dng"]) {
+ expect(canDecode(new File([], name))).toBe(true);
+ }
const workspace = createWorkspace();
// Substitute decoding only; the registry, workspace, XMP parser, and documents are real.
const image: FileLoader = {
diff --git a/tests/raw.e2e.ts b/tests/raw.e2e.ts
new file mode 100644
index 0000000..3a8e4db
--- /dev/null
+++ b/tests/raw.e2e.ts
@@ -0,0 +1,80 @@
+import { readFile } from "node:fs/promises";
+import { expect, test } from "./fixtures";
+import { readImage, readPreview } from "./images";
+
+test("RAW Bayer and JPEG XL DNG preserve color, orientation, white balance, history and export", async ({
+ page,
+}) => {
+ await page.goto("/");
+ await page.waitForFunction(() => window.openlight);
+ const load = async (name: string) => {
+ const bytes = [...(await readFile(`tests/fixtures/raw/${name}`))];
+ await page.evaluate(
+ ({ bytes, name }) =>
+ window.openlight.loadImage(
+ new File([new Uint8Array(bytes)], name, { type: "image/tiff" }),
+ ),
+ { bytes, name },
+ );
+ };
+ for (const name of ["bayer.dng", "linear-jxl.dng"]) {
+ await test.step(name, async () => {
+ await load(name);
+ const image = await readImage(page);
+ expect(image.size).toEqual([96, 128]);
+ // The fixture's camera profile is sRGB with known linear samples .5, .25, .125.
+ for (const [i, expected] of [188, 137, 99, 255].entries()) {
+ expect(Math.abs(image.center[i] - expected)).toBeLessThanOrEqual(1);
+ }
+ const asShot = await page.evaluate(
+ () => window.openlight.getState().whiteBalance,
+ );
+ expect(asShot?.temperature).toBeGreaterThan(6000);
+ await expect(
+ page.getByRole("slider", { name: "Temperature (K)", exact: true }),
+ ).toBeVisible();
+ await page.evaluate(() =>
+ window.openlight.setWhiteBalance({ temperature: 4000, tint: 20 }),
+ );
+ const cool = await readImage(page);
+ // Full SDK camera calibration, including its white-point normalization.
+ for (const [i, expected] of [192, 161, 166, 255].entries()) {
+ expect(Math.abs(cool.center[i] - expected)).toBeLessThanOrEqual(1);
+ }
+ await expect
+ .poll(async () => (await readPreview(page)).center.slice(0, 3))
+ .toEqual(cool.center.slice(0, 3));
+ await page.evaluate(() => window.openlight.undo());
+ expect((await readImage(page)).center).toEqual(image.center);
+ await page.evaluate(() => window.openlight.redo());
+ expect((await readImage(page)).center).toEqual(cool.center);
+ await page.getByRole("button", { name: "As Shot", exact: true }).click();
+ expect(
+ await page.evaluate(() => window.openlight.getState().whiteBalance),
+ ).toEqual(asShot);
+ expect((await readImage(page)).center).toEqual(image.center);
+ });
+ }
+ // A renderer exporting a captured scene owns its source while the workspace replaces it.
+ const bytes = [...(await readFile("tests/fixtures/raw/bayer.dng"))];
+ const replacement = await page.evaluate(async (bytes) => {
+ window.openlight.setWhiteBalance({ temperature: 4000, tint: 20 });
+ const exporting = window.openlight.exportImage();
+ await window.openlight.loadImage(
+ new File([new Uint8Array(bytes)], "replacement.dng"),
+ );
+ const image = await createImageBitmap(await exporting);
+ const size = [image.width, image.height];
+ image.close();
+ return size;
+ }, bytes);
+ expect(replacement).toEqual([96, 128]);
+ await page.evaluate(() =>
+ window.openlight.loadImage(new File(["invalid RAW"], "broken.dng")),
+ );
+ await expect(
+ page.getByText("Couldn't open broken.dng:", { exact: false }),
+ ).toBeVisible();
+ await load("linear-jxl.dng");
+ expect((await readImage(page)).size).toEqual([96, 128]);
+});
diff --git a/tests/renderer.test.ts b/tests/renderer.test.ts
index 52480c4..b9eec12 100644
--- a/tests/renderer.test.ts
+++ b/tests/renderer.test.ts
@@ -1,19 +1,125 @@
import { expect, mock, test } from "bun:test";
+import type { Target } from "vgpu";
import {
frame,
getMockGPUDeviceInstrumentation,
init,
target,
} from "vgpu/mock";
+import { setWhiteBalance } from "@/features/white-balance/edits";
import { createDocument } from "@/lib/editor/document";
import { setAdjustments, setToneCurve } from "@/lib/editor/document/edits";
+import { createResources } from "@/lib/editor/document/resources";
import { createRenderer } from "@/lib/editor/renderer";
import { defaultAdjustments } from "@/lib/editor/scene";
import { createDisplay } from "@/lib/image-display";
import { imageFrame } from "@/lib/image-frame/geometry";
+import { createImageSource } from "@/lib/image-source";
import { defaultCurve } from "@/lib/tone-curves/curve";
import { createUnsharpMask } from "@/lib/unsharp-mask";
+test("RAW edits coalesce, recover from failure, and retain an exporting source after document replacement", async () => {
+ const gpu = await init();
+ const image = target(gpu, { size: [8, 8], format: "rgba16float" });
+ const asShot = { temperature: 5000, tint: 10 };
+ const requests: ReturnType<typeof Promise.withResolvers<void>>[] = [];
+ const outputs: Target[] = [];
+ const started = Promise.withResolvers<void>();
+ const close = mock(() => {});
+ const develop = mock(() => {
+ const request = Promise.withResolvers<void>();
+ requests.push(request);
+ started.resolve();
+ return request.promise;
+ });
+ const source = createImageSource(image, {
+ asShot,
+ createPass() {
+ const output = target(gpu, { size: image.size, format: image.format });
+ outputs.push(output);
+ return {
+ prepare: (balance) =>
+ balance.temperature === asShot.temperature &&
+ balance.tint === asShot.tint
+ ? Promise.resolve()
+ : develop(),
+ render: () => output,
+ dispose: () => output.color.dispose(),
+ };
+ },
+ dispose: close,
+ });
+ const resources = createResources();
+ const id = resources.add(new File([], "photo.nef"), source);
+ const document = createDocument(
+ {
+ source: id,
+ frame: imageFrame(image.size),
+ adjustments: { ...defaultAdjustments },
+ toneCurve: defaultCurve,
+ whiteBalance: asShot,
+ },
+ resources,
+ );
+ const preview = createRenderer(gpu, source);
+ const exported = createRenderer(gpu, source);
+ const notify = mock(() => {});
+ preview.subscribe(notify);
+ try {
+ // An edit in the same tick as the initial render must not be lost.
+ const initial = preview.update(document.scene.getState());
+ setWhiteBalance(document, { temperature: 2000 });
+ preview.update(document.scene.getState());
+ await started.promise;
+ expect(develop).toHaveBeenCalledTimes(1);
+ setWhiteBalance(document, { temperature: 3000 });
+ preview.update(document.scene.getState());
+ setWhiteBalance(document);
+ preview.update(document.scene.getState());
+ requests[0].resolve();
+ await initial;
+ expect(develop).toHaveBeenCalledTimes(1);
+ expect(notify).toHaveBeenCalledTimes(2);
+ expect(outputs).toHaveLength(2);
+ document.history.undo();
+ expect(document.scene.getState().whiteBalance?.temperature).toBe(3000);
+ const failed = preview.update(document.scene.getState());
+ requests[1].reject(Error("Decode failure"));
+ await expect(failed).rejects.toThrow("Decode failure");
+ const superseded = preview.update(document.scene.getState());
+ setWhiteBalance(document);
+ preview.update(document.scene.getState());
+ requests[2].reject(Error("Superseded failure"));
+ await superseded;
+ setWhiteBalance(document, { temperature: 6500 });
+ const recovered = preview.update(document.scene.getState());
+ requests[3].resolve();
+ await recovered;
+ expect(() => setWhiteBalance(document, { temperature: NaN })).toThrow(
+ "Invalid",
+ );
+ const exporting = exported.update(document.scene.getState());
+ preview.dispose();
+ document.dispose();
+ expect(close).not.toHaveBeenCalled();
+ expect(() => image.color.view).not.toThrow();
+ requests[4].resolve();
+ await exporting;
+ expect(exported.outputImage().size).toEqual([8, 8]);
+ exported.dispose();
+ expect(close).toHaveBeenCalledTimes(1);
+ expect(() => image.color.view).toThrow("destroyed");
+ for (const output of outputs) {
+ expect(() => output.color.view).toThrow("destroyed");
+ }
+ } finally {
+ preview.dispose();
+ exported.dispose();
+ document.dispose();
+ gpu.dispose();
+ }
+});
+
test.each([1, 16])(
"unsharp mask at reduction %s bypasses zero, reuses pipelines, and owns its outputs",
async (reduction) => {
@@ -68,7 +174,8 @@ test("rendering follows grouped edits and undo, reuses pipelines, and releases o
adjustments: { ...defaultAdjustments },
toneCurve: defaultCurve,
});
- const renderer = createRenderer(gpu, source);
+ const resource = createImageSource(source);
+ const renderer = createRenderer(gpu, resource);
const notify = mock(() => {});
const detach = renderer.subscribe(notify);
const unsubscribe = document.scene.subscribe(renderer.update);
@@ -81,6 +188,14 @@ test("rendering follows grouped edits and undo, reuses pipelines, and releases o
);
try {
expect(notify).not.toHaveBeenCalled();
+ await expect(
+ renderer.update({
+ ...document.scene.getState(),
+ get adjustments(): never {
+ throw Error("Render failure");
+ },
+ }),
+ ).rejects.toThrow("Render failure");
renderer.update(document.scene.getState());
const adjusted = renderer.inputImage();
expect(adjusted.size).toEqual(source.size);
@@ -156,6 +271,7 @@ test("rendering follows grouped edits and undo, reuses pipelines, and releases o
detach();
renderer.dispose();
document.dispose();
+ resource.dispose();
gpu.dispose();
}
});