Skip to content

Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.

fix(verse): show space only through glass, keep the hull closed (#993)

Ruben Lütke8b5f8f061a91
diff --git a/docs/verse/realtime.md b/docs/verse/realtime.md
index 0c10bc2822..9b837ef7a9 100644
--- a/docs/verse/realtime.md
+++ b/docs/verse/realtime.md
@@ -171,7 +171,7 @@ Code: `src/components/society/verse/engine/realtime/`.
| `orbitInput.ts` | Mouse: a click on the view takes the mouse (pointer lock); then movement turns the view and the left button works what the crosshair is on; Esc gives it back (the HUD's buttons work again). A drag still turns the view where the lock is refused. Touch and pen: drag turns, pinch zooms, a tap picks. The wheel zooms through the stage, which hands over between third and first person. |
| `viewSettings.ts` | The person's settings, kept in this browser: sensitivity, inverted look, field of view per person, quality, and the view they last chose. |
| `eyePick.ts` | Pointer picking at eye level: bodies as upright cylinders (only in front of the deck's geometry), stations near the deck point hit, floor points to walk to. |
-| `spaceView.ts`, `spaceShader.ts` | Outside the hull: the Earth, the sun and the stars behind the deck (below). |
+| `spaceView.ts`, `spaceShader.ts`, `hullMask.ts` | Outside the hull: the Earth, the sun and the stars behind the deck's glass (below). |
The first-person viewmodel (the commander's own arms and, in a duel, the
blaster) lives in `src/components/society/verse/viewmodel/` (below).
@@ -231,9 +231,23 @@ ghosts visible), then move the values into the code.
### Outside the hull
The deck's baked star cards (`*Starfield` materials) are hidden at eye level;
-every viewport looks into real space instead, drawn first into the same HDR
-target (`SpacePass` before the deck's `RenderPass`, which then keeps the
-colour and clears only depth), so bloom and AgX treat it like the deck.
+every viewport looks into real space instead, in the same HDR target, so
+bloom and AgX treat it like the deck.
+
+- **Only through glass** (`hullMask.ts`, after the deck's `RenderPass`; the
+ target is cleared to alpha 0 by a `ClearPass` first). The decks were built
+ for the plate camera, so some gaps (above a viewport's header, where a
+ ceiling stops short) were never closed; the star cards hid them. The glass
+ (every transparent chunk, also on `GLASS_LAYER`) marks its pixels in the
+ stencil in one draw; space is then composited *under* the deck
+ (`ONE_MINUS_DST_ALPHA, ONE`, front to back: planet, then stars) on pixels
+ whose depth is still empty and whose stencil is 1, and the rest of the
+ empty pixels get the hull's near black. The planet shader so runs on window
+ pixels alone. Every render into the multisampled target resolves it, and
+ resolving the stencil is slow on Direct3D: the target keeps
+ `resolveStencilBuffer = false`, and the glass is never drawn mesh by mesh.
+ The low tier renders to the screen, which has no stencil: there space is
+ drawn first, full screen, and the deck over it.
- **Camera.** Its own camera at the origin turns with the deck camera and
never moves: the planet is thousands of kilometres away, so walking past a
diff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/RealtimeBackdrop.tsx b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/RealtimeBackdrop.tsx
index 56610a00da..52a3a5d315 100644
--- a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/RealtimeBackdrop.tsx
+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/RealtimeBackdrop.tsx
@@ -26,6 +26,7 @@ import {
} from "three";
import { EffectComposer } from "three/examples/jsm/postprocessing/EffectComposer.js";
import { RenderPass } from "three/examples/jsm/postprocessing/RenderPass.js";
+import { ClearPass } from "three/examples/jsm/postprocessing/ClearPass.js";
import { UnrealBloomPass } from "three/examples/jsm/postprocessing/UnrealBloomPass.js";
import { OutputPass } from "three/examples/jsm/postprocessing/OutputPass.js";
import type { Deck } from "../../content/deck";
@@ -36,7 +37,8 @@ import { loadRealtimeScene, type RealtimeScene } from "./realtimeScene";
import { DECK_SHARED } from "./deckMaterial";
import { verticalFov, type SegmentTest } from "./shoulderRig";
import type { QualityChoice } from "./viewSettings";
-import { SpacePass, SpaceView, spaceTextureUrls } from "./spaceView";
+import { SpaceView, spaceTextureUrls } from "./spaceView";
+import { HullMask, HullMaskPass } from "./hullMask";
import { viewmodelVisibility } from "../../viewmodel/motion";
import { drawViewmodel, ViewmodelPass } from "../../viewmodel/viewmodelPass";
@@ -167,19 +169,25 @@ export function RealtimeBackdrop({ runtime, deck, active, commanderId, quality =
Object.assign(window, { __verseSpaceState: space.state });
return () => { abort.abort(); space.dispose(); };
}, [gl, space]);
+ // Space only through glass: other gaps in the deck read as hull.
+ const hull = useMemo(() => new HullMask(), []);
+ useEffect(() => () => hull.dispose(), [hull]);
useEffect(() => {
if (loaded?.outward) space.setOutward(loaded.outward);
- }, [loaded, space]);
+ hull.enabled = (loaded?.glass.length ?? 0) > 0;
+ }, [loaded, space, hull]);
const { composer, bloom, viewmodelPass } = useMemo(() => {
- const target = new WebGLRenderTarget(1, 1, { type: HalfFloatType, samples: 4 });
+ const target = new WebGLRenderTarget(1, 1, { type: HalfFloatType, samples: 4, stencilBuffer: true });
+ // The hull mask needs the stencil inside the multisampled buffer only; resolving it is slow on Direct3D.
+ target.resolveStencilBuffer = false;
const c = new EffectComposer(gl, target);
- const spacePass = new SpacePass();
- spacePass.view = space;
- c.addPass(spacePass);
+ // Alpha cleared to 0: space is composited under the deck (and its glass) afterwards.
+ c.addPass(new ClearPass(new Color(0, 0, 0), 0));
const deckPass = new RenderPass(scene, camera);
deckPass.clear = false;
c.addPass(deckPass);
+ c.addPass(new HullMaskPass(hull, scene, camera, space));
// First person: the commander's arms over the deck, before bloom and tone mapping like everything else.
const vm = new ViewmodelPass();
c.addPass(vm);
@@ -187,7 +195,7 @@ export function RealtimeBackdrop({ runtime, deck, active, commanderId, quality =
c.addPass(pass);
c.addPass(new OutputPass());
return { composer: c, bloom: pass, viewmodelPass: vm };
- }, [gl, scene, camera, space]);
+ }, [gl, scene, camera, space, hull]);
useEffect(() => () => composer.dispose(), [composer]);
useEffect(() => {
@@ -292,6 +300,7 @@ export function RealtimeBackdrop({ runtime, deck, active, commanderId, quality =
composer.render(dt);
} else {
gl.setRenderTarget(null);
+ // No stencil on the screen: space first, everywhere the deck does not cover.
space.render(gl);
gl.autoClear = false;
gl.render(scene, camera);
diff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/hullMask.ts b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/hullMask.ts
new file mode 100644
index 0000000000..09f9d5908a
--- /dev/null
+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/hullMask.ts
@@ -0,0 +1,128 @@
+/**
+ * Space only through glass (docs/verse/realtime.md, "Outside the hull").
+ *
+ * Runs after the deck: marks the glass, draws space behind it (spaceView's
+ * `renderBehind`, only on pixels the deck left empty), then paints the hull.
+ *
+ * The decks were built for a camera looking down at 60 degrees: above a
+ * viewport's header, behind a beam or where a ceiling stops short there can
+ * be open gaps that the plate never saw. The old star cards filled them; with
+ * real space behind the deck they would look out of the ship. After the deck
+ * is drawn, the glass (GLASS_LAYER) marks its pixels in the stencil where it
+ * is in front of the deck, and every pixel the deck left empty that no glass
+ * covers is painted the dark of the hull.
+ *
+ * Two draws in all: every render into the multisampled target resolves it,
+ * and resolving (stencil above all) is slow on Direct3D, so the glass goes
+ * in one draw through its own layer, never one draw per mesh.
+ */
+import {
+ AlwaysStencilFunc, BufferAttribute, BufferGeometry, Color, DoubleSide, KeepStencilOp, LessEqualDepth, Mesh,
+ MeshBasicMaterial, NotEqualStencilFunc, OrthographicCamera, PerspectiveCamera, ReplaceStencilOp, ShaderMaterial,
+ type Material, type Scene, type WebGLRenderer, type WebGLRenderTarget,
+} from "three";
+import { GLASS_LAYER } from "../renderOrder";
+import type { SpaceView } from "./spaceView";
+import { Pass } from "three/examples/jsm/postprocessing/Pass.js";
+
+/** Linear radiance of the hull in the gaps: near black, like a deep shadow. */
+const HULL = new Color(0.004, 0.0045, 0.005);
+
+const FILL_VERTEX = /* glsl */ `
+void main() {
+ // On the far plane: only pixels the deck left empty (depth still 1) pass.
+ gl_Position = vec4( position.xy, 1.0, 1.0 );
+}
+`;
+
+const FILL_FRAGMENT = /* glsl */ `
+uniform vec3 hull;
+void main() {
+ gl_FragColor = vec4( hull, 1.0 );
+}
+`;
+
+export class HullMask {
+ /** Whether the deck has any glass; without any, every gap is hull. */
+ enabled = false;
+ private readonly maskCamera = new PerspectiveCamera();
+ private readonly mark = new MeshBasicMaterial({
+ colorWrite: false,
+ depthWrite: false,
+ depthTest: true,
+ side: DoubleSide,
+ stencilWrite: true,
+ stencilRef: 1,
+ stencilFunc: AlwaysStencilFunc,
+ stencilZPass: ReplaceStencilOp,
+ });
+ private readonly fill: Mesh;
+ private readonly fillCamera = new OrthographicCamera(-1, 1, 1, -1, 0, 1);
+
+ constructor() {
+ const material = new ShaderMaterial({
+ vertexShader: FILL_VERTEX,
+ fragmentShader: FILL_FRAGMENT,
+ uniforms: { hull: { value: HULL } },
+ depthTest: true,
+ depthWrite: false,
+ depthFunc: LessEqualDepth,
+ stencilWrite: true,
+ stencilRef: 1,
+ stencilFunc: NotEqualStencilFunc,
+ stencilFail: KeepStencilOp,
+ stencilZFail: KeepStencilOp,
+ stencilZPass: KeepStencilOp,
+ stencilWriteMask: 0,
+ });
+ const tri = new BufferGeometry();
+ tri.setAttribute("position", new BufferAttribute(new Float32Array([-1, -1, 0, 3, -1, 0, -1, 3, 0]), 3));
+ this.fill = new Mesh(tri, material);
+ this.fill.frustumCulled = false;
+ }
+
+ /** Draws into the bound target, which must hold the deck's depth and have a stencil buffer. */
+ draw(renderer: WebGLRenderer, scene: Scene, camera: PerspectiveCamera, space: SpaceView): void {
+ const autoClear = renderer.autoClear;
+ renderer.autoClear = false;
+ renderer.clearStencil();
+ this.maskCamera.copy(camera);
+ this.maskCamera.layers.set(GLASS_LAYER);
+ const override = scene.overrideMaterial;
+ const background = scene.background;
+ if (this.enabled) {
+ scene.overrideMaterial = this.mark;
+ scene.background = null;
+ renderer.render(scene, this.maskCamera);
+ scene.overrideMaterial = override;
+ scene.background = background;
+ }
+ space.renderBehind(renderer);
+ renderer.render(this.fill, this.fillCamera);
+ renderer.autoClear = autoClear;
+ }
+
+ dispose(): void {
+ this.mark.dispose();
+ (this.fill.material as Material).dispose();
+ this.fill.geometry.dispose();
+ }
+}
+
+/** After the deck: space through the glass, hull dark in the other gaps. */
+export class HullMaskPass extends Pass {
+ constructor(
+ private readonly mask: HullMask,
+ private readonly scene: Scene,
+ private readonly camera: PerspectiveCamera,
+ private readonly space: SpaceView,
+ ) {
+ super();
+ this.needsSwap = false;
+ }
+
+ render(renderer: WebGLRenderer, _writeBuffer: WebGLRenderTarget, readBuffer: WebGLRenderTarget): void {
+ renderer.setRenderTarget(this.renderToScreen ? null : readBuffer);
+ this.mask.draw(renderer, this.scene, this.camera, this.space);
+ }
+}
diff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/realtimeScene.ts b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/realtimeScene.ts
index 3b0cbdb954..55a6c31c5a 100644
--- a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/realtimeScene.ts
+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/realtimeScene.ts
@@ -20,7 +20,7 @@ import { MeshoptDecoder } from "three/examples/jsm/libs/meshopt_decoder.module.j
import { MeshBVH } from "three-mesh-bvh";
import type { RealtimeContent, RealtimeProbe } from "../../content/deck";
import { fetchImageBitmap } from "../imageFetch";
-import { REALTIME_LAYER } from "../renderOrder";
+import { GLASS_LAYER, REALTIME_LAYER } from "../renderOrder";
import { depthOrder, makeDeckMaterial, type ProbeBinding } from "./deckMaterial";
export interface RealtimeScene {
@@ -34,6 +34,8 @@ export interface RealtimeScene {
* deck's middle), or null for a deck without any.
*/
outward: Vector3 | null;
+ /** The see-through chunks (glass, force fields): space shows only through them (hullMask). */
+ glass: readonly Mesh[];
/** Draw chunks and triangles, for the stats. */
chunks: number;
triangles: number;
@@ -236,6 +238,7 @@ export async function loadRealtimeScene(gl: WebGLRenderer, content: RealtimeCont
const order = depthOrder(Object.keys(content.materials));
// The baked star cards behind the viewports give way to the real sky (spaceView); the glass shows where they are.
const windows: Record<"stars" | "viewport" | "clear", Mesh[]> = { stars: [], viewport: [], clear: [] };
+ const glass: Mesh[] = [];
group.traverse((object) => {
const mesh = object as Mesh;
if (!mesh.isMesh) return;
@@ -267,12 +270,17 @@ export async function loadRealtimeScene(gl: WebGLRenderer, content: RealtimeCont
chunks++;
triangles += (mesh.geometry.index?.count ?? mesh.geometry.getAttribute("position").count) / 3;
if (params.collide) solid.push(mesh);
+ if (params.transparent) {
+ glass.push(mesh);
+ mesh.layers.enable(GLASS_LAYER);
+ }
});
for (const m of new Set(unused)) m.dispose();
group.updateMatrixWorld(true);
const collider = buildCollider(solid);
return {
group,
+ glass,
outward: outwardOf(group, windows.stars.length ? windows.stars : windows.viewport.length ? windows.viewport : windows.clear),
materials: [...materials.values()],
collider,
diff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/spaceView.ts b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/spaceView.ts
index 99ddbb08c8..2c870dbb6b 100644
--- a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/spaceView.ts
+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/spaceView.ts
@@ -13,12 +13,11 @@
* at about orbital speed.
*/
import {
- AdditiveBlending, BufferAttribute, BufferGeometry, Color, CustomBlending, DataTexture, LinearFilter,
- LinearMipmapLinearFilter, Matrix3, Matrix4, Mesh, OneFactor, OneMinusSrcAlphaFactor, PerspectiveCamera, Points,
- Quaternion, RedFormat, RepeatWrapping, ClampToEdgeWrapping, RGBAFormat, Scene, ShaderMaterial, SRGBColorSpace, NoColorSpace,
- Texture, Vector3, type WebGLRenderer,
+ AdditiveBlending, AlwaysDepth, BufferAttribute, BufferGeometry, CustomBlending, DataTexture, EqualStencilFunc, KeepStencilOp,
+ LessEqualDepth, LinearFilter, LinearMipmapLinearFilter, Matrix3, Matrix4, Mesh, OneFactor, OneMinusDstAlphaFactor,
+ OneMinusSrcAlphaFactor, PerspectiveCamera, Points, Quaternion, RedFormat, RepeatWrapping, ClampToEdgeWrapping, RGBAFormat,
+ Scene, ShaderMaterial, SRGBColorSpace, NoColorSpace, Texture, Vector3, type Material, type WebGLRenderer,
} from "three";
-import { Pass } from "three/examples/jsm/postprocessing/Pass.js";
import { fetchImageBitmap } from "../imageFetch";
import { SPACE_FRAGMENT, SPACE_VERTEX } from "./spaceShader";
@@ -147,6 +146,8 @@ void main() {
vColour = colour * gain;
gl_PointSize = size * pixelRatio;
gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
+ // On the far plane, like the planet: drawn behind the deck, only where it left the depth empty.
+ gl_Position.z = gl_Position.w;
}
`;
@@ -193,6 +194,7 @@ export class SpaceView {
readonly camera = new PerspectiveCamera(50, 1, 0.01, 200);
private readonly planet: ShaderMaterial;
private readonly planetMesh: Mesh;
+ private readonly starPoints: Points;
/** For QA runs: whether the textures arrived, and where the windows look. */
readonly state = { ready: false, error: "", outward: [0, 0, 1] };
private readonly stars: ShaderMaterial;
@@ -260,8 +262,8 @@ export class SpaceView {
const points = new Points(starGeometry, this.stars);
points.frustumCulled = false;
points.renderOrder = 1;
+ this.starPoints = points;
this.scene.add(points, quad);
- this.scene.background = new Color(0, 0, 0);
}
/** Loads the Earth's textures; until then only the stars and the sun show. */
@@ -360,7 +362,29 @@ export class SpaceView {
(u.verseInvViewProj.value as Matrix4).copy(this.m4);
}
- /** Clears the bound target and draws space into it, then clears depth for the deck. */
+ /**
+ * Behind a deck already drawn into the bound target (alpha cleared to 0, its
+ * glass marked 1 in the stencil): space goes only where the deck left the
+ * depth empty and glass covers, composited under what is there (front to
+ * back: the planet, then the stars it dims). The full-screen planet shader
+ * then runs on window pixels alone.
+ */
+ renderBehind(gl: WebGLRenderer): void {
+ const autoClear = gl.autoClear;
+ gl.autoClear = false;
+ this.planetMesh.renderOrder = 1;
+ this.starPoints.renderOrder = 2;
+ for (const m of [this.planet, this.stars]) behind(m, true);
+ gl.render(this.scene, this.camera);
+ for (const m of [this.planet, this.stars]) behind(m, false);
+ this.planet.blending = CustomBlending;
+ this.stars.blending = AdditiveBlending;
+ this.planetMesh.renderOrder = 2;
+ this.starPoints.renderOrder = 1;
+ gl.autoClear = autoClear;
+ }
+
+ /** Clears the bound target and draws space into it, then clears depth for the deck (the path without a stencil). */
render(gl: WebGLRenderer): void {
const autoClear = gl.autoClear;
gl.autoClear = true;
@@ -377,18 +401,27 @@ export class SpaceView {
}
}
-/** First pass of the post chain: space into the read buffer; the deck's RenderPass then draws over it without clearing. */
-export class SpacePass extends Pass {
- view: SpaceView | null = null;
-
- constructor() {
- super();
- this.needsSwap = false;
- }
-
- render(renderer: WebGLRenderer, _writeBuffer: unknown, readBuffer: Parameters<WebGLRenderer["setRenderTarget"]>[0]): void {
- renderer.setRenderTarget(readBuffer);
- if (this.view) this.view.render(renderer);
- else renderer.clear();
+/** A material's state for drawing under the deck (`renderBehind`), or back to over-black. */
+function behind(m: Material, on: boolean): void {
+ m.depthTest = on;
+ m.depthFunc = on ? LessEqualDepth : AlwaysDepth;
+ m.stencilWrite = on; // three runs the stencil test only for materials that "write" it
+ m.stencilFunc = EqualStencilFunc;
+ m.stencilRef = 1;
+ m.stencilWriteMask = 0;
+ m.stencilFail = KeepStencilOp;
+ m.stencilZFail = KeepStencilOp;
+ m.stencilZPass = KeepStencilOp;
+ if (on) {
+ m.blending = CustomBlending;
+ m.blendSrc = OneMinusDstAlphaFactor;
+ m.blendDst = OneFactor;
+ m.blendSrcAlpha = OneMinusDstAlphaFactor;
+ m.blendDstAlpha = OneFactor;
+ } else {
+ m.blendSrc = OneFactor;
+ m.blendDst = OneMinusSrcAlphaFactor;
+ m.blendSrcAlpha = null;
+ m.blendDstAlpha = null;
}
}
diff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/renderOrder.ts b/dodaios/ui/web/frontend/src/components/society/verse/engine/renderOrder.ts
index 03a59a7ae4..713bb74246 100644
--- a/dodaios/ui/web/frontend/src/components/society/verse/engine/renderOrder.ts
+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/renderOrder.ts
@@ -34,3 +34,5 @@ export const XRAY_LAYER = 1;
export const DEBUG_LAYER = 2;
/** The eye-level deck (realtime geometry); only the eye-level renderer draws it. */
export const REALTIME_LAYER = 5;
+/** The eye-level deck's see-through chunks again (also on REALTIME_LAYER), for the hull mask alone. */
+export const GLASS_LAYER = 6;