Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.
Python agent-orchestrationai-agentai-coding-assistantclaude-codecoding-agentcomputer-usedesktop-applocal-firstmcppersonal-jarvisvoice-assistantvoice-coding
fix(verse): show space only through glass, keep the hull closed (#993)
Ruben Lütke8b5f8f061a91
diff --git a/docs/verse/realtime.md b/docs/verse/realtime.mdindex 0c10bc2822..9b837ef7a9 100644--- a/docs/verse/realtime.md+++ b/docs/verse/realtime.md@@ -171,7 +171,7 @@ Code: `src/components/society/verse/engine/realtime/`.| `orbitInput.ts` | Mouse: a click on the view takes the mouse (pointer lock); then movement turns the view and the left button works what the crosshair is on; Esc gives it back (the HUD's buttons work again). A drag still turns the view where the lock is refused. Touch and pen: drag turns, pinch zooms, a tap picks. The wheel zooms through the stage, which hands over between third and first person. || `viewSettings.ts` | The person's settings, kept in this browser: sensitivity, inverted look, field of view per person, quality, and the view they last chose. || `eyePick.ts` | Pointer picking at eye level: bodies as upright cylinders (only in front of the deck's geometry), stations near the deck point hit, floor points to walk to. |-| `spaceView.ts`, `spaceShader.ts` | Outside the hull: the Earth, the sun and the stars behind the deck (below). |+| `spaceView.ts`, `spaceShader.ts`, `hullMask.ts` | Outside the hull: the Earth, the sun and the stars behind the deck's glass (below). |The first-person viewmodel (the commander's own arms and, in a duel, theblaster) lives in `src/components/society/verse/viewmodel/` (below).@@ -231,9 +231,23 @@ ghosts visible), then move the values into the code.### Outside the hullThe deck's baked star cards (`*Starfield` materials) are hidden at eye level;-every viewport looks into real space instead, drawn first into the same HDR-target (`SpacePass` before the deck's `RenderPass`, which then keeps the-colour and clears only depth), so bloom and AgX treat it like the deck.+every viewport looks into real space instead, in the same HDR target, so+bloom and AgX treat it like the deck.++- **Only through glass** (`hullMask.ts`, after the deck's `RenderPass`; the+ target is cleared to alpha 0 by a `ClearPass` first). The decks were built+ for the plate camera, so some gaps (above a viewport's header, where a+ ceiling stops short) were never closed; the star cards hid them. The glass+ (every transparent chunk, also on `GLASS_LAYER`) marks its pixels in the+ stencil in one draw; space is then composited *under* the deck+ (`ONE_MINUS_DST_ALPHA, ONE`, front to back: planet, then stars) on pixels+ whose depth is still empty and whose stencil is 1, and the rest of the+ empty pixels get the hull's near black. The planet shader so runs on window+ pixels alone. Every render into the multisampled target resolves it, and+ resolving the stencil is slow on Direct3D: the target keeps+ `resolveStencilBuffer = false`, and the glass is never drawn mesh by mesh.+ The low tier renders to the screen, which has no stencil: there space is+ drawn first, full screen, and the deck over it.- **Camera.** Its own camera at the origin turns with the deck camera andnever moves: the planet is thousands of kilometres away, so walking past adiff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/RealtimeBackdrop.tsx b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/RealtimeBackdrop.tsxindex 56610a00da..52a3a5d315 100644--- a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/RealtimeBackdrop.tsx+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/RealtimeBackdrop.tsx@@ -26,6 +26,7 @@ import {} from "three";import { EffectComposer } from "three/examples/jsm/postprocessing/EffectComposer.js";import { RenderPass } from "three/examples/jsm/postprocessing/RenderPass.js";+import { ClearPass } from "three/examples/jsm/postprocessing/ClearPass.js";import { UnrealBloomPass } from "three/examples/jsm/postprocessing/UnrealBloomPass.js";import { OutputPass } from "three/examples/jsm/postprocessing/OutputPass.js";import type { Deck } from "../../content/deck";@@ -36,7 +37,8 @@ import { loadRealtimeScene, type RealtimeScene } from "./realtimeScene";import { DECK_SHARED } from "./deckMaterial";import { verticalFov, type SegmentTest } from "./shoulderRig";import type { QualityChoice } from "./viewSettings";-import { SpacePass, SpaceView, spaceTextureUrls } from "./spaceView";+import { SpaceView, spaceTextureUrls } from "./spaceView";+import { HullMask, HullMaskPass } from "./hullMask";import { viewmodelVisibility } from "../../viewmodel/motion";import { drawViewmodel, ViewmodelPass } from "../../viewmodel/viewmodelPass";@@ -167,19 +169,25 @@ export function RealtimeBackdrop({ runtime, deck, active, commanderId, quality =Object.assign(window, { __verseSpaceState: space.state });return () => { abort.abort(); space.dispose(); };}, [gl, space]);+ // Space only through glass: other gaps in the deck read as hull.+ const hull = useMemo(() => new HullMask(), []);+ useEffect(() => () => hull.dispose(), [hull]);useEffect(() => {if (loaded?.outward) space.setOutward(loaded.outward);- }, [loaded, space]);+ hull.enabled = (loaded?.glass.length ?? 0) > 0;+ }, [loaded, space, hull]);const { composer, bloom, viewmodelPass } = useMemo(() => {- const target = new WebGLRenderTarget(1, 1, { type: HalfFloatType, samples: 4 });+ const target = new WebGLRenderTarget(1, 1, { type: HalfFloatType, samples: 4, stencilBuffer: true });+ // The hull mask needs the stencil inside the multisampled buffer only; resolving it is slow on Direct3D.+ target.resolveStencilBuffer = false;const c = new EffectComposer(gl, target);- const spacePass = new SpacePass();- spacePass.view = space;- c.addPass(spacePass);+ // Alpha cleared to 0: space is composited under the deck (and its glass) afterwards.+ c.addPass(new ClearPass(new Color(0, 0, 0), 0));const deckPass = new RenderPass(scene, camera);deckPass.clear = false;c.addPass(deckPass);+ c.addPass(new HullMaskPass(hull, scene, camera, space));// First person: the commander's arms over the deck, before bloom and tone mapping like everything else.const vm = new ViewmodelPass();c.addPass(vm);@@ -187,7 +195,7 @@ export function RealtimeBackdrop({ runtime, deck, active, commanderId, quality =c.addPass(pass);c.addPass(new OutputPass());return { composer: c, bloom: pass, viewmodelPass: vm };- }, [gl, scene, camera, space]);+ }, [gl, scene, camera, space, hull]);useEffect(() => () => composer.dispose(), [composer]);useEffect(() => {@@ -292,6 +300,7 @@ export function RealtimeBackdrop({ runtime, deck, active, commanderId, quality =composer.render(dt);} else {gl.setRenderTarget(null);+ // No stencil on the screen: space first, everywhere the deck does not cover.space.render(gl);gl.autoClear = false;gl.render(scene, camera);diff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/hullMask.ts b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/hullMask.tsnew file mode 100644index 0000000000..09f9d5908a--- /dev/null+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/hullMask.ts@@ -0,0 +1,128 @@+/**+ * Space only through glass (docs/verse/realtime.md, "Outside the hull").+ *+ * Runs after the deck: marks the glass, draws space behind it (spaceView's+ * `renderBehind`, only on pixels the deck left empty), then paints the hull.+ *+ * The decks were built for a camera looking down at 60 degrees: above a+ * viewport's header, behind a beam or where a ceiling stops short there can+ * be open gaps that the plate never saw. The old star cards filled them; with+ * real space behind the deck they would look out of the ship. After the deck+ * is drawn, the glass (GLASS_LAYER) marks its pixels in the stencil where it+ * is in front of the deck, and every pixel the deck left empty that no glass+ * covers is painted the dark of the hull.+ *+ * Two draws in all: every render into the multisampled target resolves it,+ * and resolving (stencil above all) is slow on Direct3D, so the glass goes+ * in one draw through its own layer, never one draw per mesh.+ */+import {+ AlwaysStencilFunc, BufferAttribute, BufferGeometry, Color, DoubleSide, KeepStencilOp, LessEqualDepth, Mesh,+ MeshBasicMaterial, NotEqualStencilFunc, OrthographicCamera, PerspectiveCamera, ReplaceStencilOp, ShaderMaterial,+ type Material, type Scene, type WebGLRenderer, type WebGLRenderTarget,+} from "three";+import { GLASS_LAYER } from "../renderOrder";+import type { SpaceView } from "./spaceView";+import { Pass } from "three/examples/jsm/postprocessing/Pass.js";++/** Linear radiance of the hull in the gaps: near black, like a deep shadow. */+const HULL = new Color(0.004, 0.0045, 0.005);++const FILL_VERTEX = /* glsl */ `+void main() {+ // On the far plane: only pixels the deck left empty (depth still 1) pass.+ gl_Position = vec4( position.xy, 1.0, 1.0 );+}+`;++const FILL_FRAGMENT = /* glsl */ `+uniform vec3 hull;+void main() {+ gl_FragColor = vec4( hull, 1.0 );+}+`;++export class HullMask {+ /** Whether the deck has any glass; without any, every gap is hull. */+ enabled = false;+ private readonly maskCamera = new PerspectiveCamera();+ private readonly mark = new MeshBasicMaterial({+ colorWrite: false,+ depthWrite: false,+ depthTest: true,+ side: DoubleSide,+ stencilWrite: true,+ stencilRef: 1,+ stencilFunc: AlwaysStencilFunc,+ stencilZPass: ReplaceStencilOp,+ });+ private readonly fill: Mesh;+ private readonly fillCamera = new OrthographicCamera(-1, 1, 1, -1, 0, 1);++ constructor() {+ const material = new ShaderMaterial({+ vertexShader: FILL_VERTEX,+ fragmentShader: FILL_FRAGMENT,+ uniforms: { hull: { value: HULL } },+ depthTest: true,+ depthWrite: false,+ depthFunc: LessEqualDepth,+ stencilWrite: true,+ stencilRef: 1,+ stencilFunc: NotEqualStencilFunc,+ stencilFail: KeepStencilOp,+ stencilZFail: KeepStencilOp,+ stencilZPass: KeepStencilOp,+ stencilWriteMask: 0,+ });+ const tri = new BufferGeometry();+ tri.setAttribute("position", new BufferAttribute(new Float32Array([-1, -1, 0, 3, -1, 0, -1, 3, 0]), 3));+ this.fill = new Mesh(tri, material);+ this.fill.frustumCulled = false;+ }++ /** Draws into the bound target, which must hold the deck's depth and have a stencil buffer. */+ draw(renderer: WebGLRenderer, scene: Scene, camera: PerspectiveCamera, space: SpaceView): void {+ const autoClear = renderer.autoClear;+ renderer.autoClear = false;+ renderer.clearStencil();+ this.maskCamera.copy(camera);+ this.maskCamera.layers.set(GLASS_LAYER);+ const override = scene.overrideMaterial;+ const background = scene.background;+ if (this.enabled) {+ scene.overrideMaterial = this.mark;+ scene.background = null;+ renderer.render(scene, this.maskCamera);+ scene.overrideMaterial = override;+ scene.background = background;+ }+ space.renderBehind(renderer);+ renderer.render(this.fill, this.fillCamera);+ renderer.autoClear = autoClear;+ }++ dispose(): void {+ this.mark.dispose();+ (this.fill.material as Material).dispose();+ this.fill.geometry.dispose();+ }+}++/** After the deck: space through the glass, hull dark in the other gaps. */+export class HullMaskPass extends Pass {+ constructor(+ private readonly mask: HullMask,+ private readonly scene: Scene,+ private readonly camera: PerspectiveCamera,+ private readonly space: SpaceView,+ ) {+ super();+ this.needsSwap = false;+ }++ render(renderer: WebGLRenderer, _writeBuffer: WebGLRenderTarget, readBuffer: WebGLRenderTarget): void {+ renderer.setRenderTarget(this.renderToScreen ? null : readBuffer);+ this.mask.draw(renderer, this.scene, this.camera, this.space);+ }+}diff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/realtimeScene.ts b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/realtimeScene.tsindex 3b0cbdb954..55a6c31c5a 100644--- a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/realtimeScene.ts+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/realtimeScene.ts@@ -20,7 +20,7 @@ import { MeshoptDecoder } from "three/examples/jsm/libs/meshopt_decoder.module.jimport { MeshBVH } from "three-mesh-bvh";import type { RealtimeContent, RealtimeProbe } from "../../content/deck";import { fetchImageBitmap } from "../imageFetch";-import { REALTIME_LAYER } from "../renderOrder";+import { GLASS_LAYER, REALTIME_LAYER } from "../renderOrder";import { depthOrder, makeDeckMaterial, type ProbeBinding } from "./deckMaterial";export interface RealtimeScene {@@ -34,6 +34,8 @@ export interface RealtimeScene {* deck's middle), or null for a deck without any.*/outward: Vector3 | null;+ /** The see-through chunks (glass, force fields): space shows only through them (hullMask). */+ glass: readonly Mesh[];/** Draw chunks and triangles, for the stats. */chunks: number;triangles: number;@@ -236,6 +238,7 @@ export async function loadRealtimeScene(gl: WebGLRenderer, content: RealtimeContconst order = depthOrder(Object.keys(content.materials));// The baked star cards behind the viewports give way to the real sky (spaceView); the glass shows where they are.const windows: Record<"stars" | "viewport" | "clear", Mesh[]> = { stars: [], viewport: [], clear: [] };+ const glass: Mesh[] = [];group.traverse((object) => {const mesh = object as Mesh;if (!mesh.isMesh) return;@@ -267,12 +270,17 @@ export async function loadRealtimeScene(gl: WebGLRenderer, content: RealtimeContchunks++;triangles += (mesh.geometry.index?.count ?? mesh.geometry.getAttribute("position").count) / 3;if (params.collide) solid.push(mesh);+ if (params.transparent) {+ glass.push(mesh);+ mesh.layers.enable(GLASS_LAYER);+ }});for (const m of new Set(unused)) m.dispose();group.updateMatrixWorld(true);const collider = buildCollider(solid);return {group,+ glass,outward: outwardOf(group, windows.stars.length ? windows.stars : windows.viewport.length ? windows.viewport : windows.clear),materials: [...materials.values()],collider,diff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/spaceView.ts b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/spaceView.tsindex 99ddbb08c8..2c870dbb6b 100644--- a/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/spaceView.ts+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/realtime/spaceView.ts@@ -13,12 +13,11 @@* at about orbital speed.*/import {- AdditiveBlending, BufferAttribute, BufferGeometry, Color, CustomBlending, DataTexture, LinearFilter,- LinearMipmapLinearFilter, Matrix3, Matrix4, Mesh, OneFactor, OneMinusSrcAlphaFactor, PerspectiveCamera, Points,- Quaternion, RedFormat, RepeatWrapping, ClampToEdgeWrapping, RGBAFormat, Scene, ShaderMaterial, SRGBColorSpace, NoColorSpace,- Texture, Vector3, type WebGLRenderer,+ AdditiveBlending, AlwaysDepth, BufferAttribute, BufferGeometry, CustomBlending, DataTexture, EqualStencilFunc, KeepStencilOp,+ LessEqualDepth, LinearFilter, LinearMipmapLinearFilter, Matrix3, Matrix4, Mesh, OneFactor, OneMinusDstAlphaFactor,+ OneMinusSrcAlphaFactor, PerspectiveCamera, Points, Quaternion, RedFormat, RepeatWrapping, ClampToEdgeWrapping, RGBAFormat,+ Scene, ShaderMaterial, SRGBColorSpace, NoColorSpace, Texture, Vector3, type Material, type WebGLRenderer,} from "three";-import { Pass } from "three/examples/jsm/postprocessing/Pass.js";import { fetchImageBitmap } from "../imageFetch";import { SPACE_FRAGMENT, SPACE_VERTEX } from "./spaceShader";@@ -147,6 +146,8 @@ void main() {vColour = colour * gain;gl_PointSize = size * pixelRatio;gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );+ // On the far plane, like the planet: drawn behind the deck, only where it left the depth empty.+ gl_Position.z = gl_Position.w;}`;@@ -193,6 +194,7 @@ export class SpaceView {readonly camera = new PerspectiveCamera(50, 1, 0.01, 200);private readonly planet: ShaderMaterial;private readonly planetMesh: Mesh;+ private readonly starPoints: Points;/** For QA runs: whether the textures arrived, and where the windows look. */readonly state = { ready: false, error: "", outward: [0, 0, 1] };private readonly stars: ShaderMaterial;@@ -260,8 +262,8 @@ export class SpaceView {const points = new Points(starGeometry, this.stars);points.frustumCulled = false;points.renderOrder = 1;+ this.starPoints = points;this.scene.add(points, quad);- this.scene.background = new Color(0, 0, 0);}/** Loads the Earth's textures; until then only the stars and the sun show. */@@ -360,7 +362,29 @@ export class SpaceView {(u.verseInvViewProj.value as Matrix4).copy(this.m4);}- /** Clears the bound target and draws space into it, then clears depth for the deck. */+ /**+ * Behind a deck already drawn into the bound target (alpha cleared to 0, its+ * glass marked 1 in the stencil): space goes only where the deck left the+ * depth empty and glass covers, composited under what is there (front to+ * back: the planet, then the stars it dims). The full-screen planet shader+ * then runs on window pixels alone.+ */+ renderBehind(gl: WebGLRenderer): void {+ const autoClear = gl.autoClear;+ gl.autoClear = false;+ this.planetMesh.renderOrder = 1;+ this.starPoints.renderOrder = 2;+ for (const m of [this.planet, this.stars]) behind(m, true);+ gl.render(this.scene, this.camera);+ for (const m of [this.planet, this.stars]) behind(m, false);+ this.planet.blending = CustomBlending;+ this.stars.blending = AdditiveBlending;+ this.planetMesh.renderOrder = 2;+ this.starPoints.renderOrder = 1;+ gl.autoClear = autoClear;+ }++ /** Clears the bound target and draws space into it, then clears depth for the deck (the path without a stencil). */render(gl: WebGLRenderer): void {const autoClear = gl.autoClear;gl.autoClear = true;@@ -377,18 +401,27 @@ export class SpaceView {}}-/** First pass of the post chain: space into the read buffer; the deck's RenderPass then draws over it without clearing. */-export class SpacePass extends Pass {- view: SpaceView | null = null;-- constructor() {- super();- this.needsSwap = false;- }-- render(renderer: WebGLRenderer, _writeBuffer: unknown, readBuffer: Parameters<WebGLRenderer["setRenderTarget"]>[0]): void {- renderer.setRenderTarget(readBuffer);- if (this.view) this.view.render(renderer);- else renderer.clear();+/** A material's state for drawing under the deck (`renderBehind`), or back to over-black. */+function behind(m: Material, on: boolean): void {+ m.depthTest = on;+ m.depthFunc = on ? LessEqualDepth : AlwaysDepth;+ m.stencilWrite = on; // three runs the stencil test only for materials that "write" it+ m.stencilFunc = EqualStencilFunc;+ m.stencilRef = 1;+ m.stencilWriteMask = 0;+ m.stencilFail = KeepStencilOp;+ m.stencilZFail = KeepStencilOp;+ m.stencilZPass = KeepStencilOp;+ if (on) {+ m.blending = CustomBlending;+ m.blendSrc = OneMinusDstAlphaFactor;+ m.blendDst = OneFactor;+ m.blendSrcAlpha = OneMinusDstAlphaFactor;+ m.blendDstAlpha = OneFactor;+ } else {+ m.blendSrc = OneFactor;+ m.blendDst = OneMinusSrcAlphaFactor;+ m.blendSrcAlpha = null;+ m.blendDstAlpha = null;}}diff --git a/dodaios/ui/web/frontend/src/components/society/verse/engine/renderOrder.ts b/dodaios/ui/web/frontend/src/components/society/verse/engine/renderOrder.tsindex 03a59a7ae4..713bb74246 100644--- a/dodaios/ui/web/frontend/src/components/society/verse/engine/renderOrder.ts+++ b/dodaios/ui/web/frontend/src/components/society/verse/engine/renderOrder.ts@@ -34,3 +34,5 @@ export const XRAY_LAYER = 1;export const DEBUG_LAYER = 2;/** The eye-level deck (realtime geometry); only the eye-level renderer draws it. */export const REALTIME_LAYER = 5;+/** The eye-level deck's see-through chunks again (also on REALTIME_LAYER), for the hull mask alone. */+export const GLASS_LAYER = 6;