LordOS — a free, fast and secure GNU/Linux distribution based on Debian, with its own desktop environment: display server and Wayland compositor in C, a React UI engine on QuickJS, and private native apps.
C cdebiandesktop-environmentebpffirewalllandlocklinuxlinux-distributionopen-sourceprivacyquickjsraspberry-pi-5reactsecuritywaylandwayland-compositor
Frame pipeline: while a flip is pending the next frame is painted ahead and submitted on the flip event (fpipe.c: off with LORD_PIPELINE=0, always with 2, by default on while frames take more than 60 % of the refresh period), one display with real page flips on the GPU and CPU paths; a prepared frame is undone and rebuilt when the scene, GPU context, output or drag layer changed, on CPU takeover, hotplug, session switch and DPMS; wl_frame_done and LordBench's frame records at submission, painted/submitted/shown for the synthetic pointer, heldMs and prepared in the phases; fpipe-test, engine-test pipelined drag runs; review fixes (no GPU stop while a flip is pending, no stale frame after the screen was off, no texture catch-up between paint and upload); CHANGELOG, GPU, BENCHMARK
Federico Pereira44b5fabd481d
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex a51909b..7f9a2f6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -533,6 +533,56 @@ of the matching [GitHub release](https://github.com/lordbasex/LordOS/releases),7.5 ms (under 70 %) while the frame took about 13 ms, so the threads, which run at real-time priority, stayed off.The pixels are the same (the pool's bands are exact). On VM 116 headless under load they were on for 0, 10 and 91of 720–721 frames in three runs; the gain on the Pi is not measured yet.+ - **The frame pipeline: the next frame is painted while the flip is pending** (0.8.6, in development, not yet+ measured on the Pi; `docs/GPU.md`, "The frame pipeline"; `src/fpipe.c` and `main.c`). A frame was started only+ after the previous flip completed, so input, JS, layout, paint, the upload, the GPU and the flip ran one after+ another; on the Pi 5 under an all-core load that chain (about 13 ms of CPU, 3–5 ms more) passes the 16.7 ms+ period, every frame waits for the second vblank and the monitor shows 30 frames a second, 98 % late.+ - Now, while the flip of frame N is pending, frame N+1's CPU part is done ahead: pointer input and LordBench's+ scripted step are dispatched, the JS frame callbacks and the layout run and its damage is painted into the+ display's scene (`out_prepare`), and at N's flip event only its upload, `gpu_frame_ex` (or the copy into the dumb+ buffer) and the page flip are left (`out_submit`, through the same `out_present` a frame built at once uses;+ `render_frame` is split into `out_paint` and `out_present`). G0's textures catch up after the submission and+ before the next frame is painted. At most one frame is prepared; with nothing pending a frame is still built and+ flipped at once, so an idle desktop answers input as before.+ - Only with one display that has real page flips, on the GPU and the CPU path; never without a monitor, with a+ driver paced in software, or with several displays (each frame built and submitted at once there, as before).+ - `LORD_PIPELINE`, read once at startup and named in the session log: `0` off (the previous behaviour exactly), `2`+ always, unset (the default) on while frames are heavy: after 2 frames in a row whose CPU time passes 60 % of the+ period, off after 60 in a row under 40 % (`fpipe_note`; `-d` logs each change), because when frames already fit+ a prepared frame shows input one vblank later.+ - The rules: what changes between a frame's paint and its submission goes into the next frame (the prepared one+ is shown as a frame painted at that moment would have been; the cursor and a Linux app's new buffer are still+ taken at submission). A prepare that paints nothing tries nothing more until the flip (no busy loop). A+ prepared frame that can't be submitted as painted (the scene reallocated, the GPU context or output changed, or+ for a G0 frame the drag layer began, ended, changed display or reallocated: the new `paint_g0_epoch`) is undone+ into the next frame's damage, its G0 rectangle included, and that frame is built at once; `cpu_takeover`+ (recording, protected windows for a remote viewer, a failed frame), `gpu_start`, `outputs_apply` (hotplug,+ Settings → Displays) and `display_detach` (a session switch, before the card's master goes) undo it first, and+ so does the screen going off (DPMS), so it is never shown stale when it comes back. Stopping the GPU for a+ recording or protected windows (`cpu_switch`, whose `cpu_takeover` sets the dumb buffer with a modeset) never+ runs in a render that only prepares, while the flip is pending; and nothing is prepared in a render whose+ built frame still has G0's catch-up queued (it would run between that prepared frame's paint and its upload).+ Direct scanout, the remote desktop's read-back, the protected windows' black, `LORD_GPU_VERIFY` and the -2+ path are decided at submission by the same code. The Linux apps' frame callbacks go with a submitted frame,+ never with one only prepared.+ - LordBench records a prepared frame at its submission: its prepare's paint, its submission's upload, swap and+ flip ioctl, both as `renderMs`, and a new `heldMs` (prepared → submitted) with a `prepared` count, in+ `frames.phases` and the `drag (scripted):` line (`… prepared ahead N (held for the flip p50/p99)`); such a+ frame is on time when shown by the first vblank after the wake that submitted it, and the synthetic pointer's+ events end at its own presentation (`lbf_painted`, `lbf_submitted`; `lbf_frame` gets its prepare plus its+ submission).+ - Checked on VM 116: `make -j8` and the changed files with `GPU=0`: no warnings; `make test`: 23 passed, 0 failed,+ 13 skipped; the new `fpipe-test` (39 checks: the switch, the auto mode's hysteresis, every decision, and a+ display on a vblank grid where 13 ms of CPU, 1 ms to submit and 3 ms of GPU give 299 flips in 600 periods+ without the pipeline and 598 with it, a 20 ms frame 299 and 475, and a light frame 599 either way with auto+ never engaging); `engine-test` (6,883 checks, 0 failures) runs every drag case again with each frame presented+ only after the next frame's changes, as a prepared frame is, and all 2,417 frames match+ `drag_golden_gpu.txt`, unchanged, with no change between paint and submission moving the G0 epoch;+ `lordbench-test` (1,197 ok) checks a prepared frame's phases, its class, a late one and the pointer's latency+ across the pipeline. LordBench's desktop test in an isolated headless session with `LORD_PIPELINE=1` and `=0`:+ the same results (60 fps alone and under load, `prepared 0`: no flips without a monitor). VM 116 has no free+ DRM card (lord-sessiond holds its virtio-gpu card), so the pipeline itself has not run on a display yet.- `make -j8` and lord-server with `GPU=0` (no EGL, GBM or GLES linked): no warnings; `make test`: 23 passed, 0failed, 13 skipped, every suite's checks passing; `engine-test`: 4,146 checks, 0 failures.- **`js: unsupported style '<key>'` is logged once per key** (`style_warn` in `ui/js.c`), not on everydiff --git a/docs/BENCHMARK.md b/docs/BENCHMARK.mdindex 1aba865..89115a0 100644--- a/docs/BENCHMARK.md+++ b/docs/BENCHMARK.md@@ -563,14 +563,29 @@ matrix products, and the CPU and the GPU share the Pi's memory, but that has not**What remains.** Under load a frame takes about twice as long to build on the Pi (15–16 ms instead of 7 ms), fora cause not measured yet, which no priority fixes; the Pi kernel at `CONFIG_HZ=1000` (`tools/rpi/build-kernel.sh`, a rebuild) is not built.+**The frame pipeline (in development, not yet measured on the Pi).** A frame that takes 15–16 ms to build cannot+fit with the GPU's part in a 16.7 ms period while the next frame is started only after the flip. Now, while a flip+is pending, the next frame's CPU part (input, JS frame callbacks, layout, paint into the scene) is done ahead, and at+the flip only its upload, the GPU's draw and its flip are left (`docs/GPU.md`, "The frame pipeline"). On by+default while frames are heavy (more than 60 % of the period, 2 frames in a row; off after 60 frames below 40 %),+`LORD_PIPELINE=0` turns it off and `LORD_PIPELINE=2` keeps it on. Only with one display that has real page flips:+without a monitor, with a driver paced in software or with several displays nothing changes. Expected on the Pi+under the load: the frames shown every vblank instead of every other one as long as the CPU part stays under one+period, with about the same pointer latency; that is a model (`build/fpipe-test` drives a display on a vblank+grid: 13 ms of CPU, 1 ms to submit and 3 ms of GPU give 299 flips in 600 periods without it and 598 with it, input+to screen at most 33.3 and 36.0 ms), not a measurement. To measure: LordBench → Desktop impact twice with `LORD_PIPELINE` unset and twice with+`LORD_PIPELINE=0` in the session's environment, comparing `desk.fps`, `desk.missed`, `desk.in50`/`in99` and+`frames.phases.load` (`prepared`, `heldMs`, `onTime`).+#### Frame phases (0.8.6, in development)A frame is started only after the last flip completed (one frame in flight), so any frame whose whole chain (thework before `render()`, the build, the GPU's own part, the flip) takes more than one period is shown a vblank late,and the rate locks at half the refresh. The build time alone (`desk.p50`/`p99`) can't tell which part was too long.So lord-server also records, for every frame the main display shows, while LordBench records or while a scripted-move runs (`bench/lbframes.c`: `lbf_phase_built` when `render()` ends, `lbf_phase_shown` at that frame's flip; all-of it measured by lord-server itself, never by a helper):+move runs (`bench/lbframes.c`: `lbf_phase_built` when the frame is submitted, at the end of `render()` or, for a+frame the pipeline prepared, right after its submission; `lbf_phase_shown` at that frame's flip; all of it measured+by lord-server itself, never by a helper):| Phase (JSON key, `[p50, p99]` ms) | From → to ||---|---|@@ -584,12 +599,21 @@ of it measured by lord-server itself, never by a helper):| `vblankWaitMs` | the later of those two → the flip || `wakeToFlipMs` | the wake → the flip || `catchUpMs` | G0's textures catching up after the flip was queued (`docs/GPU.md`, "Uploads without a stall") |+| `heldMs` | a prepared frame (the frame pipeline, `docs/GPU.md`): its paint done → its submission at the flip before it |++A frame the pipeline prepared (counted in `prepared`) is painted while the previous flip is pending and submitted at+that flip: its `wakeMs` runs from that flip to the wake that submitted it, `beforeRenderMs` is the input and JS+before its prepare, `paintMs` its prepare's, `uploadMs`, `swapMs` and `flipIoctlMs` its submission's, `renderMs` the+two together (its CPU time, not the time between them, which is `heldMs`), and `wakeToFlipMs` runs from its+prepare's wake, so it spans two flips. The synthetic pointer's events applied before its paint end at its own+presentation.The GPU's finish time comes from a native fence taken right after `eglSwapBuffers`(`EGL_ANDROID_native_fence_sync`, `gpu.c`) and read once at the frame's flip with `SYNC_IOC_FILE_INFO`: thekernel's signal time of the sync file, one ioctl a frame, no wait. Fences are taken only while recording. Withoutthe extension `gpuKnown` stays 0. Each frame is also put in one class against the first vblank after it started-(the vblanks fall on the previous flip plus whole periods):+(the vblanks fall on the previous flip plus whole periods; for a prepared frame, after the wake that submitted it,+since its paint was done before):- `onTime`: shown within half a period of that vblank;- `cpuLate`: late, and its flip was queued after that vblank;diff --git a/docs/GPU.md b/docs/GPU.mdindex a922fa9..692ca47 100644--- a/docs/GPU.md+++ b/docs/GPU.md@@ -255,6 +255,88 @@ over full LordBench desktop runs alone and under load, a perf run, a real screen`LORD_SW_CURSOR=1`. Until then no gain is claimed. Near the menu bar or the Dock the frame is stillpainted by the CPU.+## The frame pipeline (0.8.6, in development)++Until now a display's next frame was started only after its page flip completed: pointer input, LordBench's+scripted step, the JS frame callbacks, layout, paint, the texture upload, the GPU's draw and the flip ran one after+another, one frame in flight. When that whole chain takes more than a refresh period, every frame waits for the+second vblank and the rate locks at half the refresh. That is the Raspberry Pi 5 under an all-core load+(`docs/BENCHMARK.md`, "The desktop under load"): about 13 ms to build a frame, 3–5 ms more of GPU and submission,+98 % of the frames late, 30 fps; alone it builds in 7 ms and shows 60.++**What changed** (`main.c`; the decisions in `fpipe.c`, without I/O, tested by `build/fpipe-test`): while the flip+of frame N is pending, the CPU part of frame N+1 is done ahead (it is *prepared*): pointer input and LordBench's+scripted step are dispatched, the JS frame callbacks and the layout run (`paint_ui`) and its damage is painted into+the display's scene (`paint_out`, `out_prepare`), keeping what the submission needs on the display (`Out`): the rect+it painted, its painted rectangles for the CPU path, its G0 result (`g0taken`, `g0`), the cursor and app flags+(`cdirty`, `wl_new`). When N's flip event arrives, only what is left is done for N+1 (`out_submit` →+`out_present`, the same code a frame built at once uses): the texture upload, `gpu_frame_ex` (or the copy into the+dumb buffer on the CPU path) and the page flip. Then G0's textures catch up and N+2 is prepared. The upload copies+the scene when it is called (`glTexSubImage2D` from client memory), so painting the next frame into the scene+right after it is safe; the dumb buffers are written only at submission, when the flip has freed one.++**When it applies** (`pipe_on`): one display with real page flips. Not without a monitor (no flip: the headless+path is unchanged), not when the driver completes flips at once and frames are paced in software (`soft`, a+virtual card: the wait there is the pacing slot, not the flip), and not with several displays (paint.c's drag+layer and G0 state are shared between them; each frame is built and submitted at once there, as before). Both the+GPU and the CPU composition paths use it.++**The switch** `LORD_PIPELINE`, read once at startup (the session log says which): `0` off (exactly the previous+behaviour), `2` always, anything else (and unset, the default) *while frames are heavy*: on after 2 frames in a row+whose CPU time (JS frame callbacks, layout, paint, upload, swap and flip ioctl; for a prepared frame its prepare+plus its submission) is above 60 % of the refresh period, off after 60 frames in a row below 40 % (`fpipe_note`;+`-d` logs each change). The auto mode exists because the pipeline costs a frame of latency when frames already+fit: input that comes after a frame was prepared waits for the next one, shown a vblank later. When frames don't+fit, the latency stays about the same (input waits for at most one flip and is shown two vblanks after it, where+before the flips themselves came every other vblank) and the rate can double.++**The rules for each case:**++- **Idle desktop, input:** with nothing pending, a frame is built and flipped at once, as before. While a flip is+ pending and nothing is prepared, input is dispatched at once (before, it waited for the flip) and, if it damages+ something, painted ahead; only once a frame is prepared (or was tried) for that flip does input wait for it+ (`fpipe_blocked`). At most one frame is prepared at a time.+- **A prepare that paints nothing** (a JS frame callback with no damage) marks the display `FPIPE_EMPTY`: nothing more+ is tried until the flip, and the frame is then built as before, so there is no busy loop.+- **What changes between paint and submission** is not painted into the prepared frame: it is shown as it was+ painted, exactly as a frame built at that moment would have been, and the change goes into the next frame. The+ pointer is the exception it always was: the cursor is drawn at submission (CPU path) or moved on its plane at+ once. A Linux app's new buffer is drawn at submission (GPU path: its texture), so it is never older than before.+- **A prepared frame that can no longer be submitted as painted** is undone (`prep_undo`): what it painted, and the+ window rectangle it left to G0, join the next frame's damage, and that frame is built at once. That happens when+ the scene was reallocated, the GPU context or output changed (`gpu_generation`), or, for a G0 frame, the drag+ layer began, ended, moved to another display or reallocated its buffers (`paint_g0_epoch`, `ui/paint.c`; the+ frame's `GpuG0` points into those buffers). Every path that changes the GPU or the displays also undoes it first:+ `cpu_takeover` (screen recording, a remote viewer with protected windows, a failed frame), `gpu_start`,+ `outputs_apply` (hotplug, Settings → Displays) and `display_detach` (a session switch: the card's master is+ dropped with nothing left prepared). A frame prepared before the screen went off (idle, DPMS) is undone too, so it+ is never shown stale when the screen comes back.+- **Never while the flip is pending:** stopping the GPU for a recording or a remote viewer with protected windows+ (`cpu_switch`: `gpu_shutdown`, and `cpu_takeover` sets the dumb buffer with a modeset) waits for the flip, as it+ did before; a frame prepared for the GPU meanwhile is undone there and painted again on the CPU path.+- **G0:** the prepared frame keeps its `GpuG0` (dirty rectangles, strips, `LORD_GPU_VERIFY`'s reference) until its+ submission, and `paint_g0_done` runs only when its quad is drawn, as before. G0's textures catch up+ (`paint_g0_feed`) after the submission and before the next frame is painted, so the feed never runs between a+ frame's paint and its upload (a frame built at once queues its catch-up for the end of `render`, so nothing is+ prepared in the same call after it). `engine-test` runs every drag case again with each frame presented only after the+ next frame's changes are made ("g0 pipelined") and checks the same golden hashes, and that those changes never+ move the epoch.+- **-2 (unshown)**, direct scanout of a fullscreen app, the remote desktop's read-back and the protected windows'+ black: decided at submission, by the same code as before.+- **Frame callbacks and presentation:** the Linux apps' frame callbacks (`wl_frame_done`) go with a submitted frame,+ never with one only prepared; presentation feedback still follows the main display's flip events.++**LordBench's phases** (`docs/BENCHMARK.md`, "Frame phases") record a prepared frame at its submission:+`paintMs` is its prepare's, `uploadMs`, `swapMs` and `flipIoctlMs` its submission's, `renderMs` both, the new+`heldMs` the time it waited for the flip, and the synthetic pointer's latency ends at its own presentation+(`lbf_painted`, `lbf_submitted`).++**Not measured yet:** the Raspberry Pi 5 with its monitor, the machine this is for: LordBench's Desktop impact+alone and under load with `LORD_PIPELINE` unset and with `LORD_PIPELINE=0`. VM 116 has no free DRM card (its+virtio-gpu card belongs to lord-sessiond, and its flips would be paced in software anyway), so on it only the+headless path ran (unchanged: `prepared 0`), with the unit tests and `engine-test`'s pipelined drag runs. No gain+is claimed until the Pi run.+## The CPU path on several threads (0.8.5)Without a GPU (a virtual machine like VM 116, a board without a working driver) the CPU path paintsdiff --git a/server/Makefile b/server/Makefileindex 363ed1f..e04c057 100644--- a/server/Makefile+++ b/server/Makefile@@ -55,7 +55,7 @@ MEDIA_SRC := src/media/media.c src/media/hwdec.c src/media/mview.c src/media/capSEC_SRC := src/sec/json.c src/sec/sign.c src/sec/pkgstate.c src/sec/perms.c src/sec/cgroup.c src/sec/sandbox.c src/sec/brokersrv.c src/sec/brokercam.c src/sec/brokerkr.c src/sec/brokermail.c src/sec/restrict.c src/sec/appid.c src/sec/pwguard.c src/sec/codesig.c# App mode: lord-server --app <id> runs an .lpk app as an isolated Wayland client (src/app)APP_SRC := src/app/appclient.c src/app/appspawn.c src/app/probe.c src/ui/js_broker.c src/ui/js_brokercam.c src/ui/js_brokermail.c src/app/apphost_bc.c-SRC := src/main.c src/kms.c src/gpu/gpu.c src/input.c src/canvas.c src/par.c src/admin.c src/install/install.c src/apps.c src/lpk.c src/fsops.c src/remote/mirror.c src/remote/remote_admin.c src/remote/ipnet.c src/remote/rtcwire.c src/remote/ssh_admin.c src/remote/ssh_root.c src/users/users_admin.c src/users/homedirs.c src/keyboard.c src/keylag.c src/ime.c src/wl/wl.c src/wl/x11wire.c src/wl/xwm.c src/wl/xwayland.c \+SRC := src/main.c src/fpipe.c src/kms.c src/gpu/gpu.c src/input.c src/canvas.c src/par.c src/admin.c src/install/install.c src/apps.c src/lpk.c src/fsops.c src/remote/mirror.c src/remote/remote_admin.c src/remote/ipnet.c src/remote/rtcwire.c src/remote/ssh_admin.c src/remote/ssh_root.c src/users/users_admin.c src/users/homedirs.c src/keyboard.c src/keylag.c src/ime.c src/wl/wl.c src/wl/x11wire.c src/wl/xwm.c src/wl/xwayland.c \src/ui/node.c src/ui/style.c src/ui/layout.c src/ui/transform.c src/ui/text.c src/ui/image.c \src/ui/paint.c src/ui/js.c src/ui/js_sys.c src/ui/js_term.c src/ui/js_wasm.c src/wasm/wasm.c src/wasm/wasm_run.c src/ui/js_sound.c src/ui/js_net.c src/sound/snd.c src/sound/pulse.c src/ui/otf.c src/ui/fonts.c \src/ui/unicode.c src/ui/bidi.c src/ui/shape.c src/ui/shape_indic.c src/ui/shape_syl.c src/ui/shape_khmer.c src/ui/shape_myanmar.c src/ui/shape_use.c src/ui/thbrk.c src/ui/textlayout.c \@@ -406,6 +406,13 @@ test: build/engine-testbuild/engine-test: tests/engine_test.c $(ENGINE_SRC) src/outputs.c src/bench/lbinput.c build/obj/ui/stb.o $(HDR) tests/drag_golden_cpu.txt tests/drag_golden_gpu.txt$(CC) $(CPPFLAGS) $(CFLAGS) $(WARN) -std=c11 -o $@ tests/engine_test.c $(ENGINE_SRC) src/outputs.c src/bench/lbinput.c build/obj/ui/stb.o -lm -lpthread./$@ || { rm -f $@; exit 1; }+# the frame pipeline's decisions (src/fpipe.c, docs/GPU.md "The frame pipeline"): LORD_PIPELINE, the auto+# mode, what a display does next, when input waits, and a display on a vblank grid with and without it+test: build/fpipe-test+build/fpipe-test: tests/fpipe_test.c src/fpipe.c src/fpipe.h+ @mkdir -p build+ $(CC) $(CPPFLAGS) $(CFLAGS) $(WARN) -std=c11 -o $@ tests/fpipe_test.c src/fpipe.c+ ./$@ || { rm -f $@; exit 1; }# test_drag_exact's golden hashes for this architecture: only on the reference code (001633b plus# step 9 and the corners' restore on the drag layer), never to make a changed frame pass# (tests/engine_test.c, "The dragged window, pixel for pixel")diff --git a/server/src/bench/lbframes.c b/server/src/bench/lbframes.cindex a3bf9b5..ce3583b 100644--- a/server/src/bench/lbframes.c+++ b/server/src/bench/lbframes.c@@ -19,8 +19,10 @@ static int nwork, nshown;static uint32_t period;static double lat[MAXI]; /* ms */static int nlat;-static uint64_t applied[BATCH], inframe[BATCH]; /* events applied, not yet in a frame; in the frame being shown */-static int napplied, ninframe;+/* events applied, not yet in a frame; in a frame painted ahead but not submitted (the frame pipeline); in the+ * frame being shown */+static uint64_t applied[BATCH], painted[BATCH], inframe[BATCH];+static int napplied, npainted, ninframe;static int cmp_d(const void *a, const void *b) { double x = *(const double *)a, y = *(const double *)b; return (x > y) - (x < y); }@@ -33,13 +35,13 @@ static double pct(const double *v, int n, double p) {/* ---- frame phases (lbframes.h) ---- */-enum { PH_WAKE, PH_PRE, PH_PAINT, PH_UP, PH_G0UP, PH_SWAP, PH_SUBC, PH_BUILD, PH_GPU, PH_WAIT, PH_CHAIN, PH_FEED, PH_N };+enum { PH_WAKE, PH_PRE, PH_PAINT, PH_UP, PH_G0UP, PH_SWAP, PH_SUBC, PH_BUILD, PH_GPU, PH_WAIT, PH_CHAIN, PH_FEED, PH_HELD, PH_N };typedef struct {atomic_int on;float v[PH_N][MAXF]; /* ms */int nv[PH_N];int n, on_time, cpu_late, gpu_late, other_late, gpu_known;- int built, par, g0_tried, g0_taken, g0_refused[LBF_G0R], g0_layer[LBF_G0L], g0_full, unshown;+ int built, par, g0_tried, g0_taken, g0_refused[LBF_G0R], g0_layer[LBF_G0L], g0_full, unshown, piped;double up_px, g0_px, feed_px, g0_full_px;} Rec;static Rec rec_lb, rec_drag; /* LordBench's recording, the scripted drag's */@@ -49,7 +51,7 @@ static int have_pend;static void rec_clear(Rec *r) { /* everything but on and the values (nv says how many count) */memset(r->nv, 0, sizeof r->nv);r->n = r->on_time = r->cpu_late = r->gpu_late = r->other_late = r->gpu_known = 0;- r->built = r->par = r->g0_tried = r->g0_taken = r->g0_full = r->unshown = 0;+ r->built = r->par = r->g0_tried = r->g0_taken = r->g0_full = r->unshown = r->piped = 0;memset(r->g0_refused, 0, sizeof r->g0_refused);memset(r->g0_layer, 0, sizeof r->g0_layer);r->up_px = r->g0_px = r->feed_px = r->g0_full_px = 0;@@ -65,6 +67,7 @@ static void rec_built(Rec *r, const LbPhase *p) {r->built++;r->par += p->par != 0;r->unshown += p->unshown != 0;+ r->piped += p->piped != 0;if (p->g0 >= 0) {r->g0_tried++;if (p->g0 > 0) r->g0_taken++;@@ -95,14 +98,23 @@ static double ms_between(uint64_t a, uint64_t b) { return a && b && b >= a ? (dostatic void rec_shown(Rec *r, const LbPhase *p, uint64_t flip, uint64_t gpu) {if (!rec_on(r)) return;r->n++;- if (p->from_flip) rec_put(r, PH_WAKE, ms_between(p->prev_flip, p->wake));- rec_put(r, PH_PRE, ms_between(p->wake, p->render0));+ /* a piped frame: the flip that let it be submitted woke lord-server (swake); its CPU time is its prepare+ * and its submission, and between them it was held for that flip */+ uint64_t start = p->piped ? p->swake : p->wake;+ if (p->piped ? p->sfrom_flip : p->from_flip) rec_put(r, PH_WAKE, ms_between(p->prev_flip, start));+ rec_put(r, PH_PRE, ms_between(p->wake, p->piped ? p->pre_end : p->render0));rec_put(r, PH_PAINT, p->paint_ms);rec_put(r, PH_UP, p->up_ms);if (p->g0 > 0) rec_put(r, PH_G0UP, p->g0_ms);rec_put(r, PH_SWAP, p->swap_ms);rec_put(r, PH_SUBC, p->submit_ms);- rec_put(r, PH_BUILD, ms_between(p->render0, p->built));+ double build = ms_between(p->render0, p->built);+ if (p->piped) {+ double sub = ms_between(p->sub0, p->sub1);+ build = build >= 0 && sub >= 0 ? build + sub : -1;+ rec_put(r, PH_HELD, ms_between(p->built, p->sub0));+ }+ rec_put(r, PH_BUILD, build);uint64_t ready = p->submit;if (gpu) {r->gpu_known++;@@ -111,9 +123,10 @@ static void rec_shown(Rec *r, const LbPhase *p, uint64_t flip, uint64_t gpu) {}rec_put(r, PH_WAIT, ms_between(ready, flip));rec_put(r, PH_CHAIN, ms_between(p->wake, flip));- /* the first vblank after the frame started (the vblanks fall on prev_flip + k periods) */- if (!p->prev_flip || !p->period || !p->wake || p->wake < p->prev_flip) { r->other_late++; return; }- uint64_t k = (p->wake - p->prev_flip) / p->period + 1, target = p->prev_flip + k * p->period;+ /* the first vblank after the frame started (the vblanks fall on prev_flip + k periods); a piped frame is+ * on time when shown by the first vblank after its submission's wake (its paint was done before) */+ if (!p->prev_flip || !p->period || !start || start < p->prev_flip) { r->other_late++; return; }+ uint64_t k = (start - p->prev_flip) / p->period + 1, target = p->prev_flip + k * p->period;if (flip <= target + p->period / 2) r->on_time++;else if (p->submit > target) r->cpu_late++;else if (gpu && gpu > target) r->gpu_late++;@@ -154,6 +167,7 @@ static void rec_stats(const Rec *r, LbPhaseStats *st) {st->wait = rec_pct(r, PH_WAIT);st->chain = rec_pct(r, PH_CHAIN);st->feed = rec_pct(r, PH_FEED);+ st->held = rec_pct(r, PH_HELD);int b = r->built ? r->built : 1;st->up_mpx = r->up_px / b / 1e6;st->g0_mpx = r->g0_px / b / 1e6;@@ -171,6 +185,7 @@ static void rec_stats(const Rec *r, LbPhaseStats *st) {st->g0_full = r->g0_full;st->g0_full_px = (long)r->g0_full_px;st->unshown = r->unshown;+ st->piped = r->piped;}/* lbf_begin and lbf_end start and stop LordBench's recorder with the frame times */@@ -232,7 +247,8 @@ void lbf_phases_text(const LbPhaseStats *st, char *buf, size_t n) {if (!f2) PUT("]");}}- PUT("), whole G0 uploads %d (%.2f Mpx), unshown %d", st->g0_full, st->g0_full_px / 1e6, st->unshown);+ PUT("), whole G0 uploads %d (%.2f Mpx), unshown %d; prepared ahead %d (held for the flip %.2f/%.2f)", st->g0_full,+ st->g0_full_px / 1e6, st->unshown, st->piped, st->held.p50, st->held.p99);#undef PUTif (n) buf[k < n ? k : n - 1] = 0;}@@ -263,19 +279,45 @@ void lbf_input(uint64_t ev_ns) {pthread_mutex_unlock(&mu);}+/* n events of from[] join to[] (*nto of them), the newest kept when it is full */+static void move_events(uint64_t *to, int *nto, const uint64_t *from, int n) {+ for (int i = 0; i < n; i++) {+ if (*nto == BATCH) { memmove(to, to + 1, sizeof(uint64_t) * (BATCH - 1)); (*nto)--; }+ to[(*nto)++] = from[i];+ }+}+void lbf_built(int main_drawn) {if (!main_drawn || !atomic_load_explicit(&on, memory_order_relaxed)) return;pthread_mutex_lock(&mu);- /* a frame built while the previous one is still waiting for its flip never happens on the main display- * (main.c waits for the flip); if it did, both groups end at this frame's presentation */- for (int i = 0; i < napplied && ninframe < BATCH; i++) inframe[ninframe++] = applied[i];+ /* a frame is built and submitted only once the previous one was shown (main.c waits for the flip, or+ * prepares the next one ahead: lbf_painted, lbf_submitted); if one were not, both groups would end at+ * this frame's presentation */+ move_events(inframe, &ninframe, painted, npainted);+ move_events(inframe, &ninframe, applied, napplied);+ npainted = napplied = 0;+ pthread_mutex_unlock(&mu);+}++void lbf_painted(void) {+ if (!atomic_load_explicit(&on, memory_order_relaxed)) return;+ pthread_mutex_lock(&mu);+ move_events(painted, &npainted, applied, napplied);napplied = 0;pthread_mutex_unlock(&mu);}+void lbf_submitted(void) {+ if (!atomic_load_explicit(&on, memory_order_relaxed)) return;+ pthread_mutex_lock(&mu);+ move_events(inframe, &ninframe, painted, npainted);+ npainted = 0;+ pthread_mutex_unlock(&mu);+}+void lbf_begin(void) {pthread_mutex_lock(&mu);- nwork = nshown = nlat = napplied = ninframe = 0;+ nwork = nshown = nlat = napplied = npainted = ninframe = 0;period = 0;atomic_store(&on, 1);pthread_mutex_unlock(&mu);diff --git a/server/src/bench/lbframes.h b/server/src/bench/lbframes.hindex 90423b0..15e9b04 100644--- a/server/src/bench/lbframes.h+++ b/server/src/bench/lbframes.h@@ -30,14 +30,20 @@ void lbf_presented(uint64_t ns, uint32_t period_ns);* applied so far are in that frame and their latency ends at its presentation */void lbf_input(uint64_t ev_ns);void lbf_built(int main_drawn);+/* the frame pipeline (fpipe.h): a frame of the main display painted ahead, while the previous one's flip is+ * pending (the events applied so far are in it), then submitted at that flip (they end at its presentation).+ * lbf_built is both at once; events of a prepared frame that is dropped go with the next frame submitted. */+void lbf_painted(void);+void lbf_submitted(void);/* lbrun.c: begin clears and starts; end stops and fills st (0, or -1 with fewer than 2 shown frames) */void lbf_begin(void);int lbf_end(LbFrameStats *st);/* ---- where a frame's time goes (docs/BENCHMARK.md, "Frame phases") ----- * main.c fills one LbPhase for each frame the main display gets, when render() ends (lbf_phase_built),- * and completes it at that frame's presentation (lbf_phase_shown). Times are CLOCK_MONOTONIC ns, 0 =+ * main.c fills one LbPhase for each frame the main display gets, when it is submitted (lbf_phase_built: at+ * the end of render(), or for a frame the pipeline prepared, right after its submission), and completes it at+ * that frame's presentation (lbf_phase_shown). Times are CLOCK_MONOTONIC ns, 0 =* unknown. All measured by lord-server itself; nothing comes from a helper. Two recorders take them:* LordBench's (lbf_begin/lbf_end, lbf_phases) and the scripted drag's (lbf_drag_begin/lbf_drag_end), for* its "drag (scripted):" line. Off, each call is one atomic read. */@@ -61,6 +67,13 @@ typedef struct {int g0; /* 1 G0 composed it, 0 refused (g0_why, g0_sub), -1 not tried */int g0_why, g0_sub;int unshown; /* gpu_frame_ex's -2: nothing shown, a vblank lost */+ /* the frame pipeline (fpipe.h): piped = painted ahead while the previous flip was pending, then submitted+ * at that flip. Then wake/from_flip/render0/built are its prepare's (render0..built: JS frame callbacks,+ * layout and paint), pre_end is when that render() started (the end of "before render"), swake and+ * sfrom_flip are the wake whose flip let it be submitted, sub0..sub1 its submission (upload, swap, flip+ * ioctl), and prev_flip is the presentation just before that submission */+ int piped, sfrom_flip;+ uint64_t pre_end, swake, sub0, sub1;} LbPhase;typedef struct { double p50, p99; } LbPct;typedef struct {@@ -73,6 +86,7 @@ typedef struct {LbPct wait; /* the later of the two to the flip: the vblank */LbPct chain; /* the wake to the flip */LbPct feed;+ LbPct held; /* piped frames: prepared until submitted (the previous flip's wait) */double up_mpx, g0_mpx, feed_mpx; /* pixels sent per frame, average, millions */int on_time; /* shown by the first vblank after the frame started (+ half a period) */int cpu_late; /* late, and queued after that vblank */@@ -83,6 +97,7 @@ typedef struct {int g0_tried, g0_taken, g0_refused[LBF_G0R], g0_layer[LBF_G0L], g0_full;long g0_full_px;int unshown;+ int piped; /* frames prepared ahead (fpipe.h) */} LbPhaseStats;void lbf_phase_built(const LbPhase *p);void lbf_phase_shown(uint64_t flip_ns, uint64_t gpu_done_ns);diff --git a/server/src/bench/lbrun.c b/server/src/bench/lbrun.cindex b48531a..f6abfe5 100644--- a/server/src/bench/lbrun.c+++ b/server/src/bench/lbrun.c@@ -143,14 +143,15 @@ static void phases_json(HBuf *b, const LbPhaseStats *p) {{ "wakeMs", p->wake }, { "beforeRenderMs", p->pre }, { "renderMs", p->build }, { "paintMs", p->paint },{ "uploadMs", p->up }, { "g0UploadMs", p->g0up }, { "swapMs", p->swap }, { "flipIoctlMs", p->submit_call },{ "gpuMs", p->gpu }, { "vblankWaitMs", p->wait }, { "wakeToFlipMs", p->chain }, { "catchUpMs", p->feed },+ { "heldMs", p->held },};hb_printf(b, "{\"frames\":%d", p->n);for (size_t i = 0; i < sizeof t / sizeof *t; i++) hb_printf(b, ",\"%s\":[%.3f,%.3f]", t[i].k, t[i].v.p50, t[i].v.p99);hb_printf(b, ",\"uploadMpx\":%.4f,\"g0UploadMpx\":%.4f,\"catchUpMpx\":%.4f,\"onTime\":%d,\"cpuLate\":%d,\"gpuLate\":%d,"- "\"otherLate\":%d,\"gpuKnown\":%d,\"paintThreads\":%d,\"unshown\":%d,\"g0\":{\"tried\":%d,\"taken\":%d,"+ "\"otherLate\":%d,\"gpuKnown\":%d,\"paintThreads\":%d,\"unshown\":%d,\"prepared\":%d,\"g0\":{\"tried\":%d,\"taken\":%d,""\"wholeUploads\":%d,\"wholeMpx\":%.3f,\"refused\":{",p->up_mpx, p->g0_mpx, p->feed_mpx, p->on_time, p->cpu_late, p->gpu_late, p->other_late, p->gpu_known, p->par,- p->unshown, p->g0_tried, p->g0_taken, p->g0_full, p->g0_full_px / 1e6);+ p->unshown, p->piped, p->g0_tried, p->g0_taken, p->g0_full, p->g0_full_px / 1e6);for (int i = 0, k = 0; i < LBF_G0R; i++)if (p->g0_refused[i] && lbf_g0r_name(i)) hb_printf(b, "%s\"%s\":%d", k++ ? "," : "", lbf_g0r_name(i), p->g0_refused[i]);hb_printf(b, "},\"layer\":{");diff --git a/server/src/fpipe.c b/server/src/fpipe.cnew file mode 100644index 0000000..f4a9924--- /dev/null+++ b/server/src/fpipe.c@@ -0,0 +1,32 @@+/* fpipe.c — see fpipe.h. */+#include "fpipe.h"++#include <string.h>++void fpipe_init(FPipe *p, const char *env) {+ memset(p, 0, sizeof *p);+ p->mode = env && !strcmp(env, "0") ? FPIPE_OFF : env && !strcmp(env, "2") ? FPIPE_ALWAYS : FPIPE_AUTO;+}++void fpipe_note(FPipe *p, double cpu_ms, double period_ms) {+ if (p->mode != FPIPE_AUTO || period_ms <= 0) return;+ if (!p->on) {+ p->heavy = cpu_ms > period_ms * 0.6 ? p->heavy + 1 : 0;+ if (p->heavy >= 2) { p->on = 1; p->heavy = p->light = 0; }+ return;+ }+ p->light = cpu_ms < period_ms * 0.4 ? p->light + 1 : 0;+ if (p->light >= 60) { p->on = 0; p->heavy = p->light = 0; }+}++int fpipe_engaged(const FPipe *p) { return p->mode == FPIPE_ALWAYS || (p->mode == FPIPE_AUTO && p->on); }++int fpipe_step(int eligible, int pending, int prep, int needs, int paint) {+ if (!pending) {+ if (prep == FPIPE_READY) return FPIPE_SUBMIT;+ return needs || paint ? FPIPE_BUILD : FPIPE_WAIT;+ }+ return eligible && prep == FPIPE_NONE && paint ? FPIPE_PREPARE : FPIPE_WAIT;+}++int fpipe_blocked(int eligible, int pending, int prep) { return pending && (!eligible || prep != FPIPE_NONE); }diff --git a/server/src/fpipe.h b/server/src/fpipe.hnew file mode 100644index 0000000..7d3a002--- /dev/null+++ b/server/src/fpipe.h@@ -0,0 +1,50 @@+/* fpipe.h — the frame pipeline's decisions (docs/GPU.md, "The frame pipeline"). Without it a display's next+ * frame is built (input, JS, layout, paint) only once its page flip is done, so building, the GPU's part+ * and the flip run one after another and a frame that takes more than a refresh period in all waits for the+ * second vblank. With it, while the flip of frame N is pending the CPU part of frame N+1 is done ahead+ * ("prepared": painted into the display's scene), and at N's flip only the upload, the GPU's draw and the+ * page flip of that frame remain. main.c keeps the per-display state; the rules are here, without I/O, so+ * tests/fpipe_test.c can check them. */+#ifndef LORD_FPIPE_H+#define LORD_FPIPE_H++/* LORD_PIPELINE, read once at startup: 0 off (every frame is built after the flip, as before 0.8.6),+ * 2 always, anything else (and unset) auto: on while frames are heavy (fpipe_note) */+enum { FPIPE_OFF, FPIPE_AUTO, FPIPE_ALWAYS };+typedef struct {+ int mode;+ int on; /* auto: engaged */+ int heavy, light; /* heavy frames in a row (to engage), light ones (to let go) */+} FPipe;+void fpipe_init(FPipe *p, const char *env);+/* after each frame the main display got: the CPU time it took (input excluded: JS frame callbacks,+ * layout, paint, the upload and the flip's submission) against the refresh period. Auto engages after+ * 2 frames in a row above 60 % of the period (where a frame plus the GPU's part stops fitting before+ * the next vblank) and lets go after 60 in a row below 40 %, like the paint threads (main.c). */+void fpipe_note(FPipe *p, double cpu_ms, double period_ms);+int fpipe_engaged(const FPipe *p);++/* A display's prepared frame */+enum {+ FPIPE_NONE, /* nothing prepared */+ FPIPE_READY, /* painted into the scene, waiting for the pending flip to be submitted */+ FPIPE_EMPTY, /* tried during this pending flip and nothing was painted: the next+ * frame is built after the flip, as without the pipeline */+};+/* What a display does now. eligible: the pipeline applies to it (engaged, one display with a real page+ * flip: not headless, not paced in software); pending: its flip is pending; prep: FPIPE_*; needs: something+ * to show (damage, the cursor, a Linux app's new frame, a fullscreen app); paint: work the CPU can do+ * ahead (UI damage, layout or a JS frame callback waiting). */+enum {+ FPIPE_WAIT, /* nothing now (or wait for the flip) */+ FPIPE_BUILD, /* build and submit a frame at once (as without the pipeline) */+ FPIPE_SUBMIT, /* submit the prepared frame: only upload, GPU and flip are left */+ FPIPE_PREPARE, /* paint the next frame while the flip is pending */+};+int fpipe_step(int eligible, int pending, int prep, int needs, int paint);+/* Pointer input and LordBench's scripted step wait for this display's flip (they belong to the next+ * frame, and only one is prepared at a time): without the pipeline while its flip is pending, with it+ * only once a frame is prepared (or was tried) for that flip. */+int fpipe_blocked(int eligible, int pending, int prep);++#endifdiff --git a/server/src/main.c b/server/src/main.cindex 3e3718e..a43768b 100644--- a/server/src/main.c+++ b/server/src/main.c@@ -57,6 +57,7 @@#include "sec/codesig.h"#include "sec/netconn.h"#include "bench/lbframes.h"+#include "fpipe.h"#include <libgen.h>#include <limits.h>@@ -164,6 +165,17 @@ typedef struct {int g0taken, g0on, g0cpu;Rect g0w;GpuG0 g0;+ int pg0, pg0why, pg0sub; /* what the frame painted last got from G0 (paint_g0_last), until it is submitted */+ /* the frame pipeline (fpipe.h, docs/GPU.md "The frame pipeline"): a frame painted into the scene while the+ * flip was pending (prep, an FPIPE_*), what it painted (prep_r; prep_whole: all of it) and what it was painted+ * against, checked before it is submitted (the scene's pixels, the GPU context and output, paint.c's G0+ * epoch); its CPU time so far and its phases for LordBench (main display) */+ int prep, prep_whole, prep_gpu;+ Rect prep_r;+ const uint32_t *prep_px;+ unsigned prep_gen, prep_epoch;+ double prep_ms;+ LbPhase pph;} Out;typedef struct {@@ -221,6 +233,25 @@ static void g0_stale(Out *o) {o->g0on = 0;}+/* The frame pipeline: a prepared frame that will not be submitted (the GPU, the displays or G0's layer changed+ * under it). What it painted is in the scene but not on the screen, so it joins the next frame's damage (with+ * the rectangle it left to G0, stale in the scene). */+static void prep_undo(Out *o) {+ if (o->prep == FPIPE_READY) {+ Rect r = o->prep_r;+ if (o->g0taken) r = rect_join(r, o->g0.w);+ if (o->prep_whole) o->full = 1;+ else if (r.x1 > r.x0 && r.y1 > r.y0) {+ float x0, y0, x1, y1;+ out_logical_box(&o->v, r, &x0, &y0, &x1, &y1);+ if (box_empty(o->pend)) o->pl.n = 0;+ o->pend = box_join(o->pend, (Box){ x0, y0, x1, y1 });+ }+ o->g0taken = 0;+ }+ o->prep = FPIPE_NONE;+}+/* ---- the displays ---- */static int out_count(const Server *s) {@@ -233,6 +264,9 @@ static Out *main_out(Server *s) { return &s->out[s->out[s->main_out].on ? s->maistatic int flip_instant, flip_told;static double pace_next; /* the next frame's slot when frames are paced in software, ms */+static FPipe fpipe; /* LORD_PIPELINE (fpipe.h), read once at startup */+static void prep_undo_all(Server *s) { for (int i = 0; i < OUT_MAX; i++) prep_undo(&s->out[i]); }+static int any_flip_pending(const Server *s) {for (int i = 0; i < OUT_MAX; i++) if (s->out[i].on && s->out[i].kms.flip_pending) return 1;return 0;@@ -614,6 +648,7 @@ static int gpu_allowed(const Server *s) {/* The GPU stopped: every display gets its dumb buffer back (CPU path). */static void cpu_takeover(Server *s) {+ prep_undo_all(s); /* a frame prepared for the GPU is painted again */for (int i = 0; i < OUT_MAX; i++) {Out *o = &s->out[i];if (!o->on || o->gpu < 0) continue;@@ -693,6 +728,7 @@ static void outputs_apply(Server *s) {!memcmp(&a->v, &b->v, sizeof a->v) && !strcmp(a->id, b->id);}if (same) { display_resync_done(1, &p); return; }+ prep_undo_all(s); /* (outputs: views, frames and the GPU change below) */if (gpu_active()) { gpu_tex_free(s->cursor_tex); s->cursor_tex = NULL; gpu_shutdown(); }cpu_takeover(s);kms_cursor_show(k0, 0);@@ -874,6 +910,7 @@ static void display_detach(void) {parked_drop();splash_wait = 0;if (s->headless) return;+ prep_undo_all(s); /* a prepared frame is dropped with its display */gpu_tex_free(s->cursor_tex);s->cursor_tex = NULL;gpu_shutdown();@@ -1252,6 +1289,7 @@ static void update_cursor(Server *s) {/* GPU (Mesa) on the session's card; without a usable one, the CPU path goes on. */static int gpu_start(Server *s) {if (!gpu_allowed(s) || gpu_init(K0(s))) return -1;+ prep_undo_all(s); /* frames prepared for the CPU path (full_damage below) */static uint32_t arrow[64 * 64];arrow_argb(arrow, 64, 64);for (int i = 0; i < OUT_MAX; i++) s->out[i].gpu = -1;@@ -1294,9 +1332,13 @@ static uint64_t loop_wake;static int loop_wake_flip;static LbPhase fph;static int fph_main;+/* the frame pipeline: when render() started (a prepared frame's "before render" ends there), and whether this+ * render() submitted a prepared frame of the main display (its work and phases were recorded there) */+static uint64_t render_r0;+static int call_piped;-static void render_frame(Server *s);-static void g0_feed_now(Server *s);+static int render_frame(Server *s);+static void g0_feed_now(Server *s, LbPhase *ph);static void render(Server *s) {int phases = lbf_recording();gpu_fence_on(phases); /* the GPU's finish time of each frame, only while recording */@@ -1304,11 +1346,14 @@ static void render(Server *s) {double t0 = now_ms();fph = (LbPhase){ .g0 = -1, .g0_why = -1, .g0_sub = -1 };fph_main = 0;- render_frame(s);+ render_r0 = r0;+ call_piped = 0;+ int shown = render_frame(s); /* 0: it only prepared the next frame (the frame pipeline) */double t1 = now_ms();uint64_t r1 = phases ? mono_ns() : 0;- lbf_frame(t1 - t0); /* LordBench's compositor measurement (bench/lbframes.c) */- g0_feed_now(s); /* after the frame's time: its flip is queued already */+ /* LordBench's compositor measurement (bench/lbframes.c); a prepared frame's goes at its submission */+ if (shown && !call_piped) lbf_frame(t1 - t0);+ g0_feed_now(s, &fph); /* after the frame's time: its flip is queued already */if (phases && fph_main) {fph.wake = loop_wake;fph.from_flip = loop_wake_flip;@@ -1317,7 +1362,7 @@ static void render(Server *s) {lbf_phase_built(&fph);if (s->headless) lbf_phase_shown(fph.submit, 0);}- if (dstat.active) {+ if (dstat.active && shown) {double js = dstat.js - dstat.work_js;double work = (t1 - t0) + js;dstat.work_js = dstat.js;@@ -1521,19 +1566,248 @@ static int g0_feed_up(void *ctx, int which, const uint32_t *px, int w, int h, Re/* the displays whose G0 textures catch up after this frame: after its flips are queued and its time is* taken, while the CPU would wait for the vblank anyway (docs/GPU.md, "Uploads without a stall") */static int feed_idx[OUT_MAX], nfeed;-static void g0_feed_now(Server *s) {+static void g0_feed_now(Server *s, LbPhase *ph) {for (int k = 0; k < nfeed; k++) {Out *o = &s->out[feed_idx[k]];if (!o->on || o->gpu < 0 || !gpu_g0_ready(o->gpu) || paint_g0_backlog(feed_idx[k]) <= 0) continue;double f0 = now_ms();long sent = paint_g0_feed(feed_idx[k], G0_FEED_PX, g0_feed_up, &o->gpu);- if (o == main_out(s)) { fph.feed_ms = now_ms() - f0; fph.feed_px = sent; }+ if (o == main_out(s)) { ph->feed_ms = now_ms() - f0; ph->feed_px = sent; }}nfeed = 0;}-static void render_frame(Server *s) {+/* the frame pipeline (fpipe.h): it applies to this display now. One display with real page flips: without a+ * monitor there is no flip, a driver paced in software waits for its slot rather than the flip, and with+ * several displays paint.c's drag layer and G0 state are shared between them (each frame is built and+ * submitted at once there, as before). */+static int pipe_on(const Server *s, const Out *o) {+ return fpipe_engaged(&fpipe) && !s->headless && flip_instant < 3 && o->on && out_count(s) == 1;+}+/* pointer input and LordBench's scripted step wait for this display's flip (fpipe_blocked) */+static int out_blocked(const Server *s, const Out *o) { return fpipe_blocked(pipe_on(s, o), o->kms.flip_pending, o->prep); }+static int any_blocked(const Server *s) {+ for (int i = 0; i < OUT_MAX; i++) if (s->out[i].on && out_blocked(s, &s->out[i])) return 1;+ return 0;+}+/* the next frame's CPU work that can be done ahead: UI damage, layout or a JS frame callback waiting */+static int ahead_work(const Out *o) { return ui.dirty || js_wants_frame() || o->full || !box_empty(o->pend); }+static int can_prepare(const Server *s) {+ for (int i = 0; i < OUT_MAX; i++) {+ const Out *o = &s->out[i];+ if (o->on && fpipe_step(pipe_on(s, o), o->kms.flip_pending, o->prep, 0, ahead_work(o)) == FPIPE_PREPARE) return 1;+ }+ return 0;+}+/* a prepared frame whose display's flip is done: it is submitted now */+static int prep_ready(const Server *s) {+ for (int i = 0; i < OUT_MAX; i++)+ if (s->out[i].on && fpipe_step(1, s->out[i].kms.flip_pending, s->out[i].prep, 0, 0) == FPIPE_SUBMIT) return 1;+ return 0;+}+/* the prepared frame can be submitted as it was painted: the same scene, GPU context and output, and with+ * G0 the same layer buffers and textures' bookkeeping (paint_g0_epoch) */+static int prep_valid(const Out *o) {+ if (o->prep_px != o->scene.px || o->prep_gen != gpu_generation() || o->prep_gpu != (gpu_active() ? o->gpu : -1)) return 0;+ return !o->g0taken || (o->prep_epoch == paint_g0_epoch() && gpu_g0_ready(o->gpu));+}++static int par_on, par_light; /* the paint threads (render_frame) */++/* Recording the screen, or a remote viewer with protected windows: the screen must be in memory (CPU path).+ * 1 if the GPU was stopped now (every display is painted whole next). */+static int cpu_switch(Server *s) {+ if (!gpu_active() || !need_cpu()) return 0;+ fprintf(stderr, "remote: composing by CPU (%s)\n", media_needs_cpu() ? "screen recording" : "protected windows on screen");+ gpu_tex_free(s->cursor_tex);+ s->cursor_tex = NULL;+ gpu_shutdown();+ s->gpu_paused = 1;+ cpu_takeover(s);+ full_damage(s);+ return 1;+}+static void cursor_mark(Server *s) {+ if (!s->cursor_dirty) return;+ for (int i = 0; i < OUT_MAX; i++) s->out[i].cdirty = 1;+ s->cap.cdirty = 1;+ s->cursor_dirty = 0;+}++/* The CPU part of a display's frame: its damage painted into the scene (paint_out), and what G0 did. */+static Rect out_paint(Server *s, Out *o, int i, const Out *feed) {+ /* G0 with no UI damage: the GPU draws the window again from its textures, if they are still this+ * display's; otherwise its rectangle is painted by the CPU now */+ if (o->g0on && box_empty(o->pend) && !o->full && !gpu_g0_holds(o->gpu)) g0_stale(o);+ Rect r = paint_out(s, o, i, o == feed);+ o->pg0why = o->pg0sub = -1;+ o->pg0 = r.x1 > r.x0 ? paint_g0_last(&o->pg0why, &o->pg0sub) : -1;+ return r;+}++/* The rest of the frame: what paint_out painted (r) and the layers go to the display (GPU: upload, draw, swap+ * and page flip; CPU: copied into its KMS buffer, then the flip). ph: the main display's phases, else NULL. */+static void out_present(Server *s, Out *o, int i, Rect r, const Out *feed, LbPhase *ph) {+ int g0why = o->pg0why, g0sub = o->pg0sub, g0last = o->pg0;+ if (ph) { ph->g0 = g0last; ph->g0_why = g0why; ph->g0_sub = g0sub; }+ if (gpu_active() && o->gpu >= 0) {+ update_cursor(s);+ gpu_server = s;+ gpu_out = o;+ view_of(o, i);+ /* the remote desktop watches: the composed frame is read back once for every viewer */+ static uint32_t *rb;+ static size_t rb_n;+ int readback = o == feed && mirror_watchers() > 0;+ if (readback) {+ size_t need = (size_t)o->scene.w * o->scene.h;+ if (need > rb_n) { free(rb); rb = malloc(need * 4); rb_n = rb ? need : 0; }+ if (rb) gpu_read_next(rb, o->scene.w, o->scene.h);+ else readback = 0;+ }+ /* compatible fullscreen (one display): straight to the card; otherwise composed (always+ * composed while the remote desktop reads it back) */+ /* G0: the strips around the window, and the window from its textures (gpu_frame_ex); a frame+ * with nothing painted while the last one had it draws it again (its rectangle is stale) */+ Rect up[4] = { r };+ int nup = 1, g0drawn = 0;+ GpuG0 again;+ const GpuG0 *g0 = NULL;+ if (o->g0taken) {+ g0 = &o->g0;+ nup = o->g0.nstrip;+ memcpy(up, o->g0.strip, sizeof up);+ } else if (o->g0on && r.x1 <= r.x0) {+ again = o->g0;+ again.lpx = again.bpx = NULL;+ again.nl = again.nb = again.lfull = again.bfull = 0;+ again.ref = NULL;+ g0 = &again;+ }+ int scanned = !readback && out_count(s) == 1 && wl_try_scanout(s->hw_cursor || s->cursor_mode == 2);+ int res = scanned ? 0 : gpu_frame_ex(o->gpu, &o->scene, up, nup, g0, &g0drawn, gpu_layer);+ int failed = res == -1, unshown = res == -2;+ if (o->g0taken && g0drawn) paint_g0_done(); /* the textures have what paint.c had */+ if (ph) {+ /* a frame scanned out straight has no times of its own (gpu_frame_times keeps the last+ * composed frame's) and no fence (one left from before would give an old GPU time) */+ GpuFrameTimes ft = { 0 };+ if (!scanned) gpu_frame_times(o->gpu, &ft);+ else if (lbf_recording()) { uint64_t old; gpu_fence_time(o->gpu, &old); }+ ph->up_ms = ft.up_ms;+ ph->up_px = ft.up_px;+ ph->g0_ms = ft.g0_ms;+ ph->g0_px = ft.g0_px;+ ph->g0_full = ft.g0_full;+ ph->swap_ms = ft.swap_ms;+ ph->submit_ms = ft.submit_ms;+ ph->unshown = unshown;+ }+ /* G0's textures catch up after the frame (render, g0_feed_now): only when this frame+ * would have been G0's but for what they lack (paint.c, paint_g0_feed); sent while G0 is+ * refused for another reason, most of it would be sent again before it is used */+ if (!failed && !g0drawn && g0last == 0 && g0why == G0R_FEED && nfeed < OUT_MAX) feed_idx[nfeed++] = i;+ if (unshown) {+ /* the quad could not be drawn: nothing was shown (the display keeps the frame before,+ * which was right); W joins the damage and the next frame paints it on the CPU */+ o->g0w = g0->w;+ g0_stale(o);+ o->g0cpu = 1;+ } else if (g0 && !g0drawn && !failed) {+ /* shown without the quad (direct scanout, or G0 turned off): painted next frame */+ o->g0w = g0->w;+ g0_stale(o);+ } else {+ o->g0on = g0drawn;+ if (g0drawn) o->g0w = g0->w;+ }+ o->g0taken = 0;+ if (readback && !res && gpu_read_done()) {+ static int said;+ if (!said++) fprintf(stderr, "remote: composing on the GPU; frames read back for the remote desktop\n");+ Cur k;+ cursor_now(s, &k, out_scale(o));+ int cx, cy;+ pointer_phys(s, o, &cx, &cy);+ mirror_composed_on_gpu(1);+ mirror_frame(rb, o->scene.w, o->scene.w, o->scene.h, NULL);+ mirror_cursor(cx, cy, k.mode, k.px, k.w, k.h, k.hx, k.hy, k.gen);+ }+ view_reset();+ gpu_out = NULL;+ if (!unshown) { /* else they are presented with the next frame */+ o->cdirty = 0;+ o->wl_new = 0;+ }+ if (failed) {+ fprintf(stderr, "gpu: a frame failed; going on by CPU\n");+ gpu_shutdown();+ cpu_takeover(s);+ full_damage(s);+ }+ } else present_cpu(s, o, i, r, o == feed);+ if (ph) {+ ph->prev_flip = o->kms.flip_ns;+ ph->submit = o->kms.flip_pending ? o->kms.flip_submit_ns : 0;+ ph->period = o->kms.mode.vrefresh ? 1000000000u / o->kms.mode.vrefresh : 16666667u;+ }+}++static double out_period_ms(const Out *o) { return o->kms.mode.vrefresh ? 1000.0 / o->kms.mode.vrefresh : 1000.0 / 60; }+/* a main display's frame for the pipeline's auto mode (fpipe_note); with -d its changes are logged */+static void pipe_note(const Server *s, double ms, const Out *o) {+ int was = fpipe_engaged(&fpipe);+ fpipe_note(&fpipe, ms, out_period_ms(o));+ if (s->debug && was != fpipe_engaged(&fpipe))+ fprintf(stderr, "frames: pipeline %s (a frame took %.1f ms of %.1f)\n", was ? "off" : "on", ms, out_period_ms(o));+}++/* The frame pipeline: the next frame painted into the scene while this display's flip is pending. Its CPU+ * time starts at ui0 (after any submission in this render: the JS frame callbacks, layout and paint). */+static void out_prepare(Server *s, Out *o, int i, const Out *feed, int is_main, double ui0, uint64_t ui0_ns) {+ double paint0 = ui.paint_ms;+ int whole = o->full;+ Rect r = out_paint(s, o, i, feed);+ /* nothing painted: the frame is built after the flip as without the pipeline, and nothing more is tried+ * before it (input waits for it as before, and a JS frame callback without damage does not spin) */+ if (r.x1 <= r.x0 || r.y1 <= r.y0) { o->prep = FPIPE_EMPTY; return; }+ o->prep = FPIPE_READY;+ o->prep_r = r;+ o->prep_whole = whole;+ o->prep_px = o->scene.px;+ o->prep_gen = gpu_generation();+ o->prep_gpu = gpu_active() ? o->gpu : -1;+ o->prep_epoch = paint_g0_epoch();+ o->prep_ms = now_ms() - ui0;+ if (!is_main) return;+ lbf_painted(); /* the synthetic pointer's events so far are in it */+ o->pph = (LbPhase){ .wake = loop_wake, .from_flip = loop_wake_flip, .pre_end = render_r0, .render0 = ui0_ns,+ .built = mono_ns(), .paint_ms = ui.layout_ms + (ui.paint_ms - paint0), .par = par_on, .piped = 1 };+}++/* The frame pipeline: the prepared frame goes to the display (only upload, GPU and flip were left). */+static void out_submit(Server *s, Out *o, int i, const Out *feed, int is_main) {+ uint64_t s0 = lbf_recording() ? mono_ns() : 0;+ double t0 = now_ms();+ o->prep = FPIPE_NONE;+ LbPhase *ph = is_main ? &o->pph : NULL;+ out_present(s, o, i, o->prep_r, feed, ph);+ if (!is_main) return;+ double ms = o->prep_ms + (now_ms() - t0);+ lbf_submitted(); /* its events end at its presentation */+ lbf_frame(ms); /* its work: the prepare and the submission */+ pipe_note(s, ms, o);+ ph->swake = loop_wake;+ ph->sfrom_flip = loop_wake_flip;+ ph->sub0 = s0;+ ph->sub1 = s0 ? mono_ns() : 0;+ call_piped = 1;+}++/* Returns 0 when it only painted the next frame ahead (every display was waiting for its flip). */+static int render_frame(Server *s) {double t0 = now_ms();+ int prep_only = !s->headless && all_busy(s);ui.layout_ms = ui.paint_ms = 0;ui.damage_px = 0;if (keylag_enabled) keylag_render();@@ -1543,26 +1817,37 @@ static void render_frame(Server *s) {* second late and Chromium logged "buggy presentation feedback" (docs/TROUBLESHOOTING.md). */if (wl_wants_frame() && gpu_active())for (int i = 0; i < OUT_MAX; i++) if (s->out[i].on && s->out[i].gpu >= 0) s->out[i].wl_new = 1;+ Out *feed = feed_out(s), *mo = main_out(s);+ int main_shown = 0, main_built = 0; /* the main display got this frame; built here (not prepared before) */+ int done[OUT_MAX] = { 0 }; /* submitted or built in this render */+ /* The frame pipeline: the prepared frames are submitted first, before the JS frame callbacks and the+ * layout of the next one; G0's textures catch up before that one is painted. One that can no longer be+ * submitted as painted is undone and built below like any frame. */+ int ready = !s->headless && prep_ready(s);+ if (ready) {+ cursor_mark(s);+ cpu_switch(s);+ for (int i = 0; i < OUT_MAX; i++) {+ Out *o = &s->out[i];+ if (!o->on || o->kms.flip_pending || o->prep != FPIPE_READY) continue;+ if (!prep_valid(o)) { prep_undo(o); continue; }+ out_submit(s, o, i, feed, o == mo);+ done[i] = 1;+ if (o == mo) main_shown = 1;+ }+ if (call_piped) {+ g0_feed_now(s, &mo->pph);+ lbf_phase_built(&mo->pph);+ }+ }+ double ui0 = now_ms();+ uint64_t ui0_ns = lbf_recording() ? mono_ns() : 0;paint_ui(s);s->last_render = now_ms();- if (s->cursor_dirty) {- for (int i = 0; i < OUT_MAX; i++) s->out[i].cdirty = 1;- s->cap.cdirty = 1;- s->cursor_dirty = 0;- }- if (gpu_active() && need_cpu()) { /* recording the screen, or a remote viewer with protected windows */- /* the screen must be in memory (CPU path) */- fprintf(stderr, "remote: composing by CPU (%s)\n", media_needs_cpu() ? "screen recording" : "protected windows on screen");- gpu_tex_free(s->cursor_tex);- s->cursor_tex = NULL;- gpu_shutdown();- s->gpu_paused = 1;- cpu_takeover(s);- full_damage(s);- paint_ui(s);- }- Out *feed = feed_out(s), *mo = main_out(s);- int main_shown = 0; /* the main display got this frame */+ cursor_mark(s);+ /* (not while every display waits for its flip: stopping the GPU sets the dumb buffer with a modeset, and a+ * pending flip's buffer must stay; a frame prepared meanwhile is undone at the flip by cpu_takeover) */+ if (!ready && !prep_only && cpu_switch(s)) paint_ui(s);if (s->headless) {Out *o = &s->out[0];Rect drawn = paint_out(s, o, 0, 1);@@ -1573,117 +1858,30 @@ static void render_frame(Server *s) {for (int i = 0; i < OUT_MAX; i++) {Out *o = &s->out[i];/* a fullscreen app's new frame (render runs when something wants a frame) goes to every display */- if (!o->on || o->kms.flip_pending || !(out_needs(s, o) || wl_fullscreen_active())) continue;- /* G0 with no UI damage: the GPU draws the window again from its textures, if they are still- * this display's; otherwise its rectangle is painted by the CPU now */- if (o->g0on && box_empty(o->pend) && !o->full && !gpu_g0_holds(o->gpu)) g0_stale(o);- Rect r = paint_out(s, o, i, o == feed);- int g0why = -1, g0sub = -1, g0last = r.x1 > r.x0 ? paint_g0_last(&g0why, &g0sub) : -1;- if (o == mo) { fph.g0 = g0last; fph.g0_why = g0why; fph.g0_sub = g0sub; }- if (gpu_active() && o->gpu >= 0) { /* (the loop above skips what has nothing new) */- update_cursor(s);- gpu_server = s;- gpu_out = o;- view_of(o, i);- /* the remote desktop watches: the composed frame is read back once for every viewer */- static uint32_t *rb;- static size_t rb_n;- int readback = o == feed && mirror_watchers() > 0;- if (readback) {- size_t need = (size_t)o->scene.w * o->scene.h;- if (need > rb_n) { free(rb); rb = malloc(need * 4); rb_n = rb ? need : 0; }- if (rb) gpu_read_next(rb, o->scene.w, o->scene.h);- else readback = 0;- }- /* compatible fullscreen (one display): straight to the card; otherwise composed (always- * composed while the remote desktop reads it back) */- /* G0: the strips around the window, and the window from its textures (gpu_frame_ex); a frame- * with nothing painted while the last one had it draws it again (its rectangle is stale) */- Rect up[4] = { r };- int nup = 1, g0drawn = 0;- GpuG0 again;- const GpuG0 *g0 = NULL;- if (o->g0taken) {- g0 = &o->g0;- nup = o->g0.nstrip;- memcpy(up, o->g0.strip, sizeof up);- } else if (o->g0on && r.x1 <= r.x0) {- again = o->g0;- again.lpx = again.bpx = NULL;- again.nl = again.nb = again.lfull = again.bfull = 0;- again.ref = NULL;- g0 = &again;- }- int scanned = !readback && out_count(s) == 1 && wl_try_scanout(s->hw_cursor || s->cursor_mode == 2);- int res = scanned ? 0 : gpu_frame_ex(o->gpu, &o->scene, up, nup, g0, &g0drawn, gpu_layer);- int failed = res == -1, unshown = res == -2;- if (o->g0taken && g0drawn) paint_g0_done(); /* the textures have what paint.c had */- if (o == mo) {- /* a frame scanned out straight has no times of its own (gpu_frame_times keeps the last- * composed frame's) and no fence (one left from before would give an old GPU time) */- GpuFrameTimes ft = { 0 };- if (!scanned) gpu_frame_times(o->gpu, &ft);- else if (lbf_recording()) { uint64_t old; gpu_fence_time(o->gpu, &old); }- fph.up_ms = ft.up_ms;- fph.up_px = ft.up_px;- fph.g0_ms = ft.g0_ms;- fph.g0_px = ft.g0_px;- fph.g0_full = ft.g0_full;- fph.swap_ms = ft.swap_ms;- fph.submit_ms = ft.submit_ms;- fph.unshown = unshown;- }- /* G0's textures catch up after the frame (render, g0_feed_now): only when this frame- * would have been G0's but for what they lack (paint.c, paint_g0_feed); sent while G0 is- * refused for another reason, most of it would be sent again before it is used */- if (!failed && !g0drawn && g0last == 0 && g0why == G0R_FEED && nfeed < OUT_MAX) feed_idx[nfeed++] = i;- if (unshown) {- /* the quad could not be drawn: nothing was shown (the display keeps the frame before,- * which was right); W joins the damage and the next frame paints it on the CPU */- o->g0w = g0->w;- g0_stale(o);- o->g0cpu = 1;- } else if (g0 && !g0drawn && !failed) {- /* shown without the quad (direct scanout, or G0 turned off): painted next frame */- o->g0w = g0->w;- g0_stale(o);- } else {- o->g0on = g0drawn;- if (g0drawn) o->g0w = g0->w;- }- o->g0taken = 0;- if (readback && !res && gpu_read_done()) {- static int said;- if (!said++) fprintf(stderr, "remote: composing on the GPU; frames read back for the remote desktop\n");- Cur k;- cursor_now(s, &k, out_scale(o));- int cx, cy;- pointer_phys(s, o, &cx, &cy);- mirror_composed_on_gpu(1);- mirror_frame(rb, o->scene.w, o->scene.w, o->scene.h, NULL);- mirror_cursor(cx, cy, k.mode, k.px, k.w, k.h, k.hx, k.hy, k.gen);- }- view_reset();- gpu_out = NULL;- if (!unshown) { /* else they are presented with the next frame */- o->cdirty = 0;- o->wl_new = 0;- }- if (failed) {- fprintf(stderr, "gpu: a frame failed; going on by CPU\n");- gpu_shutdown();- cpu_takeover(s);- full_damage(s);- }- } else present_cpu(s, o, i, r, o == feed);- if (o == mo) main_shown = 1;+ if (!o->on || o->kms.flip_pending || done[i] || !(out_needs(s, o) || wl_fullscreen_active())) continue;+ prep_undo(o); /* (an attempt that painted nothing during the flip) */+ Rect r = out_paint(s, o, i, feed);+ out_present(s, o, i, r, feed, o == mo ? &fph : NULL);+ done[i] = 1;if (o == mo) {+ main_shown = main_built = 1;fph_main = 1;- fph.prev_flip = o->kms.flip_ns;- fph.submit = o->kms.flip_pending ? o->kms.flip_submit_ns : 0;- fph.period = o->kms.mode.vrefresh ? 1000000000u / o->kms.mode.vrefresh : 16666667u;+ pipe_note(s, now_ms() - t0, o);}}+ /* the frame pipeline: the next frame of a display waiting for its flip is painted now. Right after+ * a submission or a frame built here only when there is damage already; called for it alone, it is+ * tried once per flip even without (a JS frame callback may bring it) */+ for (int i = 0; i < OUT_MAX; i++) {+ Out *o = &s->out[i];+ if (!o->on) continue;+ int work = prep_only || o->full || !box_empty(o->pend);+ /* (not with G0's catch-up still queued by a frame built here: it runs after render_frame, and must+ * never run between a frame's paint and its upload) */+ if (nfeed) continue;+ if (fpipe_step(pipe_on(s, o), o->kms.flip_pending, o->prep, 0, work) == FPIPE_PREPARE)+ out_prepare(s, o, i, feed, o == mo, ui0, ui0_ns);+ }/* every display in one frame for the remote desktop and recording (capture view) */Out *c = &s->cap;if (feed == c && (mirror_watchers() || media_needs_cpu())) {@@ -1699,14 +1897,13 @@ static void render_frame(Server *s) {}} else if (c->on) { out_free(c); c->on = 0; c->pend = (Box){ 0, 0, 0, 0 }; }}- lbf_built(s->headless || main_shown); /* LordBench: the synthetic pointer's events are in this frame */- s->frames++;+ lbf_built(s->headless || main_built); /* LordBench: the synthetic pointer's events are in this frame */+ if (!prep_only) s->frames++;/* The pool only while frames are heavy: it costs CPU to wake (a light frame painted on one thread* spends less), so it turns on when painting takes more than 70 % of a frame at the target rate (the* monitor's refresh, or the fastest remote viewer's without a monitor) and off after 60 light frames* in a row (with the pool a heavy frame paints fast, so not at once: it would flap).* A 1080p video under blurred windows: 45 → 57 frames a second on VM 116. */- static int par_on, light;Out *pm = main_out(s);int hz = s->headless ? (mirror_fps() > 60 ? mirror_fps() : 60) : pm && pm->kms.mode.vrefresh ? (int)pm->kms.mode.vrefresh : 60;double frame_ms = 1000.0 / hz;@@ -1717,12 +1914,13 @@ static void render_frame(Server *s) {* paint alone (about 7.5 ms) stayed under 70 % while the frame (about 13 ms) missed every vblank. On* when the frame takes more than 60 % of the period; light only with both paint and frame short. */double built = now_ms() - t0;- if (!par_on && (ui.paint_ms > frame_ms * 0.7 || built > frame_ms * 0.6) && par_threads() > 1) { par_on = 1; light = 0; canvas_parallel(par_rows); }+ if (!par_on && (ui.paint_ms > frame_ms * 0.7 || built > frame_ms * 0.6) && par_threads() > 1) { par_on = 1; par_light = 0; canvas_parallel(par_rows); }else if (par_on) {- light = ui.paint_ms < frame_ms * 0.25 && built < frame_ms * 0.45 ? light + 1 : 0;- if (light >= 60) { par_on = 0; canvas_parallel(NULL); }+ par_light = ui.paint_ms < frame_ms * 0.25 && built < frame_ms * 0.45 ? par_light + 1 : 0;+ if (par_light >= 60) { par_on = 0; canvas_parallel(NULL); }}- wl_frame_done((unsigned)now_ms());+ /* the Linux apps' frame callbacks go with a frame submitted, not with one only painted ahead */+ if (!prep_only) wl_frame_done((unsigned)now_ms());if (s->headless) { /* no flip without a monitor: the frame is out now */struct timespec t;clock_gettime(CLOCK_MONOTONIC, &t);@@ -1746,7 +1944,8 @@ static void render_frame(Server *s) {lbf_presented(ns, mo->kms.mode.vrefresh ? 1000000000u / mo->kms.mode.vrefresh : 16666667u);if (keylag_enabled) keylag_presented(ns);}- frame_stats(s, now_ms() - t0, ui.layout_ms + ui.paint_ms);+ if (!prep_only) frame_stats(s, now_ms() - t0, ui.layout_ms + ui.paint_ms);+ return !prep_only;}/* Cambio de tema pedido desde JS (lord.themes.use): se descarta el contexto JS y el árbol. */@@ -2046,6 +2245,9 @@ int main(int argc, char **argv) {sigaddset(&mask, SIGTERM);sigprocmask(SIG_BLOCK, &mask, NULL);keylag_init(s.greeter); /* key latency trace, only with LORD_INPUT_DEBUG=1 (keylag.h) */+ fpipe_init(&fpipe, getenv("LORD_PIPELINE"));+ fprintf(stderr, "lord-server: frame pipeline %s (LORD_PIPELINE: 0 off, 2 always)\n",+ fpipe.mode == FPIPE_OFF ? "off" : fpipe.mode == FPIPE_ALWAYS ? "always" : "while frames are heavy");int sfd = signalfd(-1, &mask, SFD_CLOEXEC);double start = now_ms();@@ -2057,17 +2259,17 @@ int main(int argc, char **argv) {}/* El movimiento del puntero juntado: una vez por cuadro, cuando terminó el page flip. Así un* mouse de 1000 Hz no llama a JS (ni a las apps Linux) mil veces por segundo. */- if (s.motion_pending && !main_out(&s)->kms.flip_pending) {+ if (s.motion_pending && !out_blocked(&s, main_out(&s))) {dispatch_pointer(&s);if (gpu_active()) update_cursor(&s);}/* LordBench's scene moves its window one step per frame, as a drag does (ui/grab.c) */- if (grab_scripted() && !any_flip_pending(&s)) grab_script_tick(now_ms());+ if (grab_scripted() && !any_blocked(&s)) grab_script_tick(now_ms());static int scripted; /* its end: the layer's counters, as a drag's line has them */if (scripted && !grab_scripted()) {long lr = 0, lb = 0, lp = paint_layer_parts();paint_layer_stats(&lr, &lb);- char bs[192], gs[512], ps[1024];+ char bs[192], gs[512], ps[1536];below_stats_text(bs, sizeof bs);g0_stats_text(gs, sizeof gs);LbPhaseStats ph;@@ -2122,6 +2324,8 @@ int main(int argc, char **argv) {if (!s.headless && display_is_off()) unseen = 1; /* screen off after idle (DPMS): nothing to draw *//* nothing is drawn: the Linux apps waiting for a frame get it at most once a second (wl.c) */if (unseen) {+ /* a frame prepared before the screen went off would be shown stale when it comes back */+ prep_undo_all(&s);static double unseen_tick;double since = now_ms() - unseen_tick;if (since >= 1000) { wl_frame_unseen((unsigned)now_ms()); unseen_tick = now_ms(); since = 0; }@@ -2130,7 +2334,11 @@ int main(int argc, char **argv) {if (delay < 0 || left < delay) delay = left;}}- int wants = !unseen && (ui.dirty || s.cursor_dirty || js_wants_frame() || wl_wants_frame() || media_wants_frame() || outs_have_work(&s));+ /* the frame pipeline (fpipe.h): a prepared frame whose flip is done is submitted, and the next one may+ * be painted while every display waits for its flip */+ int ahead = !unseen && can_prepare(&s);+ int wants = !unseen && (ui.dirty || s.cursor_dirty || js_wants_frame() || wl_wants_frame() || media_wants_frame() ||+ outs_have_work(&s) || prep_ready(&s) || ahead);/* no monitor, no vblank: 60 frames a second, or as fast as the fastest remote viewer's screen* (120, 144 Hz…) */int hz = mirror_fps() > 60 ? mirror_fps() : 60;@@ -2145,7 +2353,7 @@ int main(int argc, char **argv) {if (!pace_next || tnow - pace_next > period) pace_next = tnow; /* idle, or fell behind */pace = pace_next - tnow;}- if (wants && !all_busy(&s) && pace <= 0) {+ if (wants && (!all_busy(&s) || ahead) && pace <= 0) {render(&s);wants = 0;if (s.headless || soft) pace_next += period;@@ -2153,7 +2361,7 @@ int main(int argc, char **argv) {if (wants && pace > 0 && (delay < 0 || pace < delay)) delay = pace;int wait = delay < 0 ? -1 : (int)ceil(delay);- if ((wants && !all_busy(&s) && pace <= 0) || ui.theme_request[0] || ui.quit_requested) wait = 0;+ if ((wants && (!all_busy(&s) || ahead) && pace <= 0) || ui.theme_request[0] || ui.quit_requested) wait = 0;if (timeout_ms > 0) {double left = start + timeout_ms - now_ms();if (left <= 0) break;diff --git a/server/src/ui/paint.c b/server/src/ui/paint.cindex 545a0bc..4e21d0d 100644--- a/server/src/ui/paint.c+++ b/server/src/ui/paint.c@@ -549,7 +549,11 @@ static void g0_dirty(G0Dirty *d, Rect r) {d->r[bi] = g0_join(d->r[bi], d->r[bj]);d->r[bj] = r;}-static void g0_dirty_all(void) { g0_ld = (G0Dirty){ .full = 1 }; g0_bd = (G0Dirty){ .full = 1 }; }+/* bumped whenever a GpuG0 filled earlier may no longer be submitted as it is (its buffers or the textures'+ * bookkeeping changed): see paint_g0_epoch */+static unsigned g0_epoch;+unsigned paint_g0_epoch(void) { return g0_epoch; }+static void g0_dirty_all(void) { g0_ld = (G0Dirty){ .full = 1 }; g0_bd = (G0Dirty){ .full = 1 }; g0_epoch++; }void paint_g0_done(void) { g0_ld = (G0Dirty){ 0 }; g0_bd = (G0Dirty){ 0 }; }void paint_g0_stats(G0Stats *out) {*out = g0stat;@@ -593,6 +597,7 @@ int paint_layer_node(void) { return layer.want ? layer.node : 0; }void paint_layer_view(int index) {if (layer.view == index) return;+ g0_epoch++;layer.view = index;layer.valid = 0;layer.lg_valid = 0;@@ -782,6 +787,7 @@ static int layer_render(Node *n, float opacity) {uint8_t *hole = realloc(layer.hole, (size_t)w * h);if (px) layer.px = px;if (hole) layer.hole = hole;+ g0_epoch++;if (!px || !hole) { layer.want = 0; return 0; }layer.cap = w * h;}@@ -1042,6 +1048,7 @@ static int view_eq(PaintView a, PaintView b) { return a.x == b.x && a.y == b.y &static int below_alloc(const Canvas *c) {if (layer.below && layer.bw == c->w && layer.bh == c->h) return 1;free(layer.below);+ g0_epoch++;layer.below = malloc((size_t)c->w * c->h * 4);layer.bw = layer.below ? c->w : 0;layer.bh = layer.below ? c->h : 0;diff --git a/server/src/ui/ui.h b/server/src/ui/ui.hindex e6547b3..9372600 100644--- a/server/src/ui/ui.h+++ b/server/src/ui/ui.h@@ -320,6 +320,10 @@ void paint_below_stats(BelowStats *s); /* builds (whole, part) and switchedstruct GpuG0;int paint_root_g0(Canvas *c, Node *root, struct GpuG0 *g);void paint_g0_done(void);+/* changes whenever a GpuG0 that paint_root_g0 filled may no longer be submitted as it is: the drag layer began,+ * ended or moved to another display, its buffers were reallocated or the textures need everything again. The+ * frame pipeline (main.c) checks it before submitting a prepared G0 frame. */+unsigned paint_g0_epoch(void);/* why a frame that used the copy was not composited on the GPU: the layer (invalid, re-rendered, app* windows or invalid pixels in it, other opacity, a transform), its rectangle not inside the damage,* an ancestor painting over it, an app window under it, a node painted after the window in thediff --git a/server/tests/engine_test.c b/server/tests/engine_test.cindex 6d7f342..f4019fd 100644--- a/server/tests/engine_test.c+++ b/server/tests/engine_test.c@@ -1598,6 +1598,12 @@ typedef struct {GpuG0 g;} DOut;static int test_g0;+/* the frame pipeline (main.c, fpipe.h): each frame painted is presented only after the next frame's changes+ * (the pointer, JS) are made, right before that frame is painted, as a frame prepared during a pending flip+ * is submitted at the flip; its pixels must be the same */+static int test_pipe;+typedef struct { int on, taken, last, why; Rect pr; } DPend;+static DPend dpend[4];static struct { uint32_t *b, *l; int bw, bh, lw, lh, owner[2]; } g0tex; /* the stub GPU's G0 textures (owner: 0 the copy, 1 the layer) */static long g0_fed; /* pixels sent between frames (paint_g0_feed) */static long g0_whole; /* G0 frames that had to send a texture whole */@@ -1679,12 +1685,11 @@ static int g0_tex_feed(void *ctx, int which, const uint32_t *px, int w, int h, R* strips) go to the UI texture; with G0 the two textures get their dirty rectangles and the window is* written over the texture's stale rectangle in the composed frame. A frame refused only for what the* textures lack (G0R_FEED) then sends a slice of it (paint_g0_feed), as main.c does after queueing the flip. */-static void g0_present(DOut *o, int index, Rect pr, int taken) {+static void g0_present(DOut *o, int index, Rect pr, int taken, int last, int why) {int W = o->c.w, H = o->c.h;if (!taken) {copy_rect(o->tex, W, o->c.px, o->c.stride, pr);- int why, sub;- if (paint_g0_last(&why, &sub) == 0 && why == G0R_FEED && paint_g0_backlog(index) > 0)+ if (last == 0 && why == G0R_FEED && paint_g0_backlog(index) > 0)paint_g0_feed(index, G0_FEED_PX, g0_tex_feed, &index);} else {const GpuG0 *g = &o->g;@@ -1742,7 +1747,19 @@ static void drag_paint_out(Node *root, DOut *o, int index, Box d) {canvas_set_clip(&o->c, (Rect){ 0, 0, o->c.w, o->c.h });ui.damage = (Box){ 0, 0, 0, 0 };paint_view = (PaintView){ 0, 0, 1, 0, 0, 0 };- if (test_g0) g0_present(o, index, pr, taken);+ if (!test_g0) return;+ int why, sub, last = paint_g0_last(&why, &sub);+ if (test_pipe && index < 4) dpend[index] = (DPend){ 1, taken, last, why, pr };+ else g0_present(o, index, pr, taken, last, why);+}++/* the frame pipeline: the frames painted are presented now (main.c's out_submit) */+static void drag_present(DOut *const *os, int n) {+ for (int i = 0; i < n && i < 4; i++)+ if (dpend[i].on) {+ dpend[i].on = 0;+ g0_present(os[i], i, dpend[i].pr, dpend[i].taken, dpend[i].last, dpend[i].why);+ }}/* main.c's frame: paint_ui (layout, damage, the layer's display: the one with most of the window), then@@ -2046,8 +2063,10 @@ static void drag_run_n(int dc, DOut *const *os, int nos, uint64_t *hash, long *fdamage_add((Box){ 0, 0, DW, DH });ui.dirty = 1;drag_paint_n(s.root, os, nos);+ drag_present(os, nos);paint_layer_begin(s.win->id, 0); /* grab.c's setup */float gx = x0, gy = y0, rot = 0;+ unsigned pipe_epoch = paint_g0_epoch();*full = 0;*worst = 0;for (int f = 0; f < n; f++) {@@ -2091,13 +2110,30 @@ static void drag_run_n(int dc, DOut *const *os, int nos, uint64_t *hash, long *fdamage_add((Box){ floorf(term_at.x0), floorf(term_at.y0 + 8 - 2), ceilf(term_at.x1), ceilf(term_at.y0 + 24 + 2) });ui.dirty = 1;}+ if (test_pipe) {+ /* the frame before is submitted only now, after this frame's changes: they must not touch what it+ * hands the GPU (paint_g0_epoch, else main.c paints it again) */+ CHECK(paint_g0_epoch() == pipe_epoch, "pipelined: frame %d's changes moved G0's epoch", f);+ drag_present(os, nos);+ if (f) hash[f - 1] = outs_hash(os, nos);+ drag_paint_n(s.root, os, nos);+ pipe_epoch = paint_g0_epoch();+ continue; /* (a full repaint now would be of the next frame's tree) */+ }drag_paint_n(s.root, os, nos);hash[f] = outs_hash(os, nos);if (nos == 1 && f % FULL_EVERY == FULL_EVERY - 1) { long d = drag_full_diff(s.root, o, worst); if (d > *full) *full = d; }}+ if (test_pipe) {+ drag_present(os, nos);+ hash[n - 1] = outs_hash(os, nos);+ }+ unsigned before_end = paint_g0_epoch();paint_layer_end(); /* grab_finish */+ CHECK(paint_g0_epoch() != before_end, "the drag's end moves G0's epoch (a prepared G0 frame is painted again)");ui.dirty = 1;drag_paint_n(s.root, os, nos);+ drag_present(os, nos);hash[n] = outs_hash(os, nos);if (nos == 1) {long d = drag_full_diff(s.root, o, worst);@@ -2235,7 +2271,7 @@ done:}static void test_drag_exact(int write) {- static const char *const variant[3] = { "cpu", "gpu", "g0" };+ static const char *const variant[4] = { "cpu", "gpu", "g0", "g0 pipelined" };DOut o, o2; /* the display at 100 %, and at 200 % (DC_K2) */int ok = dout_init(&o, out_view(0, 0, DW, DH, 100));ok = dout_init(&o2, out_view(0, 0, 2 * DW, 2 * DH, 200)) && ok;@@ -2246,11 +2282,13 @@ static void test_drag_exact(int write) {return;}/* g0: the GPU path with G0 (the window composited by canvas_g0_model), against the GPU's hashes */- for (int g = 0; g < (write ? 2 : 3); g++) {+ /* g0 pipelined: the same, each frame presented after the next frame's changes (the frame pipeline) */+ for (int g = 0; g < (write ? 2 : 4); g++) {test_gpu = g > 0;- test_g0 = g == 2;+ test_g0 = g >= 2;+ test_pipe = g == 3;char path[64];- snprintf(path, sizeof path, "tests/drag_golden_%s.txt", variant[g == 2 ? 1 : g]);+ snprintf(path, sizeof path, "tests/drag_golden_%s.txt", variant[g >= 2 ? 1 : g]);long fullw[DC_N];int worst[DC_N];long parts = 0, renders = 0, blits = 0;@@ -2282,7 +2320,7 @@ static void test_drag_exact(int write) {if (!test_g0)CHECK(!paint_below_bad && paint_below_checked == bs.switched, "drag-exact %s: frames switched to the copy of what is below ""differ from painting everything: %ld pixels in %ld frames", variant[g], paint_below_bad, paint_below_checked);- else g0_report("g0", 1);+ else g0_report(variant[g], 1);printf("drag-exact %s: most pixels differing from a full repaint (largest difference):", variant[g]);for (int dc = 0; dc < DC_N; dc++) printf(" %s %ld (%d)", dc_name[dc], fullw[dc], worst[dc]);printf("\n");@@ -2309,7 +2347,7 @@ static void test_drag_exact(int write) {}free(file);}- test_gpu = test_g0 = 0;+ test_gpu = test_g0 = test_pipe = 0;dout_free(&o);dout_free(&o2);free(hash);diff --git a/server/tests/fpipe_test.c b/server/tests/fpipe_test.cnew file mode 100644index 0000000..a304827--- /dev/null+++ b/server/tests/fpipe_test.c@@ -0,0 +1,167 @@+/* fpipe_test.c — the frame pipeline's decisions (src/fpipe.c, docs/GPU.md "The frame pipeline"):+ * - LORD_PIPELINE and the auto mode's hysteresis;+ * - what a display does next (build, submit, prepare, wait) and when input waits for the flip;+ * - a display driven the way main.c drives it, on a vblank grid: a frame whose CPU part plus the GPU's+ * part passes the period is shown every other vblank without the pipeline and at every vblank with+ * it; a light frame is shown at every vblank either way; an idle display builds at once; nothing is+ * prepared twice for one flip (no busy loop). */+#include "fpipe.h"++#include <stdio.h>+#include <string.h>++static int fails, checks;+#define CHECK(c) do { checks++; if (!(c)) { fails++; fprintf(stderr, "%s:%d: CHECK(%s) failed\n", __FILE__, __LINE__, #c); } } while (0)++static void test_mode(void) {+ FPipe p;+ fpipe_init(&p, NULL);+ CHECK(p.mode == FPIPE_AUTO && !fpipe_engaged(&p));+ fpipe_init(&p, "0");+ CHECK(p.mode == FPIPE_OFF && !fpipe_engaged(&p));+ fpipe_note(&p, 30, 16.67); /* off stays off, however heavy */+ fpipe_note(&p, 30, 16.67);+ CHECK(!fpipe_engaged(&p));+ fpipe_init(&p, "2");+ CHECK(p.mode == FPIPE_ALWAYS && fpipe_engaged(&p));+ fpipe_note(&p, 1, 16.67);+ CHECK(fpipe_engaged(&p));+ fpipe_init(&p, "1");+ CHECK(p.mode == FPIPE_AUTO);+ fpipe_init(&p, "yes"); /* anything else: auto */+ CHECK(p.mode == FPIPE_AUTO);+}++static void test_auto(void) {+ FPipe p;+ fpipe_init(&p, NULL);+ fpipe_note(&p, 13, 16.67); /* one heavy frame: not yet */+ CHECK(!fpipe_engaged(&p));+ fpipe_note(&p, 5, 16.67); /* the run is broken */+ fpipe_note(&p, 13, 16.67);+ CHECK(!fpipe_engaged(&p));+ fpipe_note(&p, 13, 16.67); /* two in a row above 60 % */+ CHECK(fpipe_engaged(&p));+ for (int i = 0; i < 59; i++) fpipe_note(&p, 5, 16.67);+ CHECK(fpipe_engaged(&p)); /* 59 light ones: still on */+ fpipe_note(&p, 8, 16.67); /* 48 %: neither light nor a reason to stay; the run restarts */+ for (int i = 0; i < 59; i++) fpipe_note(&p, 5, 16.67);+ CHECK(fpipe_engaged(&p));+ fpipe_note(&p, 5, 16.67); /* the 60th light frame in a row */+ CHECK(!fpipe_engaged(&p));+ fpipe_note(&p, 9, 16.67); /* 54 %: under 60 %, stays off */+ fpipe_note(&p, 9, 16.67);+ CHECK(!fpipe_engaged(&p));+ fpipe_note(&p, 13, 0); /* no period: ignored */+ fpipe_note(&p, 13, 0);+ CHECK(!fpipe_engaged(&p));+}++static void test_step(void) {+ /* no flip pending: a prepared frame is submitted; else built if there is something to show */+ CHECK(fpipe_step(1, 0, FPIPE_READY, 0, 0) == FPIPE_SUBMIT);+ CHECK(fpipe_step(0, 0, FPIPE_READY, 1, 1) == FPIPE_SUBMIT); /* even after the pipeline let go */+ CHECK(fpipe_step(1, 0, FPIPE_NONE, 1, 0) == FPIPE_BUILD);+ CHECK(fpipe_step(1, 0, FPIPE_EMPTY, 1, 0) == FPIPE_BUILD); /* the attempt painted nothing: built now */+ CHECK(fpipe_step(1, 0, FPIPE_NONE, 0, 0) == FPIPE_WAIT);+ /* flip pending: prepared only with the pipeline, once, and only with work to do ahead */+ CHECK(fpipe_step(1, 1, FPIPE_NONE, 1, 1) == FPIPE_PREPARE);+ CHECK(fpipe_step(0, 1, FPIPE_NONE, 1, 1) == FPIPE_WAIT);+ CHECK(fpipe_step(1, 1, FPIPE_NONE, 1, 0) == FPIPE_WAIT);+ CHECK(fpipe_step(1, 1, FPIPE_READY, 1, 1) == FPIPE_WAIT);+ CHECK(fpipe_step(1, 1, FPIPE_EMPTY, 1, 1) == FPIPE_WAIT);+ /* input waits: always without the pipeline while the flip is pending; with it only once prepared */+ CHECK(!fpipe_blocked(0, 0, FPIPE_NONE) && !fpipe_blocked(1, 0, FPIPE_READY));+ CHECK(fpipe_blocked(0, 1, FPIPE_NONE));+ CHECK(!fpipe_blocked(1, 1, FPIPE_NONE));+ CHECK(fpipe_blocked(1, 1, FPIPE_READY) && fpipe_blocked(1, 1, FPIPE_EMPTY));+}++/* One display driven like main.c's loop on a vblank grid (period P, µs). Every frame has cpu µs of CPU work+ * (input, JS, layout, paint) and sub µs to submit (upload, swap, flip ioctl); the GPU then needs gpu µs, and a+ * flip lands on the first vblank after both. animate: there is always a next frame to build. Returns the flips+ * shown in n periods; *prepares counts frames painted ahead, *spins render calls that did nothing. */+typedef struct { long shown, prepares, spins, max_latency; } Sim;+static Sim simulate(int mode, long cpu, long sub, long gpu, int animate, long n) {+ const long P = 16667;+ FPipe fp;+ fpipe_init(&fp, mode == FPIPE_OFF ? "0" : mode == FPIPE_ALWAYS ? "2" : NULL);+ Sim r = { 0, 0, 0, 0 };+ long t = 0, flip_at = -1; /* when the pending flip lands (-1: none) */+ int prep = FPIPE_NONE;+ long input_t = 0, prep_input = 0, inflight_input = 0; /* when the newest input shown by a frame came */+ int damage = 1;+ while (t < n * P) {+ int pending = flip_at >= 0;+ if (pending && t >= flip_at) { /* the flip event */+ pending = 0;+ flip_at = -1;+ r.shown++;+ long lat = t - inflight_input;+ if (lat > r.max_latency) r.max_latency = lat;+ }+ if (animate) { damage = 1; input_t = t; } /* new input every wake: the frame shows it */+ int eligible = fpipe_engaged(&fp);+ int step = fpipe_step(eligible, pending, prep, damage, damage);+ long busy = 0;+ if (step == FPIPE_SUBMIT) {+ busy += sub;+ prep = FPIPE_NONE;+ inflight_input = prep_input;+ flip_at = ((t + busy + gpu) / P + 1) * P; /* the next vblank after the GPU is done */+ pending = 1;+ fpipe_note(&fp, (double)(cpu + sub) / 1000, P / 1000.0);+ } else if (step == FPIPE_BUILD) {+ busy += cpu + sub;+ damage = 0;+ inflight_input = input_t;+ flip_at = ((t + busy + gpu) / P + 1) * P;+ pending = 1;+ fpipe_note(&fp, (double)(cpu + sub) / 1000, P / 1000.0);+ }+ /* then, like render_frame, the next frame may be painted ahead */+ if (fpipe_step(fpipe_engaged(&fp), pending, prep, 0, damage) == FPIPE_PREPARE) {+ busy += cpu;+ prep = damage ? FPIPE_READY : FPIPE_EMPTY;+ prep_input = input_t;+ damage = 0;+ r.prepares++;+ } else if (step == FPIPE_WAIT && pending && !fpipe_blocked(fpipe_engaged(&fp), pending, prep) && damage) r.spins++;+ t += busy;+ if (!busy) t = flip_at >= 0 ? (flip_at > t ? flip_at : t + 1) : t + P / 4; /* poll until the next event */+ }+ return r;+}++static void test_sim(void) {+ /* the Pi 5 under an all-core load (docs/BENCHMARK.md): about 13 ms of CPU, 1 ms to submit, 3 ms of GPU */+ Sim off = simulate(FPIPE_OFF, 13000, 1000, 3000, 1, 600);+ Sim on = simulate(FPIPE_ALWAYS, 13000, 1000, 3000, 1, 600);+ Sim au = simulate(FPIPE_AUTO, 13000, 1000, 3000, 1, 600);+ CHECK(off.shown >= 299 && off.shown <= 301 && off.prepares == 0); /* every other vblank: 30 Hz */+ CHECK(on.shown >= 598 && on.prepares >= 597); /* every vblank: 60 Hz */+ CHECK(au.shown >= 596); /* auto engages after two frames */+ CHECK(on.max_latency <= 3 * 16667 && off.max_latency <= 3 * 16667); /* input to screen: within 3 periods */+ /* a light frame: every vblank either way, and auto never engages */+ Sim l_off = simulate(FPIPE_OFF, 5000, 1000, 2000, 1, 600), l_au = simulate(FPIPE_AUTO, 5000, 1000, 2000, 1, 600);+ CHECK(l_off.shown >= 598 && l_au.shown >= 598 && l_au.prepares == 0);+ CHECK(l_au.max_latency <= 2 * 16667); /* not one frame more than without it */+ /* heavier than a period even pipelined: one frame per 21 ms of CPU (about 47 a second instead of 30),+ * never more than one ahead (input within 3 periods), and no busy loop */+ Sim h = simulate(FPIPE_ALWAYS, 20000, 1000, 3000, 1, 600), h_off = simulate(FPIPE_OFF, 20000, 1000, 3000, 1, 600);+ CHECK(h.shown >= 460 && h.shown <= 480 && h.spins == 0 && h.max_latency <= 3 * 16667);+ CHECK(h_off.shown >= 299 && h_off.shown <= 301);+ CHECK(on.spins == 0 && au.spins == 0);+ /* idle: one change is built and shown at once, nothing is prepared */+ Sim idle = simulate(FPIPE_ALWAYS, 13000, 1000, 3000, 0, 60);+ CHECK(idle.shown == 1 && idle.prepares == 0);+}++int main(void) {+ test_mode();+ test_auto();+ test_step();+ test_sim();+ printf("fpipe-test: %d checks, %d failed\n", checks, fails);+ return fails ? 1 : 0;+}diff --git a/server/tests/lordbench_test.c b/server/tests/lordbench_test.cindex c5201e0..210047d 100644--- a/server/tests/lordbench_test.c+++ b/server/tests/lordbench_test.c@@ -168,6 +168,30 @@ int main(void) {lbf_presented(5000, 0); /* a presentation stamped before the event: not a latency */lbf_presented(6000, 0);CHECK(lbf_end(&fs) == 0 && fs.inputs == 0, "no negative latency");+ /* the frame pipeline (fpipe.h): a frame painted ahead holds the events applied before its paint, and they end+ * at its own presentation, one flip after the one it was painted during */+ lbf_begin();+ lbf_presented(10000000, 16666667);+ lbf_input(11000000);+ lbf_painted(); /* frame A, while the flip before it is pending */+ lbf_input(12000000); /* after A's paint: the next frame */+ lbf_presented(26666667, 16666667); /* the flip A was painted during: none of A's */+ lbf_submitted(); /* A submitted at that flip */+ lbf_input(27000000);+ lbf_painted(); /* frame B: 12 and 27 */+ lbf_presented(43333334, 16666667); /* A shown: 32.33 ms */+ lbf_submitted();+ lbf_presented(60000000, 16666667); /* B shown: 48 and 33 ms */+ CHECK(lbf_end(&fs) == 0 && fs.inputs == 3 && fabs(fs.in_p50 - 33) < 0.001 && fabs(fs.in_p99 - 48) < 0.001,+ "pipelined input: %d events, p50 %f p99 %f", fs.inputs, fs.in_p50, fs.in_p99);+ lbf_begin(); /* a prepared frame dropped: its events go with the next one */+ lbf_input(1000);+ lbf_painted();+ lbf_input(2000);+ lbf_built(1);+ lbf_presented(5000, 0);+ lbf_presented(6000, 0);+ CHECK(lbf_end(&fs) == 0 && fs.inputs == 2, "a dropped prepared frame's events: %d", fs.inputs);/* ---- frame phases: what each part of a frame took, and why it was late ---- */{@@ -233,7 +257,7 @@ int main(void) {ps.g0_refused[6] == 1 && ps.unshown == 1, "counts: par %d, G0 %d of %d, refused %d %d (%d), unshown %d", ps.par,ps.g0_taken, ps.g0_tried, ps.g0_refused[0], ps.g0_refused[6], ps.g0_layer[1], ps.unshown);CHECK(fabs(ps.feed_mpx - 262144 / 4.0 / 1e6) < 1e-9 && fabs(ps.feed.p99 - 1.5) < 1e-6, "catch-up %f Mpx %f ms", ps.feed_mpx, ps.feed.p99);- char pt[1024];+ char pt[1536];lbf_phases_text(&ps, pt, sizeof pt);CHECK(strstr(pt, "CPU-late 1") && strstr(pt, "not layer 1 [invalid 1]") && strstr(pt, "not feed 1") && strstr(pt, "unshown 1"),"phases text: %s", pt);@@ -250,6 +274,36 @@ int main(void) {lbf_phase_shown(F0 + P, 0);lbf_phases(&ps);CHECK(ps.n == 1 && ps.on_time == 1 && ps.gpu_known == 0 && ps.gpu.p50 == 0, "LordBench's phases: %d frames", ps.n);+ /* a frame the pipeline prepared: painted during the flip before (12 ms), held until that flip, submitted+ * at it in 2 ms and shown at the next vblank: on time, its build is both parts */+ LbPhase e = { .piped = 1, .prev_flip = F0 + 7 * P, .wake = F0 + 6 * P + 100000, .from_flip = 1, .period = (uint32_t)P,+ .paint_ms = 10, .g0 = -1, .g0_why = -1, .g0_sub = -1 };+ e.pre_end = e.wake + 300000;+ e.render0 = e.pre_end + 200000;+ e.built = e.render0 + 12000000;+ e.swake = F0 + 7 * P + 100000;+ e.sfrom_flip = 1;+ e.sub0 = e.swake + 200000;+ e.sub1 = e.sub0 + 2000000;+ e.submit = e.sub1;+ lbf_drag_begin();+ lbf_phase_built(&e);+ lbf_phase_shown(F0 + 8 * P, e.sub1 + 3000000);+ lbf_drag_end(&ps);+ CHECK(ps.n == 1 && ps.piped == 1 && ps.on_time == 1 && fabs(ps.build.p50 - 14) < 1e-3 && fabs(ps.wake.p50 - 0.1) < 1e-3 &&+ fabs(ps.pre.p50 - 0.3) < 1e-3 && fabs(ps.held.p50 - (P / 1e6 + 0.3 - 12.6)) < 1e-3 && fabs(ps.gpu.p50 - 3) < 1e-3,+ "a prepared frame: %d (%d piped), on time %d, render %f, wake %f, before %f, held %f, GPU %f", ps.n, ps.piped,+ ps.on_time, ps.build.p50, ps.wake.p50, ps.pre.p50, ps.held.p50, ps.gpu.p50);+ lbf_phases_text(&ps, pt, sizeof pt);+ CHECK(strstr(pt, "prepared ahead 1 (held for the flip 4.37/4.37)") != NULL, "phases text: %s", pt);+ e.sub0 = e.swake + 17000000; /* submitted late: after the vblank it was meant for */+ e.sub1 = e.sub0 + 1000000;+ e.submit = e.sub1;+ lbf_drag_begin();+ lbf_phase_built(&e);+ lbf_phase_shown(F0 + 9 * P, 0);+ lbf_drag_end(&ps);+ CHECK(ps.n == 1 && ps.cpu_late == 1, "a prepared frame submitted late: CPU-late %d", ps.cpu_late);}/* ---- the synthetic pointer: its path, and the thread's events through the eventfd ---- */