Skip to content

Apple's open-source XNU kernel on Android phones: its own loader and drivers, Metal on the Adreno. Runs from RAM; a reboot gives the phone back to Android.

C

A screen recording that keeps the last of a long run, and a line for every frame of the window server

vcvkk1a7a760fbb5d
diff --git a/README.md b/README.md
index 3580317..4149006 100644
--- a/README.md
+++ b/README.md
@@ -184,7 +184,7 @@ XL_SECS=900 XL_CMDLINE="touchscreen=1 cpufreq=1 bootcpu=7 cpus=4" ./xl run #
| `bootcpu=7` | запустить ядро на самом большом ядре процессора (загрузчик остаётся на нулевом и наблюдает) |
| `cpus=N` | использовать N ядер процессора: то, на котором ядро запущено, и ещё N−1 ближайших к нему по номеру (после 7 — 6, 5, 4, потом малые 3, 2, 1); их запускает само ядро через PSCI. `cpumask=M` — выбрать по позиции в этом порядке (бит 0 — загрузочное). Без параметра — одно ядро. **На телефоне не проверено**, порядок проверки — в конце раздела |
| `cpufreq=1` | поднять частоту до максимума (иначе 300 МГц) всем группам ядер, в которых есть наши |
-| `rec_secs=N` | записывать экран N секунд (`rec_start=S`, `rec_mb=M`, не больше 640); потом `./xl recording out.mp4` |
+| `rec_secs=N` | записывать экран N секунд (`rec_start=S`, `rec_mb=M`, не больше 640); потом `./xl recording out.mp4`. `rec_ring=1` — по кругу: когда память кончилась, запись идёт поверх самых старых кадров, и к концу сеанса в ней последнее (для того, что случается раз за долгий сеанс: сеанс кончают сразу после — пятью пальцами); `rec_step=N` — каждый N-й пиксель вместо третьего (4 — в 1,8 раза больше кадров). Память на запись отнимается у системы на весь сеанс: с `rec_mb=640` в долгом сеансе iOS убивала процессы ради памяти, вплоть до SpringBoard (респринги), с 256 — нет. `XL_REC_KEEP=1 ./xl recording` оставляет саму запись в `logs/recording.raw`; `tools/recording.py logs/recording.raw out.mp4 --frames DIR` раскладывает её на кадры с именами по времени ядра в миллисекундах — тому же, что в строках `glframes=1` |
| `-v` | текст загрузки на экране (по умолчанию у `xl run`, если `XL_CMDLINE` не задана) |
| `applog=1` / `applog=2` | stderr приложений в журнал / плюс os_log всех процессов |
| `applog_name=ИМЯ[,ИМЯ]` | вместе с `applog=1`: os_log только тех программ, чьи имена файлов начинаются с одного из ИМЁН (так читают расширения приложений — виджеты, обои: их запускает launchd по просьбе других процессов, описания задания у нас для них нет) |
@@ -264,6 +264,7 @@ CoreAnimation; если GPU не запустился, в журнале одн
Шейдеры CoreAnimation считаются с полной точностью (`lowp` и `mediump` в них определены как `highp`): 16-битного
`mediump` у Adreno не хватало на таблицу цветов градиента — фон виджета «Погода» шёл полосами; `glprecision=0`
возвращает как написано.
+`glframes=1` — строка на каждый кадр оконного сервера: когда он кончен (время ядра), сколько отрисовок, в какую поверхность экрана, прямоугольник отсечения первой отрисовки в неё и отрисовки размытия с переменным радиусом; вместе с `rec_ring=1` кадры записи сопоставляются со строками.
`glcensus=1` ведёт перепись: каждая отрисовка без программы (номер и имя функции, где на экране, с какими
текстурами и числами, чем нарисована здесь) — при первой встрече, и каждые десять секунд счёт по функциям; туда же
попадают функции, о которых CoreAnimation спросил и получил отказ.
diff --git a/tools/recording.py b/tools/recording.py
index cfb8a6a..883d7df 100755
--- a/tools/recording.py
+++ b/tools/recording.py
@@ -1,19 +1,23 @@
#!/usr/bin/env python3
"""Make a video of a screen recording left by the kernel's display driver (boot-arg rec_secs=N).
-usage: recording.py RECORDING OUT.mp4 [--max-hold SECONDS]
+usage: recording.py RECORDING OUT.mp4 [--max-hold SECONDS] [--frames FOLDER]
With --max-hold, no picture stays longer than that: the stretches in which nothing changes on the
screen (an emulated machine thinking about starting an application) are cut short, while whatever
moves keeps its real speed.
RECORDING is the piece of memory as the loader hands it over ("xl recording"; under QEMU, the
-monitor's pmemsave): a header { "RWREC001", width, height, frames }, then for every frame the
-time it was shown (nanoseconds from the first) and width x height pixels of three bytes (R, G, B).
+monitor's pmemsave): a header { "RWREC003", width, height, frames, oldest, began }, then for every
+frame the time it was shown (nanoseconds from the first) and width x height pixels of three bytes
+(R, G, B). With boot-arg rec_ring=1 the frames are a ring, "oldest" the place of the oldest, and what
+is kept begins wherever the run's last frames began; "began" is when the first frame was recorded,
+in nanoseconds of the kernel's time. ("RWREC002", a kernel before that, has no "began".)
The frames are not evenly spaced -- one is recorded whenever the screen changes -- so each is
repeated for as long as it stayed on the screen, at 30 pictures a second, and ffmpeg encodes
that. The picture is enlarged twice (it was recorded at a third of the panel's resolution).
"""
+import os
import struct
import subprocess
import sys
@@ -25,17 +29,35 @@ def main():
if len(sys.argv) < 3:
sys.exit(__doc__)
data = open(sys.argv[1], 'rb').read()
- magic, width, height, frames, _ = struct.unpack_from('<8sIIII', data, 0)
- if magic != b'RWREC002' or not width or not height:
+ magic, width, height, frames, oldest = struct.unpack_from('<8sIIII', data, 0)
+ if magic not in (b'RWREC002', b'RWREC003') or not width or not height:
sys.exit('recording.py: not a recording')
+ began = struct.unpack_from('<Q', data, 24)[0] / 1e9 if magic == b'RWREC003' else 0.0
size = width * height * 3
- times, offsets, at = [], [], 24
+ times, offsets, at = [], [], 24 if magic == b'RWREC002' else 32
for _ in range(frames):
if at + 8 + size > len(data):
break
times.append(struct.unpack_from('<Q', data, at)[0] / 1e9)
offsets.append(at + 8)
at += 8 + size
+ if magic == b'RWREC003' and 0 < oldest < len(times):
+ # (rec_ring: the memory went round; the oldest frame is at that place, the newest just before it)
+ times = times[oldest:] + times[:oldest]
+ offsets = offsets[oldest:] + offsets[:oldest]
+ if '--frames' in sys.argv:
+ # every frame as a picture of its own (PPM), named by when it was shown in the kernel's time, in
+ # milliseconds: what the window server's lines with glframes=1 say ("f N at S.mmm s")
+ folder = sys.argv[sys.argv.index('--frames') + 1]
+ os.makedirs(folder, exist_ok=True)
+ for t, o in zip(times, offsets):
+ with open(os.path.join(folder, f'{int((began + t) * 1000):010d}.ppm'), 'wb') as f:
+ f.write(b'P6 %d %d 255\n' % (width, height) + data[o:o + size])
+ print(f'{folder}: {len(times)} frames, from {began + times[0]:.3f} s to {began + times[-1]:.3f} s of the kernel\'s time')
+ first = times[0] if times else 0.0
+ if first:
+ print(f'recording.py: the frames kept begin at {first:.3f} s of the recording (rec_ring)')
+ times = [t - first for t in times]
if not times:
sys.exit('recording.py: no frames in the recording')
real = times[-1]
diff --git a/userland/rc.jobs.strip b/userland/rc.jobs.strip
new file mode 100644
index 0000000..db6fc72
--- /dev/null
+++ b/userland/rc.jobs.strip
@@ -0,0 +1,8 @@
+# Commands run by the launchd job com.redwood.demo (see com.redwood.demo.plist).
+# The dark strip in Search, on demand (RC_JOBS=rc.jobs.strip, a run of about a minute and a half;
+# the way the user found): Search pulled down, then the gesture from the bottom edge begun and let
+# go after half a centimetre, eight times. Each time the strip under the status bar darkens to a
+# quarter and comes back -- and now and then stays at about half (two of eight in run 093851). To
+# be watched with boot-args rec_secs=3600 rec_ring=1 rec_step=4 rec_mb=256 glframes=1.
+/redwood/bin/rwtouch wait 40 swipe 0.5 0.985 0.5 0.3 200 wait 8 swipe 0.5 0.4 0.5 0.7 300 wait 6 swipe 0.5 0.995 0.5 0.965 150 wait 4 swipe 0.5 0.995 0.5 0.965 150 wait 4 swipe 0.5 0.995 0.5 0.96 200 wait 4 swipe 0.5 0.995 0.5 0.97 120 wait 4
+/redwood/bin/rwtouch swipe 0.5 0.995 0.5 0.965 150 wait 4 swipe 0.5 0.995 0.5 0.96 250 wait 4 swipe 0.5 0.995 0.5 0.965 150 wait 4 swipe 0.5 0.995 0.5 0.95 200 wait 8
diff --git a/userland/src/eagl.c b/userland/src/eagl.c
index 83f868e..6f5dc09 100644
--- a/userland/src/eagl.c
+++ b/userland/src/eagl.c
@@ -52,7 +52,8 @@
* wait_for_frames), glfetch=0 keeps EXT_shader_framebuffer_fetch from CoreAnimation (see
* fetch_offered), glprecision=0 leaves the shaders' "lowp" and "mediump" as they are (see all_highp), glenv=NAME=VALUE puts that into the environment for the library to find
* (its own switches: FD_MESA_DEBUG=sysmem, say), vblur=0 blurs nothing whose radius varies and
- * vblur=2 paints such blurs by their masks instead (see vblur_on).
+ * vblur=2 paints such blurs by their masks instead (see vblur_on), glframes=1 writes a line for
+ * every frame (see frame_end()).
*/
#include <CoreFoundation/CoreFoundation.h>
#include <IOKit/IOKitLib.h>
@@ -125,6 +126,7 @@ typedef GLDEBUGPROC GLDEBUGPROCKHR;
struct eagl;
static int eagl_tracing; /* this frame's calls are written down (boot-arg gltrace=FIRST-LAST) */
+static int frame_lines; /* boot-arg glframes=1: a line for every frame (see frame_end()) */
static void eagl_trace(const char *format, ...) __attribute__((format(printf, 1, 2)));
static void eagl_trace_floats(const float *values, int count);
static void eagl_enter(struct eagl *e, unsigned offset);
@@ -255,6 +257,12 @@ struct eagl {
GLenum type;
} arrays[8]; /* the vertex arrays, as given */
bool in_frame;
+ struct { /* what the line a frame gets with glframes=1 says (frame_end()) */
+ bool taken, boxed; /* the first draw or clear into the display's picture seen; scissored */
+ GLint box[4];
+ uint32_t surface; /* that picture's IOSurface */
+ unsigned vrd, vrb, into_display;
+ } line;
/* a frame's worth */
unsigned calls, draws, triangles, uploads, clears, last_draws;
uint64_t upload_bytes, began, attaching, flushing;
@@ -803,6 +811,20 @@ frame_end(struct eagl *e, bool finished)
after = now_us();
e->in_frame = false;
e->frames++;
+ if (frame_lines) {
+ /*
+ * One line a frame, for matching with a screen recording (the kernel's rec_ring, whose log
+ * says when it began in the same time): when the frame ended, in the kernel's time; what was
+ * drawn; into which of the display's surfaces and, of the first draw or clear into it, the
+ * scissor box -- CoreAnimation draws only what has changed, inside that box, and leaves the
+ * rest of the surface as it was --; and the draws of the blur whose radius varies.
+ */
+ say("redwood-gl: f %u at %llu.%03llu s: %u draws, %u clears; display surface %u, %u draws into it, box %s %d,%d %dx%d; "
+ "vrd %u vrb %u; %u in place, %u copied\n", e->frames, after / 1000000, after / 1000 % 1000, e->draws, e->clears,
+ e->line.surface, e->line.into_display, e->line.boxed ? "on" : "off", e->line.box[0], e->line.box[1], e->line.box[2],
+ e->line.box[3], e->line.vrd, e->line.vrb, in_place, copied);
+ bzero(&e->line, sizeof(e->line));
+ }
if (in_place == 0 && copied == 0 && e->draws != 0 && unfollowed < 8) {
/* (whatever it was drawn into is shown as it is, drawn or not: this is to be seen in the log) */
unfollowed++;
@@ -851,6 +873,16 @@ drawing(struct eagl *e)
frame_begin(e);
if (e->framebuffer != 0 && e->colour != 0 && (image = image_of(e, e->colour)) != NULL && image->target) {
image->drawn = image->painted = true;
+ if (frame_lines && image->width >= 1000 && image->height >= 2000) {
+ /* the display's picture (a surface of its size): where this frame first draws into it */
+ e->line.into_display++;
+ if (!e->line.taken) {
+ e->line.taken = true;
+ e->line.boxed = glIsEnabled(GL_SCISSOR_TEST);
+ glGetIntegerv(GL_SCISSOR_BOX, e->line.box);
+ e->line.surface = IOSurfaceGetID(image->surface);
+ }
+ }
}
}
@@ -1903,6 +1935,11 @@ plain_draw_begin(struct eagl *e, GLint enabled[4])
GLint rows = 0;
GLuint program;
+ if (e->program == 0) {
+ e->line.vrd += function == FUNCTION_VARIABLE_DOWNSAMPLE;
+ e->line.vrb += function == FUNCTION_VARIABLE_BLUR;
+ }
+
if (e->unclear != 0 && reads_what_it_writes(e)) {
e->unclear = 0;
}
@@ -2933,6 +2970,7 @@ install(void)
}
all_shaders = strstr(arguments, "glshaders=1") != NULL;
census_on = strstr(arguments, "glcensus=1") != NULL;
+ frame_lines = strstr(arguments, "glframes=1") != NULL;
full_precision = strstr(arguments, "glprecision=0") == NULL;
vblur_on = strstr(arguments, "vblur=0") != NULL ? 0 : strstr(arguments, "vblur=2") != NULL ? 2 : 1;
if ((argument = strstr(arguments, "glplain=")) != NULL) {
diff --git a/xl b/xl
index d978ea2..1f45952 100755
--- a/xl
+++ b/xl
@@ -157,7 +157,8 @@ recording() { # recording [file.mp4]
local out=${1:-logs/recording.mp4}
fb oem recording | grep -q OKAY || { echo "xl: the last run left no recording (boot-arg rec_secs=N)"; return 1; }
FB_TIMEOUT=120 fb get_staged logs/recording.raw >/dev/null
- tools/recording.py logs/recording.raw "$out" && rm -f logs/recording.raw
+ # (XL_REC_KEEP=1: the recording itself stays in logs/recording.raw, for tools/recording.py --frames)
+ tools/recording.py logs/recording.raw "$out" && { [ -n "$XL_REC_KEEP" ] || rm -f logs/recording.raw; }
}
ramdisk() { # ramdisk [<file>|none]
diff --git a/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp b/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp
index edbeaf4..69744f5 100644
--- a/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp
+++ b/xnu/files/iokit/Kernel/MiniIOMobileFramebuffer.cpp
@@ -122,10 +122,23 @@ static TUNABLE(uint32_t, gFBTrace, "fbtrace", 3);
* hands it to the host ("xl recording"), where tools/recording.py makes a video of it.
*
* struct rec_header, then per frame: struct rec_frame, width * height pixels (R, G, B)
+ *
+ * Boot-arg rec_ring=1: when the memory is full the recording does not stop but goes on over its
+ * oldest frames, so that what it holds at the end is the last of the run: for something that
+ * happens once in a long session, the run is ended right after it (five fingers) and the frames
+ * before it are there. The header ("RWREC003") says where the oldest frame is (frames are all
+ * the same size, one after the other) and when the recording began, in the kernel's time, for the
+ * frames to be matched with what the window server's per-frame lines (glframes=1,
+ * userland/src/eagl.c) say. rec_step=N (2 to 8, default 3) keeps one pixel of every N instead of 3:
+ * N = 4 holds 1.8 times as many frames, 6 four times. The memory is taken from the system's for the
+ * whole run: 640 MB of it under a long session of applications had iOS killing processes for
+ * memory, SpringBoard and the window server among them (two respring in one run); 256 did not.
*/
static TUNABLE(uint32_t, gRecSeconds, "rec_secs", 0);
static TUNABLE(uint32_t, gRecStart, "rec_start", 0);
static TUNABLE(uint32_t, gRecMegabytes, "rec_mb", 480);
+static TUNABLE(uint32_t, gRecRing, "rec_ring", 0);
+static TUNABLE(uint32_t, gRecStep, "rec_step", 3);
static TUNABLE(uint32_t, gDiff, "fb_diff", 0);
/*
* Boot-arg fb_filter=0: the picture is scaled to the panel nearest neighbour, as it once was. The
@@ -144,13 +157,19 @@ refreshHz(void)
{
return gRefreshHz >= 1 && gRefreshHz <= 240 ? gRefreshHz : kFBRefreshHz;
}
-#define REC_STEP 3
+static inline uint32_t
+recStep(void)
+{
+ return gRecStep >= 2 && gRecStep <= 8 ? gRecStep : 3;
+}
+#define REC_STEP recStep()
struct rec_header {
- char magic[8]; /* "RWREC002" */
+ char magic[8]; /* "RWREC003" */
uint32_t width, height;
uint32_t frames;
- uint32_t reserved;
+ uint32_t oldest; /* the place of the oldest frame (rec_ring; 0 until the memory has once been full) */
+ uint64_t began; /* when the first frame was recorded, in nanoseconds of the kernel's time */
};
struct rec_frame {
uint64_t nanoseconds; /* since the first frame of the recording */
@@ -235,6 +254,7 @@ public:
uint64_t fRecSize, fRecUsed;
uint64_t fFirstFrame, fRecFirst; /* when the first frame was shown, and the first recorded */
bool fRecDone;
+ bool fRecWrapped; /* (rec_ring) the memory has been full once: frames go over the oldest */
virtual bool start(IOService * provider) APPLE_KEXT_OVERRIDE;
void publishModes(void);
@@ -383,14 +403,15 @@ IOMobileFramebuffer::start(IOService * provider)
fRecSize = (uint64_t) gRecMegabytes << 20;
fRecUsed = sizeof(*header);
bzero(header, sizeof(*header));
- memcpy(header->magic, "RWREC002", 8);
+ memcpy(header->magic, "RWREC003", 8);
header->width = fWidth / REC_STEP;
header->height = fHeight / REC_STEP;
loader_recording[0] = memory->getPhysicalSegment(0, NULL);
loader_recording[1] = fRecUsed;
loader_recording[2] = (uint64_t) (uintptr_t) fRecBase;
- IOLog("MiniIOMobileFramebuffer: recording %u s of the screen at %ux%u from second %u, into %u MB at %#llx\n",
- gRecSeconds, header->width, header->height, gRecStart, gRecMegabytes, loader_recording[0]);
+ IOLog("MiniIOMobileFramebuffer: recording %u s of the screen at %ux%u from second %u, into %u MB at %#llx%s\n",
+ gRecSeconds, header->width, header->height, gRecStart, gRecMegabytes, loader_recording[0],
+ gRecRing ? ", the last of it kept (rec_ring)" : "");
} else {
IOLog("MiniIOMobileFramebuffer: no memory for a recording of %u MB\n", gRecMegabytes);
}
@@ -410,7 +431,7 @@ void
IOMobileFramebuffer::record(MiniIOSurface * s)
{
struct rec_header * header = (struct rec_header *) fRecBase;
- uint64_t now = mach_absolute_time(), since_first, since_start = 0;
+ uint64_t now = mach_absolute_time(), since_first, since_start = 0, frame_size;
struct rec_frame * frame;
uint8_t * to;
@@ -425,15 +446,26 @@ IOMobileFramebuffer::record(MiniIOSurface * s)
return;
}
if (fRecFirst == 0) {
+ uint64_t began;
+
fRecFirst = now;
+ absolutetime_to_nanoseconds(now, &began);
+ header->began = began;
+ IOLog("MiniIOMobileFramebuffer: the recording begins at %llu.%03llu s of the kernel's time (mach_absolute_time)\n",
+ began / NSEC_PER_SEC, began / NSEC_PER_MSEC % 1000);
}
absolutetime_to_nanoseconds(now - fRecFirst, &since_start);
- if (since_start > (uint64_t) gRecSeconds * NSEC_PER_SEC ||
- fRecUsed + sizeof(*frame) + (uint64_t) header->width * header->height * 3 > fRecSize) {
+ frame_size = sizeof(*frame) + (uint64_t) header->width * header->height * 3;
+ if (since_start > (uint64_t) gRecSeconds * NSEC_PER_SEC || (fRecUsed + frame_size > fRecSize && !gRecRing)) {
fRecDone = true;
IOLog("MiniIOMobileFramebuffer: recording finished: %u frames, %llu MB\n", header->frames, fRecUsed >> 20);
return;
}
+ if (fRecUsed + frame_size > fRecSize) {
+ /* (rec_ring) round again: over the oldest, which is the next one */
+ fRecUsed = sizeof(*header);
+ fRecWrapped = true;
+ }
frame = (struct rec_frame *) (fRecBase + fRecUsed);
frame->nanoseconds = since_start;
to = (uint8_t *) (frame + 1);
@@ -448,9 +480,18 @@ IOMobileFramebuffer::record(MiniIOSurface * s)
*to++ = (uint8_t) pixel;
}
}
- fRecUsed += sizeof(*frame) + (uint64_t) header->width * header->height * 3;
- header->frames++;
- loader_recording[1] = fRecUsed;
+ fRecUsed += frame_size;
+ if (!fRecWrapped) {
+ header->frames++;
+ loader_recording[1] = fRecUsed;
+ } else {
+ /* the memory full of frames: all of them are handed over, from the one after this */
+ header->oldest = (uint32_t) ((fRecUsed - sizeof(*header)) / frame_size);
+ if (header->oldest >= header->frames) {
+ header->oldest = 0;
+ }
+ loader_recording[1] = sizeof(*header) + (uint64_t) header->frames * frame_size;
+ }
}
MiniIOSurface *