Skip to content

Apple's open-source XNU kernel on Android phones: its own loader and drivers, Metal on the Adreno. Runs from RAM; a reboot gives the phone back to Android.

C

The CPU reference: 255 of Apple's shaders agree with LLVM, bit for bit, with no GPU

vcvkke4d3ef76ae39
diff --git a/docs/air.md b/docs/air.md
index 14e6395..0733e71 100644
--- a/docs/air.md
+++ b/docs/air.md
@@ -214,11 +214,26 @@ LLVM parameter type, and the NIR value is cut or converted to it.
A shader is reported as *working* only when its output was compared, never because it compiled.
Three references, each independent of the translator:
-1. **LLVM on the CPU (the strongest).** `tools/air/cpuref` retargets the module to the host,
- compiles it with `llc` and calls the entry point from a harness that fills its parameters from
- the same metadata (buffers with seeded data, varyings, position, thread ids) and implements the
- `air.*` functions in C. LLVM — not our reader, not our translator — decides what the shader
- means. Needs no Apple hardware and no GPU.
+1. **LLVM on the CPU (the strongest).** `tools/air/cpuref.py` changes the module's triple to the
+ host's, compiles it with `llc`, fills a buffer per `air.buffer` argument with a fixed pattern
+ and calls the entry point; `air2nir --eval` replaces the same buffer loads in *our* NIR with
+ the same bytes and folds it with `nir_opt_constant_folding` (no algebraic rewriting, so the
+ arithmetic stays IEEE and comparable); the two sets of numbers are compared. LLVM — not our
+ reader, not our translator — decides what the shader means, and neither side can see the
+ other's answer. Needs no Apple hardware and no GPU.
+
+ **255 shaders agree, bit for bit, and none differ.** The call into the shader is made from a
+ wrapper written in LLVM IR rather than from C, because a shader returns a struct of vectors by
+ value and how `llc` lowers that is its own business: a C declaration of the same struct agrees
+ for a single vector and stops agreeing past it (this cost an afternoon's false "243 differ").
+ With the wrapper, only `void(const void *, void *)` crosses into C and the question does not
+ arise.
+
+ What it cannot decide, and says so rather than passing: a shader with a non-buffer argument —
+ a varying, `air.position`, a thread id — because what is in it is the GPU's business (19 in
+ AppleMetalGLRenderer, all of them the texture ones), and a compute kernel, which returns
+ nothing and writes through pointers (47 of them). Reading the buffers back after the call
+ would settle the kernels and is M4's acceptance test; it is not written.
2. **The same shader through our NIR on two different Mesa backends.** `air-run` renders (or
dispatches) the translated NIR with Gallium: on the host through softpipe (NIR → TGSI →
an interpreter in C), on the phone through freedreno/ir3 on the Adreno. Both outputs are
@@ -277,14 +292,17 @@ an unhandled metadata key), so "features not yet handled" is a ranked list, not
`llvm-dis` on every one.
- **M1** — `clearFragment01` reaches *adreno*: translated, compiled by ir3, rendered on the
Adreno, framebuffer equal to the colour in its constant buffer.
- **Compiled, not yet run.** It is `nir_load_ubo` + a store to `FRAG_RESULT_DATA0` as predicted,
- and ir3 turns it into five Adreno instructions (four `mov`s out of the constant file, which is
- where ir3 promoted the UBO load to). The framebuffer half of the test needs the phone.
+ **Compiled and checked against LLVM, not yet run on the GPU.** It is `nir_load_ubo` + a store
+ to `FRAG_RESULT_DATA0` as predicted, and ir3 turns it into five Adreno instructions (four
+ `mov`s out of the constant file, where ir3 promoted the UBO load). Over a buffer of known
+ bytes, LLVM's own execution of the module and our folded NIR give the same four floats, bit for
+ bit. The framebuffer half of the test — ir3's code actually running on the Adreno and the
+ colour read back — needs the phone and has not been done.
- **M2** — the 283 modules with no `air.*` call: a stated fraction reach *ir3*, a growing subset
*softpipe*/*adreno*.
- **262 of the 283 reach ir3.** The other 21 are `air.visible` library functions, which are not
- entry points at all (below). Counting every entry point rather than every module: 275 of 321,
- 86%.
+ **262 of the 283 reach ir3**, and 255 of them agree with LLVM numerically. The other 21 are
+ `air.visible` library functions, which are not entry points at all (below). Counting every
+ entry point rather than every module: 275 of 321, 86%. Nothing has run on the Adreno.
- **M3** — textures and samplers, each form with a test that renders a known image.
Not started, and it is what nearly everything left is waiting for: 46 modules stop at a texture
call, including all 11 of AV1SW's.
@@ -308,7 +326,7 @@ M2 count (262) differ.
```
tools/air/mk-host.sh # Mesa for this machine (x86-64): the frontend, ir3, softpipe
-tools/air/check.sh # the reader against llvm-dis, then the corpus report
+tools/air/check.sh # the reader against llvm-dis, the corpus report, and LLVM
```
and on one shader:
@@ -316,8 +334,11 @@ and on one shader:
```
build/mesa-host/build/src/compiler/air/air2nir --ll FILE[.metallib] [FUNCTION] # what the reader read
build/mesa-host/build/src/compiler/air/air2nir FILE[.metallib] [FUNCTION] # the NIR
+build/mesa-host/build/src/compiler/air/air2nir --eval FILE[.metallib] [FUNCTION] # folded to numbers
build/mesa-host/build/src/gallium/drivers/freedreno/ir3_compiler \
--chip-id 0x007506030500 FILE[.metallib] [FUNCTION] # Adreno code
+tools/air/cpuref.py --compare FILE[.metallib] [FUNCTION] # ours against LLVM's own answer
+tools/air/cpuref.py --all FILE.metallib # every function of a library
```
`0x007506030500` is how freedreno names the redwood's Adreno 7c+ Gen 3 (A635); it has no
diff --git a/tools/air/check.sh b/tools/air/check.sh
index 677f811..da1bf15 100755
--- a/tools/air/check.sh
+++ b/tools/air/check.sh
@@ -52,4 +52,17 @@ else
echo "check.sh: wrote docs/air-report.md"
fi
-exit $reader_rc
+echo
+echo "== against LLVM on the processor, shader by shader"
+# The strongest check there is, and it needs no GPU: LLVM runs the same module
+# over the same buffer bytes, our NIR is folded over them, and the numbers are
+# compared. Only the class where this is decidable -- an entry point whose
+# arguments are all buffers and which returns its result -- is counted; the
+# rest say why not. See tools/air/cpuref.py.
+cpu_rc=0
+for f in "${LIBS[@]}"; do
+ printf '%-52s ' "$(basename "$(dirname "$f")")/$(basename "$f")"
+ python3 "$ROOT/tools/air/cpuref.py" --all "$f" 2>&1 | tail -1 || cpu_rc=1
+done
+
+exit $((reader_rc | cpu_rc))
diff --git a/tools/air/cpuref.py b/tools/air/cpuref.py
new file mode 100755
index 0000000..8b5292b
--- /dev/null
+++ b/tools/air/cpuref.py
@@ -0,0 +1,432 @@
+#!/usr/bin/env python3
+"""The CPU reference: run an AIR shader through LLVM on this machine.
+
+This is the strongest check the project has (docs/air.md section 4) and it
+needs no Apple hardware and no GPU. LLVM -- not our reader and not our
+translator -- decides what the shader means:
+
+ 1. the module's triple is changed to the host's and llc compiles it,
+ 2. a harness is generated from the entry point's own metadata: one buffer
+ per air.buffer argument, filled with a seeded pattern, and the result
+ printed field by field,
+ 3. the same shader goes through our frontend, and the two results are
+ compared.
+
+Step 3 is not automatic for every shader: a fragment shader's result depends
+on where it is in the buffer, and only the GPU knows that. What is automatic
+is the class this covers -- an entry point whose arguments are all constant or
+device buffers and whose result is a struct of vectors, which is 256 of the
+corpus's 271 fragment shaders. For those the NIR is interpreted here too, by
+nir_opt_constant_folding on a shader whose buffer loads have been replaced
+with the harness's bytes, so both sides are numbers and the comparison is
+exact.
+
+usage: cpuref.py FILE[.metallib] [FUNCTION] run and print LLVM's result
+ cpuref.py --compare FILE [FUNCTION] and compare against our NIR
+"""
+import argparse
+import os
+import re
+import struct
+import subprocess
+import sys
+import tempfile
+
+ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+sys.path.insert(0, os.path.join(ROOT, "tools"))
+import metallib # noqa: E402
+
+AIR2NIR = os.environ.get(
+ "AIR2NIR", os.path.join(ROOT, "build/mesa-host/build/src/compiler/air/air2nir"))
+
+# The pattern every buffer is filled with. Arbitrary, but fixed and not all
+# zeros: a shader that ignores its input and one that reads it must differ.
+def pattern(n):
+ return bytes((i * 7 + 13) & 0xff for i in range(n))
+
+
+class CType:
+ """One LLVM type, as C and as a struct.unpack format."""
+
+ def __init__(self, c, fmt, size):
+ self.c, self.fmt, self.size = c, fmt, size
+
+
+SCALARS = {
+ "float": CType("float", "f", 4),
+ "half": CType("unsigned short", "H", 2), # no C half; compared as bits
+ "double": CType("double", "d", 8),
+ "i1": CType("unsigned char", "B", 1),
+ "i8": CType("unsigned char", "B", 1),
+ "i16": CType("unsigned short", "H", 2),
+ "i32": CType("unsigned int", "I", 4),
+ "i64": CType("unsigned long long", "Q", 8),
+}
+
+
+def parse_type(t):
+ """<4 x float> -> (CType, 4); float -> (CType, 1)."""
+ t = t.strip()
+ m = re.match(r"<(\d+) x (.+)>$", t)
+ if m:
+ base = SCALARS.get(m.group(2).strip())
+ return (base, int(m.group(1))) if base else (None, 0)
+ base = SCALARS.get(t)
+ return (base, 1) if base else (None, 0)
+
+
+def split_fields(s):
+ """The top-level comma-separated members of a struct body."""
+ out, depth, cur = [], 0, ""
+ for ch in s:
+ if ch in "<[{":
+ depth += 1
+ elif ch in ">]}":
+ depth -= 1
+ if ch == "," and depth == 0:
+ out.append(cur)
+ cur = ""
+ else:
+ cur += ch
+ if cur.strip():
+ out.append(cur)
+ return out
+
+
+def disassemble(bc_path):
+ return subprocess.run(["llvm-dis", "-o", "-", bc_path], capture_output=True,
+ text=True, check=True).stdout
+
+
+def entry_of(ll):
+ """The entry point's name, its parameter types and its result type."""
+ stage = None
+ for name in ("vertex", "fragment", "kernel", "visible"):
+ if re.search(r"^!air\.%s = " % name, ll, re.M):
+ stage = name
+ break
+ m = re.search(r"^define[^@\n]*@([A-Za-z_0-9.$]+)\(([^)]*)\)[^\n]*\{$", ll, re.M)
+ if not m:
+ return None
+ # the LAST define is not necessarily the entry point; find the one the
+ # metadata names
+ ep = re.search(r"^!air\.(?:vertex|fragment|kernel|visible) = !\{!(\d+)\}", ll, re.M)
+ if ep:
+ node = re.search(r"^!%s = !\{ptr @([A-Za-z_0-9.$]+)," % ep.group(1), ll, re.M)
+ if node:
+ want = node.group(1)
+ for m2 in re.finditer(r"^define([^@\n]*)@([A-Za-z_0-9.$]+)\(([^)]*)\)[^\n]*\{$",
+ ll, re.M):
+ if m2.group(2) == want:
+ m = m2
+ break
+ ret = m.group(1).strip().split()[-1] if m.group(1).strip() else "void"
+ # "define <{ <4 x float> }> @f(...)": the return type may contain spaces
+ head = ll[m.start():m.start(1) + len(m.group(1))]
+ ret = head[len("define"):].strip()
+ for word in ("local_unnamed_addr", "internal", "fastcc", "unnamed_addr", "dso_local"):
+ ret = ret.replace(word, "").strip()
+ return dict(stage=stage, name=m.group(2), params=split_fields(m.group(3)), ret=ret)
+
+
+def buffer_args(ll, entry):
+ """Which parameters are buffers, from the entry-point metadata."""
+ ep = re.search(r"^!air\.(?:vertex|fragment|kernel|visible) = !\{!(\d+)\}", ll, re.M)
+ if not ep:
+ return {}
+ node = re.search(r"^!%s = !\{ptr @[^,]+, !(\d+), !(\d+)" % ep.group(1), ll, re.M)
+ if not node:
+ return {}
+ ins = re.search(r"^!%s = !\{([^}]*)\}" % node.group(2), ll, re.M)
+ if not ins:
+ return {}
+ out = {}
+ for ref in re.findall(r"!(\d+)", ins.group(1)):
+ line = re.search(r"^!%s = !\{(.*)\}$" % ref, ll, re.M)
+ if not line:
+ continue
+ body = line.group(1)
+ pm = re.match(r"\s*i32 (\d+)", body)
+ if not pm or '"air.buffer"' not in body:
+ continue
+ size = re.search(r'"air\.buffer_size", i32 (\d+)', body)
+ out[int(pm.group(1))] = int(size.group(1)) if size else 256
+ return out
+
+
+def build_and_run(ll, entry, bufs, workdir):
+ """Compile the module for this machine and call the entry point.
+
+ The call is made from a wrapper written in LLVM IR, not from C. A shader
+ returns a struct of vectors by value, and how llc lowers that is its own
+ business -- a C declaration of "the same" struct is not guaranteed to
+ agree, and for anything past a single vector it does not. So LLVM compiles
+ both sides of that call, the wrapper stores each result field into a flat
+ buffer, and the only thing C ever sees is void(const void *, void *).
+ """
+ ret = entry["ret"].strip()
+ body = re.match(r"<?\{(.*)\}>?$", ret)
+ members = split_fields(body.group(1)) if body else ([ret] if ret != "void" else [])
+ fields = []
+ for i, t in enumerate(members):
+ base, n = parse_type(t)
+ if base is None:
+ return None, "result field %d is %s, which this harness cannot read" % (i, t)
+ fields.append((base, n, t.strip()))
+ if not fields:
+ return None, "the entry point returns nothing to compare"
+
+ # the wrapper's own arguments: one pointer per buffer, and the output
+ wrap_params, call_args, pre = [], [], []
+ for i, t in enumerate(entry["params"]):
+ t = t.strip()
+ if "ptr" in t:
+ if i not in bufs:
+ return None, "parameter %d is a pointer that is not a buffer" % i
+ wrap_params.append("ptr %%in%d" % i)
+ m = re.search(r"addrspace\((\d+)\)", t)
+ if m:
+ pre.append(" %%p%d = addrspacecast ptr %%in%d to ptr addrspace(%s)"
+ % (i, i, m.group(1)))
+ call_args.append("ptr addrspace(%s) %%p%d" % (m.group(1), i))
+ else:
+ call_args.append("ptr %%in%d" % i)
+ else:
+ base, n = parse_type(t)
+ if base is None:
+ return None, "parameter %d is %s, which this harness cannot make" % (i, t)
+ # a non-buffer argument (a varying, a vertex id) is zero: what it
+ # really is only the GPU knows, so such a shader is not compared
+ return None, "parameter %d is %s, not a buffer" % (i, t)
+
+ # store each field at its own 16-byte slot of the output buffer
+ stores, slots = [], []
+ for i, (base, n, t) in enumerate(fields):
+ stores.append(" %%f%d = extractvalue %s %%r, %d" % (i, ret, i))
+ stores.append(" %%o%d = getelementptr i8, ptr %%out, i64 %d" % (i, i * 16))
+ stores.append(" store %s %%f%d, ptr %%o%d, align 1" % (t, i, i))
+ slots.append((base, n))
+
+ wrapper = """
+define void @__cpuref_call(%s, ptr %%out) {
+%s
+ %%r = call %s @%s(%s)
+%s
+ ret void
+}
+""" % (", ".join(wrap_params + []) if wrap_params else "ptr %unused",
+ "\n".join(pre), ret, entry["name"], ", ".join(call_args), "\n".join(stores))
+
+ host_ll = os.path.join(workdir, "m.ll")
+ with open(host_ll, "w") as f:
+ f.write(re.sub(r'target triple = "[^"]*"',
+ 'target triple = "%s"' % host_triple(), ll))
+ f.write(wrapper)
+ obj = os.path.join(workdir, "m.o")
+ llc = subprocess.run(["llc", "-filetype=obj", "-o", obj, host_ll],
+ capture_output=True, text=True)
+ if llc.returncode:
+ last = (llc.stderr or "").strip().splitlines()
+ return None, "llc refused it: " + (last[-1][:140] if last else "no message")
+
+ setup, args = [], []
+ for i in sorted(bufs):
+ setup.append(" static unsigned char buf%d[%d];" % (i, bufs[i]))
+ setup.append(" for (int i = 0; i < %d; i++) buf%d[i] = (i * 7 + 13) & 0xff;"
+ % (bufs[i], i))
+ args.append("buf%d" % i)
+ decl = ", ".join(["const void *"] * len(args) + ["void *"]) if args else "void *"
+
+ prints = []
+ for i, (base, n) in enumerate(slots):
+ for k in range(n):
+ at = "out + %d" % (i * 16 + k * base.size)
+ if base.fmt == "f":
+ prints.append(' { float v; __builtin_memcpy(&v, %s, 4);'
+ ' printf("%%.9g\\n", (double)v); }' % at)
+ elif base.fmt == "d":
+ prints.append(' { double v; __builtin_memcpy(&v, %s, 8);'
+ ' printf("%%.17g\\n", v); }' % at)
+ else:
+ prints.append(' { %s v; __builtin_memcpy(&v, %s, %d);'
+ ' printf("%%llu\\n", (unsigned long long)v); }'
+ % (base.c, at, base.size))
+
+ src = """#include <stdio.h>
+extern void __cpuref_call(%s);
+int main(void) {
+%s
+ static unsigned char out[%d];
+ __cpuref_call(%s);
+%s
+ return 0;
+}
+""" % (decl, "\n".join(setup), max(16 * len(fields), 16),
+ ", ".join(args + ["out"]) if args else "out", "\n".join(prints))
+
+ csrc = os.path.join(workdir, "harness.c")
+ with open(csrc, "w") as f:
+ f.write(src)
+ exe = os.path.join(workdir, "harness")
+ cc = subprocess.run(["cc", "-O0", "-o", exe, csrc, obj, "-lm"],
+ capture_output=True, text=True)
+ if cc.returncode:
+ last = cc.stderr.strip().splitlines()
+ return None, "the harness did not build: " + (last[-1][:140] if last else "?")
+ run = subprocess.run([exe], capture_output=True, text=True, timeout=60)
+ if run.returncode:
+ return None, "the harness crashed (signal %d)" % -run.returncode
+ return [l for l in run.stdout.splitlines() if l], None
+
+
+def host_triple():
+ out = subprocess.run(["llc", "--version"], capture_output=True, text=True).stdout
+ m = re.search(r"Default target: (\S+)", out)
+ return m.group(1) if m else "x86_64-unknown-linux-gnu"
+
+
+def main():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("file")
+ ap.add_argument("function", nargs="?")
+ ap.add_argument("--compare", action="store_true",
+ help="run ours too and compare the numbers")
+ ap.add_argument("--all", action="store_true",
+ help="every function of the file, comparing each")
+ ap.add_argument("--tolerance", type=float, default=0.0,
+ help="relative difference allowed (default: none, bit-exact)")
+ args = ap.parse_args()
+ if args.all:
+ return run_all(args)
+
+ with tempfile.TemporaryDirectory() as work:
+ if args.file.endswith(".metallib"):
+ b, header, funcs = metallib.parse(args.file)
+ picked = None
+ for f in funcs:
+ if not args.function or f.get("name") == args.function:
+ picked = f
+ break
+ if not picked:
+ sys.exit("cpuref.py: no function %s in %s" % (args.function, args.file))
+ bc, _ = metallib.bitcode_of(b, header, picked)
+ bc_path = os.path.join(work, "m.bc")
+ with open(bc_path, "wb") as f:
+ f.write(bc)
+ else:
+ bc_path = args.file
+
+ ll = disassemble(bc_path)
+ entry = entry_of(ll)
+ if not entry:
+ sys.exit("cpuref.py: no entry point found")
+ bufs = buffer_args(ll, entry)
+ print("cpuref: %s (%s), %d parameters, %d of them buffers" %
+ (entry["name"], entry["stage"], len(entry["params"]), len(bufs)))
+ for i, n in sorted(bufs.items()):
+ print(" parameter %d: a buffer of %d bytes, filled with (i * 7 + 13) & 0xff"
+ % (i, n))
+
+ values, why = build_and_run(ll, entry, bufs, work)
+ if why:
+ print("cpuref: cannot run this one: %s" % why)
+ return 1
+ print("cpuref: LLVM on this machine says the result is:")
+ for v in values:
+ print(" %s" % v)
+
+ if args.compare:
+ ours, why = evaluate_ours(args.file, entry["name"])
+ if why:
+ print("cpuref: ours could not be evaluated: %s" % why)
+ return 1
+ print("cpuref: and our frontend, folded over the same bytes, says:")
+ for v in ours:
+ print(" %s" % v)
+ bad = compare(values, ours, args.tolerance)
+ print("cpuref: %s" % ("they agree" if not bad else "they DIFFER: " + bad))
+ return 1 if bad else 0
+ return 0
+
+
+def evaluate_ours(path, function):
+ """air2nir --eval: the same shader folded over the same buffer bytes."""
+ out = subprocess.run([AIR2NIR, "--eval", path, function],
+ capture_output=True, text=True)
+ if out.returncode:
+ return None, (out.stderr or out.stdout).strip().splitlines()[-1][:160]
+ values = []
+ for line in out.stdout.splitlines():
+ if not line.startswith(" ") or line.startswith(" constant") or \
+ line.startswith(" device") or line.startswith(" ("):
+ continue
+ parts = line.split()
+ if len(parts) < 2:
+ continue
+ if "did not fold" in line:
+ return None, "an output did not fold to a constant"
+ values.extend(parts[1:])
+ return values, None
+
+
+def compare(theirs, ours, tol):
+ """What differs between LLVM's numbers and ours, or None."""
+ if len(theirs) != len(ours):
+ return "LLVM printed %d values, we printed %d" % (len(theirs), len(ours))
+ for i, (a, b) in enumerate(zip(theirs, ours)):
+ fa, fb = float(a), float(b)
+ if fa == fb:
+ continue
+ if tol > 0:
+ scale = max(abs(fa), abs(fb))
+ if scale > 0 and abs(fa - fb) / scale <= tol:
+ continue
+ return "value %d: LLVM %s, ours %s" % (i, a, b)
+ return None
+
+
+def run_all(args):
+ """Every function of a .metallib, with a count at the end."""
+ b, header, funcs = metallib.parse(args.file)
+ agree = differ = skipped = 0
+ for f in funcs:
+ name = f.get("name")
+ if not name:
+ continue
+ with tempfile.TemporaryDirectory() as work:
+ bc, _ = metallib.bitcode_of(b, header, f)
+ bc_path = os.path.join(work, "m.bc")
+ with open(bc_path, "wb") as fp:
+ fp.write(bc)
+ try:
+ ll = disassemble(bc_path)
+ entry = entry_of(ll)
+ except subprocess.CalledProcessError:
+ entry = None
+ if not entry:
+ skipped += 1
+ continue
+ theirs, why = build_and_run(ll, entry, buffer_args(ll, entry), work)
+ if why:
+ print("skip %-44s LLVM: %s" % (name, why))
+ skipped += 1
+ continue
+ ours, why = evaluate_ours(args.file, name)
+ if why:
+ print("skip %-44s ours: %s" % (name, why))
+ skipped += 1
+ continue
+ bad = compare(theirs, ours, args.tolerance)
+ if bad:
+ print("DIFFER %-44s %s" % (name, bad))
+ differ += 1
+ else:
+ agree += 1
+ print("agree %-44s %s" % (name, " ".join(theirs[:4])))
+ print("\ncpuref: %d agree with LLVM, %d differ, %d could not be compared (of %d)"
+ % (agree, differ, skipped, len(funcs)))
+ return 1 if differ else 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/tools/mesa/air/air2nir.c b/tools/mesa/air/air2nir.c
index dd5915e..4676172 100644
--- a/tools/mesa/air/air2nir.c
+++ b/tools/mesa/air/air2nir.c
@@ -26,7 +26,10 @@
#include <sys/wait.h>
#include <unistd.h>
+#include <inttypes.h>
+
#include "nir.h"
+#include "nir_builder.h"
#include "nir_air.h"
#include "air_module.h"
#include "util/ralloc.h"
@@ -102,6 +105,174 @@ stage_name(int stage)
}
}
+/* -- evaluating a shader by folding it -------------------------------------
+ *
+ * For the class of shader that only reads buffers and computes -- 256 of the
+ * corpus's 271 fragment shaders -- the whole thing folds to constants once the
+ * buffer loads are replaced by the bytes actually in the buffers. No
+ * interpreter is needed: NIR's own constant folding is the interpreter, and
+ * what comes out can be compared against LLVM's answer for the same bytes
+ * (tools/air/cpuref.py). This is the check docs/air.md section 4 calls the
+ * strongest one, and it needs no GPU.
+ */
+
+/* the same pattern cpuref.py fills its buffers with */
+static uint8_t
+pattern_byte(unsigned i)
+{
+ return (i * 7 + 13) & 0xff;
+}
+
+struct eval_state {
+ struct air_shader_info *info;
+ bool folded_a_load;
+};
+
+static bool
+fold_buffer_loads(struct nir_builder *b, nir_intrinsic_instr *intr, void *data)
+{
+ struct eval_state *st = data;
+ if (intr->intrinsic != nir_intrinsic_load_ubo && intr->intrinsic != nir_intrinsic_load_ssbo)
+ return false;
+
+ unsigned isrc = 0, osrc = 1;
+ if (!nir_src_is_const(intr->src[isrc]) || !nir_src_is_const(intr->src[osrc]))
+ return false;
+ unsigned index = nir_src_as_uint(intr->src[isrc]);
+ unsigned offset = nir_src_as_uint(intr->src[osrc]);
+
+ /* only a buffer the interface actually declared, and only within its size */
+ unsigned size = 0;
+ bool found = false;
+ for (unsigned i = 0; i < st->info->num_resources; i++) {
+ const struct air_resource *r = &st->info->resources[i];
+ bool want = intr->intrinsic == nir_intrinsic_load_ubo
+ ? r->kind == AIR_RESOURCE_CONSTANT_BUFFER
+ : r->kind == AIR_RESOURCE_DEVICE_BUFFER;
+ if (want && r->index == index) {
+ size = r->size ? r->size : 256;
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ return false;
+
+ unsigned bytes = intr->def.num_components * intr->def.bit_size / 8;
+ if (offset + bytes > size)
+ return false;
+
+ nir_const_value v[NIR_MAX_VEC_COMPONENTS] = { 0 };
+ for (unsigned c = 0; c < intr->def.num_components; c++) {
+ uint64_t bits = 0;
+ unsigned nb = intr->def.bit_size / 8;
+ for (unsigned k = 0; k < nb; k++)
+ bits |= (uint64_t)pattern_byte(offset + c * nb + k) << (8 * k);
+ v[c].u64 = bits;
+ }
+
+ b->cursor = nir_before_instr(&intr->instr);
+ nir_def *imm = nir_build_imm(b, intr->def.num_components, intr->def.bit_size, v);
+ nir_def_replace(&intr->def, imm);
+ st->folded_a_load = true;
+ return true;
+}
+
+/* print one stored output value, if it folded to a constant */
+static void
+print_output(nir_variable *var, nir_def *value)
+{
+ printf(" %-24s ", var->name ? var->name : "?");
+ if (!nir_src_is_const(nir_src_for_ssa(value))) {
+ printf("did not fold to a constant\n");
+ return;
+ }
+ const struct glsl_type *t = glsl_without_array(var->type);
+ bool is_float = glsl_get_base_type(t) == GLSL_TYPE_FLOAT ||
+ glsl_get_base_type(t) == GLSL_TYPE_FLOAT16;
+ for (unsigned c = 0; c < value->num_components; c++) {
+ uint64_t bits = nir_src_comp_as_uint(nir_src_for_ssa(value), c);
+ if (is_float && value->bit_size == 32) {
+ float f;
+ uint32_t u = bits;
+ memcpy(&f, &u, 4);
+ printf("%s%.9g", c ? " " : "", (double)f);
+ } else if (is_float && value->bit_size == 16) {
+ printf("%s0x%04" PRIx64, c ? " " : "", bits);
+ } else {
+ printf("%s%" PRIu64, c ? " " : "", bits);
+ }
+ }
+ printf("\n");
+}
+
+static int
+eval_module(const struct module_ref *mod)
+{
+ void *ctx = ralloc_context(NULL);
+ const char *err = NULL;
+ struct air_constant_value *fc;
+ unsigned num_fc = air_module_function_constants(ctx, mod->bitcode, mod->size, &fc);
+ struct air_to_nir_options options = { .constants = fc, .num_constants = num_fc };
+ struct air_shader_info info;
+ nir_shader *nir = air_to_nir(ctx, mod->bitcode, mod->size, NULL, &options,
+ &default_options, &info, &err);
+ if (!nir) {
+ fprintf(stderr, "air2nir: %s: %s\n", mod->name, err);
+ ralloc_free(ctx);
+ return 1;
+ }
+
+ printf("eval: %s (%s), buffers filled with (i * 7 + 13) & 0xff\n", info.entry_point,
+ gl_shader_stage_name(info.stage));
+ for (unsigned i = 0; i < info.num_resources; i++) {
+ const struct air_resource *r = &info.resources[i];
+ if (r->kind == AIR_RESOURCE_CONSTANT_BUFFER || r->kind == AIR_RESOURCE_DEVICE_BUFFER)
+ printf(" %s buffer at index %u, %u bytes\n",
+ r->kind == AIR_RESOURCE_CONSTANT_BUFFER ? "constant" : "device",
+ r->index, r->size);
+ }
+
+ struct eval_state st = { .info = &info };
+ nir_shader_intrinsics_pass(nir, fold_buffer_loads, nir_metadata_control_flow, &st);
+
+ /* fold what the loads made foldable, and nothing else: no algebraic
+ * rewriting, so the arithmetic is IEEE and comparable with LLVM's
+ */
+ bool progress;
+ unsigned rounds = 0;
+ do {
+ progress = false;
+ NIR_PASS(progress, nir, nir_copy_prop);
+ NIR_PASS(progress, nir, nir_opt_constant_folding);
+ NIR_PASS(progress, nir, nir_opt_dce);
+ NIR_PASS(progress, nir, nir_lower_vars_to_ssa);
+ } while (progress && ++rounds < 16);
+
+ printf("eval: the outputs this shader stores:\n");
+ unsigned printed = 0;
+ nir_foreach_function_impl(impl, nir) {
+ nir_foreach_block(block, impl) {
+ nir_foreach_instr(instr, block) {
+ if (instr->type != nir_instr_type_intrinsic)
+ continue;
+ nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
+ if (intr->intrinsic != nir_intrinsic_store_deref)
+ continue;
+ nir_deref_instr *deref = nir_src_as_deref(intr->src[0]);
+ if (!deref || deref->deref_type != nir_deref_type_var)
+ continue;
+ print_output(deref->var, intr->src[1].ssa);
+ printed++;
+ }
+ }
+ }
+ if (!printed)
+ printf(" (none: every store was removed, or none folded)\n");
+ ralloc_free(ctx);
+ return 0;
+}
+
static void
one_line(const char *s, FILE *fp)
{
@@ -170,7 +341,7 @@ report_one(const struct module_ref *mod)
int
main(int argc, char **argv)
{
- enum { TRANSLATE, LL, LIST, REPORT } mode = TRANSLATE;
+ enum { TRANSLATE, LL, LIST, REPORT, EVAL } mode = TRANSLATE;
int i = 1;
if (i < argc && !strcmp(argv[i], "--ll"))
mode = LL, i++;
@@ -178,8 +349,11 @@ main(int argc, char **argv)
mode = LIST, i++;
else if (i < argc && !strcmp(argv[i], "--report"))
mode = REPORT, i++;
+ else if (i < argc && !strcmp(argv[i], "--eval"))
+ mode = EVAL, i++;
if (i >= argc) {
- fprintf(stderr, "usage: air2nir [--ll | --list | --report] FILE [FUNCTION]\n");
+ fprintf(stderr,
+ "usage: air2nir [--ll | --list | --report | --eval] FILE [FUNCTION]\n");
return 2;
}
@@ -221,6 +395,10 @@ main(int argc, char **argv)
for (unsigned k = 0; k < n; k++) {
const char *err = NULL;
+ if (mode == EVAL) {
+ status |= eval_module(&mods[k]);
+ continue;
+ }
if (mode == LIST) {
printf("%-60s %-12s %zu\n", mods[k].name,
air_metallib_type_name(mods[k].container_type), mods[k].size);