Skip to content

Apple's open-source XNU kernel on Android phones: its own loader and drivers, Metal on the Adreno. Runs from RAM; a reboot gives the phone back to Android.

C

air: the integer and gather textures do work, and a claim that said two things

vcvkkc1d9ec04f8a7
diff --git a/docs/air.md b/docs/air.md
index 4872208..754aa99 100644
--- a/docs/air.md
+++ b/docs/air.md
@@ -33,9 +33,10 @@ And of the 119 distinct `air.*` operations those modules call (`tools/air/intrin
| of those, exactly defined and agreeing on every case (45 607 cases) | 71 |
| of those, approximate, so measured against libm and nothing concluded | 7 |
| of those, also *run* by a Gallium driver rather than folded (softpipe, 2 586 cases) | 47 |
-| the atomics and the barrier, against a kernel written for them (`atomiccheck.py`) | 10 |
-| texture operations checked exactly, on several texels each (`texopcheck.py`) | 6 |
-| texture operations with no check: integer-typed, multisample, write, depth | 25 |
+| the atomics and the barrier, against kernels written for them (`atomiccheck.py`) | 10 |
+| texture operations verified exactly, on several texels each (`texopcheck.py`) | 13 |
+| texture operations with a case softpipe cannot run: a 16-bit result | 7 |
+| texture operations with no case: multisample, depth, write | 11 |
313 of 456 is 313 of 321 entry points — 97.5% — because 135 modules are `air.visible` library
functions and not entry points at all.
@@ -63,19 +64,26 @@ functions and not entry points at all.
however the lowering is written. That needs the phone. Two things are settled without running
anything: every atomic in the corpus asks for `memory_order_relaxed`, and a stronger order is
now refused by name instead of ignored, which is what the lowering used to do.
-- **Every one of the 119 declarations the corpus calls now has some numerical check**, which is
- new as of tonight and is the first time that has been true: 78 arithmetic ones against C's own
- functions, 10 atomics and the barrier against a kernel written for them, and 31 texture ones
- partly — 6 declarations exactly, on several texels each (section 4, item 6), and the rest not,
- for reasons given there. The derivatives and the simd reductions have no check either, and no
- module of the corpus calls them: they are unexercised code, not untested behaviour, and saying
- "unverified" of them would overstate what is at stake.
-- **The weakest of the three is the textures**, and the gap is not in the fetch but in everything
- Metal's sampler state decides. The 64-bit `constexpr sampler` word is still not decoded
- (section 3.3), every coordinate checked is a texel centre where nearest filtering has no
- weights to round, and the integer-typed reads and samples are not checked at all because
- `air_run` binds one float texture and a shader declaring an integer one against it is a type
- mismatch whose answer would mean nothing.
+- **All 119 declarations are now accounted for**, which is new as of tonight and is a weaker
+ claim than "all checked" — the distinction matters and an earlier draft of this paragraph, and
+ the commit message of `bbf6c3c`, ran the two together:
+
+ | | of 119 |
+ | --- | --- |
+ | verified: the answer was compared with one computed elsewhere and agreed | 94 |
+ | measured, no verdict: approximate arithmetic, against this machine's libm | 7 |
+ | has a case, which softpipe cannot run: a 16-bit result or operand | 7 |
+ | has no case, for a reason given in section 4 item 6: multisample, depth, write | 11 |
+
+ The 94 are 71 arithmetic operations, 10 atomics and the barrier, and 13 texture operations. The
+ derivatives and the simd reductions are in none of these rows because **no module of the corpus
+ calls them**: unexercised code, not untested behaviour, and saying "unverified" of them would
+ overstate what is at stake.
+- **The weakest of the three groups is the textures**, and the gap is not in the fetch — which is
+ now checked at the corners as well as the middle, and for integer and unsigned textures too —
+ but in everything Metal's sampler state decides. The 64-bit `constexpr sampler` word is still
+ not decoded (section 3.3) and every coordinate checked is a texel centre or a clean footprint
+ corner, where nearest filtering has no weights to round and clamping has no arithmetic.
- **Nothing here tests Metal's own filtering or addressing.** The six texture shaders that drew on
the Adreno took their sampler from the API, which this harness chose: nearest, clamped, sampled
at a texel centre. That is evidence about binding, descriptors and ir3's texture instructions —
@@ -618,12 +626,28 @@ Three references, each independent of the translator:
entirely and always fetched `(1, 1)` would pass all six of them.
`tools/air/texopcheck.py` generates a fragment shader per operation *and coordinate*, asking
- for the corners as well as the middle. **All 17 calls return the exact texel**, covering six
- declarations: `read_texture_2d` with 32- and with 16-bit coordinates, `get_width_texture_2d`,
- `get_height_texture_2d`, `sample_texture_2d` and `sample_texture_3d`. The coordinates are
- immediates here, unlike the arithmetic: a texture fetch is not constant-folded by NIR or by
- ir3, so there is nothing for the compiler to answer in the shader's place and no buffer is
- needed.
+ for the corners as well as the middle. **All 35 calls return exactly what was put there**,
+ covering 13 of the 31 texture declarations: `read_texture_2d` with 32- and with 16-bit
+ coordinates, the two size queries, `sample_texture_2d` and `sample_texture_3d`, each of those
+ for float, signed-integer and unsigned-integer textures, and `gather_texture_2d`. The
+ coordinates are immediates here, unlike the arithmetic: a texture fetch is not constant-folded
+ by NIR or by ir3, so there is nothing for the compiler to answer in the shader's place and no
+ buffer is needed to defeat it.
+
+ The integer-typed textures work because `air_run` already picks the texture's format from what
+ the shader samples into, with its own comment saying why — "a Metal `texture2d<uint>` read out
+ of a float texture gives numbers that are neither right nor obviously wrong". An earlier
+ version of this section said those declarations could not be checked for exactly that reason,
+ which was written from the probe's output without reading the code beside it. They can, and
+ the render target's type has to follow as well or the bits come back read the wrong way.
+
+ A gather returns one channel of the four texels around the coordinate, and Metal fixes the
+ order they come back in. They came back in GL's order, `(i0j1, i1j1, i1j0, i0j0)`, but that
+ Metal's convention is the same is not established here — so the four values are compared as a
+ multiset. That checks the footprint and the channel selection without committing to an order,
+ and since every channel gives a different multiset a wrong channel still shows. The coordinate
+ is `1.75/4`, which is `1.25` in texel space, so the footprint is `{1,2}×{1,2}` with nothing
+ ambiguous about the rounding.
Made to fail: swapping the first two coordinate components is caught on all five asymmetric
coordinates and correctly *not* on the three symmetric ones, which is the difference between
@@ -634,15 +658,16 @@ Three references, each independent of the translator:
- Metal's own sampler state. The sampler is the API's — nearest, clamped — and every
coordinate is a texel centre, where nearest filtering has no weights to round and clamping
has no arithmetic. The 64-bit `constexpr sampler` word remains undecoded (section 3.3).
- - The integer-typed reads and samples (`read_texture_2d.s.v4i16`,
- `sample_texture_2d.u.v4i32` and the rest, 14 declarations). `air_run` binds one
- `r32g32b32a32_float` texture, and a shader declaring an integer texture against it is a type
- mismatch whose result would say nothing about the lowering. Binding a texture whose format
- follows the declaration is a change to `air_run` that is not made.
- - `write_texture_2d`, which needs the texture read back after the draw, and the multisample
- reads, which need a multisample texture. Neither is bound.
- - The depth sample: nothing binds a depth texture, and a shader reading an attachment nothing
- stored to is one of the four tests this port has already had pass for the wrong reason.
+ - The seven declarations whose result or coordinate is 16-bit. They have cases and softpipe
+ refuses them: TGSI's registers are 32-bit, so `nir_to_tgsi` asserts rather than answering
+ wrongly, which is the better of its two failure modes. ir3 has real 16-bit registers, so
+ these are a question for the phone.
+ - `write_texture_2d` and `write_texture_3d` (4 declarations), which need the texture read back
+ after the draw, and the multisample reads (5), which need a multisample texture. Neither is
+ bound by `air_run`.
+ - The depth sample and the multisample depth read (2): nothing binds a depth texture, and a
+ shader reading an attachment nothing stored to is one of the four tests this port has
+ already had pass for the wrong reason.
Where a shader has a known GLSL equivalent (AppleMetalGLRenderer's shaders *are* the GL
pipeline's fixed functions), rendering both through the port's GLES and comparing images is a
diff --git a/tools/air/check.sh b/tools/air/check.sh
index 076f811..15946eb 100755
--- a/tools/air/check.sh
+++ b/tools/air/check.sh
@@ -108,7 +108,7 @@ echo "== the texture operations, on more than one texel"
# lowering that ignored the coordinate and always fetched (1, 1) would pass all
# six of them. These ask for the corners too, so swapping x and y is caught on
# the five asymmetric coordinates and correctly not on the symmetric ones.
-python3 "$ROOT/tools/air/texopcheck.py" 2>&1 | tail -6
+python3 "$ROOT/tools/air/texopcheck.py" 2>&1 | tail -10
texop_rc=${PIPESTATUS[0]}
exit $((reader_rc | cpu_rc | intrin_rc | run_rc | atomic_rc | texop_rc))
diff --git a/tools/air/texopcheck.py b/tools/air/texopcheck.py
index 39093c9..dcc69c3 100755
--- a/tools/air/texopcheck.py
+++ b/tools/air/texopcheck.py
@@ -73,10 +73,10 @@ target triple = "air64-apple-ios16.5.0"
%(decl)s
-define <{ <4 x float> }> @tex(%(params)s) local_unnamed_addr {
+define <{ <4 x %(outty)s> }> @tex(%(params)s) local_unnamed_addr {
%(body)s
- %%out = insertvalue <{ <4 x float> }> undef, <4 x float> %%v, 0
- ret <{ <4 x float> }> %%out
+ %%out = insertvalue <{ <4 x %(outty)s> }> undef, <4 x %(outty)s> %%v, 0
+ ret <{ <4 x %(outty)s> }> %%out
}
!llvm.module.flags = !{!0}
@@ -88,7 +88,7 @@ define <{ <4 x float> }> @tex(%(params)s) local_unnamed_addr {
!2 = !{!"Metal", i32 3, i32 0, i32 0}
!3 = !{ptr @tex, !4, !6}
!4 = !{!5}
-!5 = !{!"air.render_target", i32 0, i32 0, !"air.arg_type_name", !"float4", !"air.arg_name", !"out"}
+!5 = !{!"air.render_target", i32 0, i32 0, !"air.arg_type_name", !"%(outname)s", !"air.arg_name", !"out"}
!6 = !{%(args)s}
%(argmd)s
"""
@@ -99,15 +99,23 @@ SAMPLER_MD = ('!{i32 %d, !"air.sampler", !"air.location_index", i32 0, i32 1, '
'!"air.arg_type_name", !"sampler", !"air.arg_name", !"s"}')
-def build(decl, call, needs_sampler, kind, access):
- """One shader: `call` computes %v from %t (and %s), `decl` declares it."""
+def build(decl, call, needs_sampler, kind, access, outty="float"):
+ """One shader: `call` computes %v from %t (and %s), `decl` declares it.
+
+ outty is the render target's component type. It has to follow the
+ declaration: air_run picks the texture's format from what the shader
+ samples into (its own comment says why -- "a Metal texture2d<uint> read out
+ of a float texture gives numbers that are neither right nor obviously
+ wrong"), and the render target has to be able to hold the result.
+ """
params = ["ptr addrspace(1) %t"]
md = [TEXTURE_MD % (0, access, kind)]
if needs_sampler:
params.append("ptr addrspace(2) %s")
md.append(SAMPLER_MD % 1)
return SKELETON % dict(
- decl=decl, params=", ".join(params), body=call,
+ decl=decl, params=", ".join(params), body=call, outty=outty,
+ outname="float4" if outty == "float" else "int4",
args=", ".join("!%d" % (7 + i) for i in range(len(md))),
argmd="\n".join("!%d = %s" % (7 + i, m) for i, m in enumerate(md)))
@@ -131,7 +139,7 @@ def cases(decls):
out.append(("%s (%d,%d)" % (name, x, y),
build(have[name], body, False, "2d<float, access::read>",
"air.read"),
- texel(x, y)))
+ texel(x, y), False))
# the same fetch with 16-bit coordinates, which is a separate declaration
name = "air.read_texture_2d.i16.v4f32"
@@ -143,7 +151,7 @@ def cases(decls):
out.append(("%s (%d,%d)" % (name, x, y),
build(have[name], body, False, "2d<float, access::read>",
"air.read"),
- texel(x, y)))
+ texel(x, y), False))
# -- the size queries ---------------------------------------------------
for name, want in (("air.get_width_texture_2d", TEX),
@@ -158,7 +166,7 @@ def cases(decls):
' %%v = insertelement <4 x float> %%v2, float %%f, i32 3' % name)
out.append((name, build(have[name], body, False,
"2d<float, access::read>", "air.read"),
- [want] * 4))
+ [want] * 4, False))
# -- sampling, at the centre of several texels --------------------------
name = "air.sample_texture_2d.v4f32"
@@ -173,7 +181,7 @@ def cases(decls):
out.append(("%s (%d,%d)" % (name, x, y),
build(have[name], body, True, "2d<float, sample>",
"air.sample"),
- texel(x, y)))
+ texel(x, y), False))
name = "air.sample_texture_3d.v4f32"
if name in have:
@@ -187,7 +195,127 @@ def cases(decls):
out.append(("%s (%d,%d,%d)" % (name, x, y, z),
build(have[name], body, True, "3d<float, sample>",
"air.sample"),
- texel(x, y, z)))
+ texel(x, y, z), False))
+ # -- the integer-typed reads and samples ---------------------------------
+ #
+ # air_run picks the texture's format from what the shader samples into, so
+ # these get an R32G32B32A32_SINT or _UINT texture holding the same numbers
+ # cast to int. The render target has to follow too, or the bits come back
+ # read the wrong way.
+ for name in sorted(have):
+ m = re.match(r"^air\.(read|sample)_texture_(2d|3d)"
+ r"(\.i16)?\.([su])\.v4i(16|32)$", name)
+ if not m:
+ continue
+ kindop, nd, narrow_coord, sign, width = m.groups()
+ n = 3 if nd == "3d" else 2
+ rty = "<4 x i%s>" % width
+ decl = have[name]
+ for coord in ([(1, 1, 1), (2, 3, 1)] if nd == "3d" else [(1, 1), (2, 3)]):
+ if kindop == "read":
+ cty = "i16" if narrow_coord else "i32"
+ cv = ", ".join("%s %d" % (cty, v) for v in coord[:n])
+ body = (' %%r = tail call { %s, i8 } @%s(ptr addrspace(1) %%t, '
+ '<%d x %s> <%s>, %s 0, i32 0)\n'
+ % (rty, name, n, cty, cv, cty))
+ access, kind = "air.read", "%s<%s, access::read>" % (
+ nd, "int" if sign == "s" else "uint")
+ else:
+ cv = ", ".join("float %.9e" % ((v + 0.5) / TEX) for v in coord[:n])
+ body = (' %%r = tail call { %s, i8 } @%s(ptr addrspace(1) %%t, '
+ 'ptr addrspace(2) %%s, <%d x float> <%s>, i1 true, '
+ '<%d x i32> zeroinitializer, i1 false, float 0.0, '
+ 'float 0.0, i32 0)\n' % (rty, name, n, cv, n))
+ access, kind = "air.sample", "%s<%s, sample>" % (
+ nd, "int" if sign == "s" else "uint")
+ body += ' %%t0 = extractvalue { %s, i8 } %%r, 0\n' % rty
+ if width == "16":
+ # widen to the render target, by the declaration's own sign
+ body += (' %%v = %s <4 x i16> %%t0 to <4 x i32>'
+ % ("sext" if sign == "s" else "zext"))
+ else:
+ body += ' %v = bitcast <4 x i32> %t0 to <4 x i32>'
+ out.append(("%s %s" % (name, coord[:n]),
+ build(decl, body, kindop == "sample", kind, access, "i32"),
+ texel(*coord[:3] if nd == "3d" else (coord[0], coord[1])),
+ False))
+
+ # -- the gather ----------------------------------------------------------
+ #
+ # A gather returns one channel of the four texels around the coordinate.
+ # Metal fixes the order they come back in and this does not know it, so the
+ # four values are compared as a multiset: that checks the footprint and the
+ # channel selection without committing to an order convention, and every
+ # channel gives a different multiset, so a wrong channel still shows.
+ #
+ # The coordinate is 1.75/4, so in texel space it is 1.25 and the footprint
+ # is x,y in {1, 2} with nothing ambiguous about the rounding.
+ for name in sorted(have):
+ m = re.match(r"^air\.gather_texture_2d(\.[su])?\.v4(f32|f16|i32|i16)$", name)
+ if not m:
+ continue
+ sign, width = m.group(1), m.group(2)
+ rty = "<4 x %s>" % {"f32": "float", "f16": "half",
+ "i32": "i32", "i16": "i16"}[width]
+ outty = "float" if width.startswith("f") else "i32"
+ base = {"f32": "float", "f16": "half",
+ "i32": "uint" if sign == ".u" else "int",
+ "i16": "uint" if sign == ".u" else "int"}[width]
+ u = 1.75 / TEX
+ for comp in range(4):
+ want = sorted(texel(x, y)[comp] for x in (1, 2) for y in (1, 2))
+ body = (' %%r = tail call { %s, i8 } @%s(ptr addrspace(1) %%t, '
+ 'ptr addrspace(2) %%s, <2 x float> <float %.9e, float %.9e>, '
+ 'i1 true, <2 x i32> zeroinitializer, i32 %d, i32 0)\n'
+ % (rty, name, u, u, comp))
+ body += ' %%g = extractvalue { %s, i8 } %%r, 0\n' % rty
+ if width == "f16":
+ body += ' %v = fpext <4 x half> %g to <4 x float>'
+ elif width == "i16":
+ body += (' %%v = %s <4 x i16> %%g to <4 x i32>'
+ % ("zext" if sign == ".u" else "sext"))
+ elif width == "i32":
+ body += ' %v = bitcast <4 x i32> %g to <4 x i32>'
+ else:
+ body += ' %v = bitcast <4 x float> %g to <4 x float>'
+ out.append(("%s channel %d" % (name, comp),
+ build(have[name], body, True, "2d<%s, sample>" % base,
+ "air.sample", outty),
+ want, True))
+
+ # -- the half-typed reads and samples ------------------------------------
+ #
+ # These belong with the rest rather than in a list of things with no case:
+ # they are generated, they are correct as far as this can tell, and
+ # softpipe sets them aside for the one reason it sets aside every 16-bit
+ # operation. On ir3 they are a real question.
+ for name in sorted(have):
+ m = re.match(r"^air\.(read|sample)_texture_(2d|3d)\.v4f16$", name)
+ if not m:
+ continue
+ kindop, nd = m.groups()
+ n = 3 if nd == "3d" else 2
+ coord = (1, 1, 1) if nd == "3d" else (1, 1)
+ if kindop == "read":
+ cv = ", ".join("i32 %d" % v for v in coord[:n])
+ body = (' %%r = tail call { <4 x half>, i8 } @%s(ptr addrspace(1) %%t, '
+ '<%d x i32> <%s>, i32 0, i32 0)\n' % (name, n, cv))
+ access, kind = "air.read", "%s<half, access::read>" % nd
+ sampler = False
+ else:
+ cv = ", ".join("float %.9e" % ((v + 0.5) / TEX) for v in coord[:n])
+ body = (' %%r = tail call { <4 x half>, i8 } @%s(ptr addrspace(1) %%t, '
+ 'ptr addrspace(2) %%s, <%d x float> <%s>, i1 true, '
+ '<%d x i32> zeroinitializer, i1 false, float 0.0, float 0.0, '
+ 'i32 0)\n' % (name, n, cv, n))
+ access, kind = "air.sample", "%s<half, sample>" % nd
+ sampler = True
+ body += ' %g = extractvalue { <4 x half>, i8 } %r, 0\n'
+ body += ' %v = fpext <4 x half> %g to <4 x float>'
+ out.append(("%s %s" % (name, coord[:n]),
+ build(have[name], body, sampler, kind, access),
+ texel(*coord[:3]) if nd == "3d" else texel(*coord[:2]), False))
+
return out
@@ -252,28 +380,48 @@ def main(argv):
d = keep or tempfile.mkdtemp(prefix="texopcheck.")
os.makedirs(d, exist_ok=True)
exact = other = refused = 0
- screen = None
- for i, (label, source, want) in enumerate(todo):
+ narrow = set()
+ for i, (label, source, want, unordered) in enumerate(todo):
got, err = run(d, i, source)
if got is None:
+ # TGSI's registers are 32-bit, so softpipe cannot carry a 16-bit
+ # texture result: nir_to_tgsi asserts. That is softpipe's limit and
+ # not the frontend's, and it is the same one intrincheck.py sets
+ # aside for the narrow arithmetic. ir3 has real 16-bit registers,
+ # so these are a question for the phone.
+ if "nir_to_tgsi" in err and re.search(r"v4i16|v4f16", label):
+ narrow.add(label.split(" ")[0])
+ continue
print("refused %-44s %s" % (label, err[:80]))
refused += 1
continue
- if [float(w) for w in want] == got[:4]:
+ g4 = sorted(got[:4]) if unordered else got[:4]
+ w4 = sorted(float(w) for w in want) if unordered else [float(w) for w in want]
+ if w4 == g4:
exact += 1
if verbose:
print("EXACT %-44s %s" % (label, " ".join("%g" % g for g in got[:4])))
else:
other += 1
- print("WRONG %-44s got %s want %s"
- % (label, " ".join("%g" % g for g in got[:4]),
- " ".join("%g" % w for w in want)))
+ print("WRONG %-44s got %s want %s%s"
+ % (label, " ".join("%g" % g for g in g4),
+ " ".join("%g" % w for w in w4),
+ " (in any order)" if unordered else ""))
print()
- print("texopcheck: %d texture calls run: %d returned the exact texel, %d did not, "
- "%d refused" % (len(todo), exact, other, refused))
+ print("texopcheck: %d texture calls run: %d returned exactly what was put there, "
+ "%d did not, %d refused" % (len(todo) - sum(1 for l, _, _, _ in todo
+ if l.split(" ")[0] in narrow),
+ exact, other, refused))
+ if narrow:
+ print("texopcheck: %d declarations left out: a 16-bit result, and TGSI's registers"
+ % len(narrow))
+ print("texopcheck: are 32-bit, so softpipe asserts rather than answering wrongly")
print("texopcheck: the coordinates differ between calls, so a lowering that ignored")
print("texopcheck: them and always fetched (1,1) would fail here and not in texcheck.sh")
+ print("texopcheck: a gather is compared as a multiset: it came back in GL's order,")
+ print("texopcheck: (i0j1, i1j1, i1j0, i0j0), but Metal's own order is not established")
+ print("texopcheck: here, so the footprint and the channel are checked and not the order")
print("texopcheck: nothing here tests Metal's sampler state: the sampler is the API's")
print("texopcheck: (nearest, clamped) and the 64-bit sampler word is not decoded")
if keep: