Apple's open-source XNU kernel on Android phones: its own loader and drivers, Metal on the Adreno. Runs from RAM; a reboot gives the phone back to Android.
C
air: the integer and gather textures do work, and a claim that said two things
vcvkkc1d9ec04f8a7
diff --git a/docs/air.md b/docs/air.mdindex 4872208..754aa99 100644--- a/docs/air.md+++ b/docs/air.md@@ -33,9 +33,10 @@ And of the 119 distinct `air.*` operations those modules call (`tools/air/intrin| of those, exactly defined and agreeing on every case (45 607 cases) | 71 || of those, approximate, so measured against libm and nothing concluded | 7 || of those, also *run* by a Gallium driver rather than folded (softpipe, 2 586 cases) | 47 |-| the atomics and the barrier, against a kernel written for them (`atomiccheck.py`) | 10 |-| texture operations checked exactly, on several texels each (`texopcheck.py`) | 6 |-| texture operations with no check: integer-typed, multisample, write, depth | 25 |+| the atomics and the barrier, against kernels written for them (`atomiccheck.py`) | 10 |+| texture operations verified exactly, on several texels each (`texopcheck.py`) | 13 |+| texture operations with a case softpipe cannot run: a 16-bit result | 7 |+| texture operations with no case: multisample, depth, write | 11 |313 of 456 is 313 of 321 entry points — 97.5% — because 135 modules are `air.visible` libraryfunctions and not entry points at all.@@ -63,19 +64,26 @@ functions and not entry points at all.however the lowering is written. That needs the phone. Two things are settled without runninganything: every atomic in the corpus asks for `memory_order_relaxed`, and a stronger order isnow refused by name instead of ignored, which is what the lowering used to do.-- **Every one of the 119 declarations the corpus calls now has some numerical check**, which is- new as of tonight and is the first time that has been true: 78 arithmetic ones against C's own- functions, 10 atomics and the barrier against a kernel written for them, and 31 texture ones- partly — 6 declarations exactly, on several texels each (section 4, item 6), and the rest not,- for reasons given there. The derivatives and the simd reductions have no check either, and no- module of the corpus calls them: they are unexercised code, not untested behaviour, and saying- "unverified" of them would overstate what is at stake.-- **The weakest of the three is the textures**, and the gap is not in the fetch but in everything- Metal's sampler state decides. The 64-bit `constexpr sampler` word is still not decoded- (section 3.3), every coordinate checked is a texel centre where nearest filtering has no- weights to round, and the integer-typed reads and samples are not checked at all because- `air_run` binds one float texture and a shader declaring an integer one against it is a type- mismatch whose answer would mean nothing.+- **All 119 declarations are now accounted for**, which is new as of tonight and is a weaker+ claim than "all checked" — the distinction matters and an earlier draft of this paragraph, and+ the commit message of `bbf6c3c`, ran the two together:++ | | of 119 |+ | --- | --- |+ | verified: the answer was compared with one computed elsewhere and agreed | 94 |+ | measured, no verdict: approximate arithmetic, against this machine's libm | 7 |+ | has a case, which softpipe cannot run: a 16-bit result or operand | 7 |+ | has no case, for a reason given in section 4 item 6: multisample, depth, write | 11 |++ The 94 are 71 arithmetic operations, 10 atomics and the barrier, and 13 texture operations. The+ derivatives and the simd reductions are in none of these rows because **no module of the corpus+ calls them**: unexercised code, not untested behaviour, and saying "unverified" of them would+ overstate what is at stake.+- **The weakest of the three groups is the textures**, and the gap is not in the fetch — which is+ now checked at the corners as well as the middle, and for integer and unsigned textures too —+ but in everything Metal's sampler state decides. The 64-bit `constexpr sampler` word is still+ not decoded (section 3.3) and every coordinate checked is a texel centre or a clean footprint+ corner, where nearest filtering has no weights to round and clamping has no arithmetic.- **Nothing here tests Metal's own filtering or addressing.** The six texture shaders that drew onthe Adreno took their sampler from the API, which this harness chose: nearest, clamped, sampledat a texel centre. That is evidence about binding, descriptors and ir3's texture instructions —@@ -618,12 +626,28 @@ Three references, each independent of the translator:entirely and always fetched `(1, 1)` would pass all six of them.`tools/air/texopcheck.py` generates a fragment shader per operation *and coordinate*, asking- for the corners as well as the middle. **All 17 calls return the exact texel**, covering six- declarations: `read_texture_2d` with 32- and with 16-bit coordinates, `get_width_texture_2d`,- `get_height_texture_2d`, `sample_texture_2d` and `sample_texture_3d`. The coordinates are- immediates here, unlike the arithmetic: a texture fetch is not constant-folded by NIR or by- ir3, so there is nothing for the compiler to answer in the shader's place and no buffer is- needed.+ for the corners as well as the middle. **All 35 calls return exactly what was put there**,+ covering 13 of the 31 texture declarations: `read_texture_2d` with 32- and with 16-bit+ coordinates, the two size queries, `sample_texture_2d` and `sample_texture_3d`, each of those+ for float, signed-integer and unsigned-integer textures, and `gather_texture_2d`. The+ coordinates are immediates here, unlike the arithmetic: a texture fetch is not constant-folded+ by NIR or by ir3, so there is nothing for the compiler to answer in the shader's place and no+ buffer is needed to defeat it.++ The integer-typed textures work because `air_run` already picks the texture's format from what+ the shader samples into, with its own comment saying why — "a Metal `texture2d<uint>` read out+ of a float texture gives numbers that are neither right nor obviously wrong". An earlier+ version of this section said those declarations could not be checked for exactly that reason,+ which was written from the probe's output without reading the code beside it. They can, and+ the render target's type has to follow as well or the bits come back read the wrong way.++ A gather returns one channel of the four texels around the coordinate, and Metal fixes the+ order they come back in. They came back in GL's order, `(i0j1, i1j1, i1j0, i0j0)`, but that+ Metal's convention is the same is not established here — so the four values are compared as a+ multiset. That checks the footprint and the channel selection without committing to an order,+ and since every channel gives a different multiset a wrong channel still shows. The coordinate+ is `1.75/4`, which is `1.25` in texel space, so the footprint is `{1,2}×{1,2}` with nothing+ ambiguous about the rounding.Made to fail: swapping the first two coordinate components is caught on all five asymmetriccoordinates and correctly *not* on the three symmetric ones, which is the difference between@@ -634,15 +658,16 @@ Three references, each independent of the translator:- Metal's own sampler state. The sampler is the API's — nearest, clamped — and everycoordinate is a texel centre, where nearest filtering has no weights to round and clampinghas no arithmetic. The 64-bit `constexpr sampler` word remains undecoded (section 3.3).- - The integer-typed reads and samples (`read_texture_2d.s.v4i16`,- `sample_texture_2d.u.v4i32` and the rest, 14 declarations). `air_run` binds one- `r32g32b32a32_float` texture, and a shader declaring an integer texture against it is a type- mismatch whose result would say nothing about the lowering. Binding a texture whose format- follows the declaration is a change to `air_run` that is not made.- - `write_texture_2d`, which needs the texture read back after the draw, and the multisample- reads, which need a multisample texture. Neither is bound.- - The depth sample: nothing binds a depth texture, and a shader reading an attachment nothing- stored to is one of the four tests this port has already had pass for the wrong reason.+ - The seven declarations whose result or coordinate is 16-bit. They have cases and softpipe+ refuses them: TGSI's registers are 32-bit, so `nir_to_tgsi` asserts rather than answering+ wrongly, which is the better of its two failure modes. ir3 has real 16-bit registers, so+ these are a question for the phone.+ - `write_texture_2d` and `write_texture_3d` (4 declarations), which need the texture read back+ after the draw, and the multisample reads (5), which need a multisample texture. Neither is+ bound by `air_run`.+ - The depth sample and the multisample depth read (2): nothing binds a depth texture, and a+ shader reading an attachment nothing stored to is one of the four tests this port has+ already had pass for the wrong reason.Where a shader has a known GLSL equivalent (AppleMetalGLRenderer's shaders *are* the GLpipeline's fixed functions), rendering both through the port's GLES and comparing images is adiff --git a/tools/air/check.sh b/tools/air/check.shindex 076f811..15946eb 100755--- a/tools/air/check.sh+++ b/tools/air/check.sh@@ -108,7 +108,7 @@ echo "== the texture operations, on more than one texel"# lowering that ignored the coordinate and always fetched (1, 1) would pass all# six of them. These ask for the corners too, so swapping x and y is caught on# the five asymmetric coordinates and correctly not on the symmetric ones.-python3 "$ROOT/tools/air/texopcheck.py" 2>&1 | tail -6+python3 "$ROOT/tools/air/texopcheck.py" 2>&1 | tail -10texop_rc=${PIPESTATUS[0]}exit $((reader_rc | cpu_rc | intrin_rc | run_rc | atomic_rc | texop_rc))diff --git a/tools/air/texopcheck.py b/tools/air/texopcheck.pyindex 39093c9..dcc69c3 100755--- a/tools/air/texopcheck.py+++ b/tools/air/texopcheck.py@@ -73,10 +73,10 @@ target triple = "air64-apple-ios16.5.0"%(decl)s-define <{ <4 x float> }> @tex(%(params)s) local_unnamed_addr {+define <{ <4 x %(outty)s> }> @tex(%(params)s) local_unnamed_addr {%(body)s- %%out = insertvalue <{ <4 x float> }> undef, <4 x float> %%v, 0- ret <{ <4 x float> }> %%out+ %%out = insertvalue <{ <4 x %(outty)s> }> undef, <4 x %(outty)s> %%v, 0+ ret <{ <4 x %(outty)s> }> %%out}!llvm.module.flags = !{!0}@@ -88,7 +88,7 @@ define <{ <4 x float> }> @tex(%(params)s) local_unnamed_addr {!2 = !{!"Metal", i32 3, i32 0, i32 0}!3 = !{ptr @tex, !4, !6}!4 = !{!5}-!5 = !{!"air.render_target", i32 0, i32 0, !"air.arg_type_name", !"float4", !"air.arg_name", !"out"}+!5 = !{!"air.render_target", i32 0, i32 0, !"air.arg_type_name", !"%(outname)s", !"air.arg_name", !"out"}!6 = !{%(args)s}%(argmd)s"""@@ -99,15 +99,23 @@ SAMPLER_MD = ('!{i32 %d, !"air.sampler", !"air.location_index", i32 0, i32 1, ''!"air.arg_type_name", !"sampler", !"air.arg_name", !"s"}')-def build(decl, call, needs_sampler, kind, access):- """One shader: `call` computes %v from %t (and %s), `decl` declares it."""+def build(decl, call, needs_sampler, kind, access, outty="float"):+ """One shader: `call` computes %v from %t (and %s), `decl` declares it.++ outty is the render target's component type. It has to follow the+ declaration: air_run picks the texture's format from what the shader+ samples into (its own comment says why -- "a Metal texture2d<uint> read out+ of a float texture gives numbers that are neither right nor obviously+ wrong"), and the render target has to be able to hold the result.+ """params = ["ptr addrspace(1) %t"]md = [TEXTURE_MD % (0, access, kind)]if needs_sampler:params.append("ptr addrspace(2) %s")md.append(SAMPLER_MD % 1)return SKELETON % dict(- decl=decl, params=", ".join(params), body=call,+ decl=decl, params=", ".join(params), body=call, outty=outty,+ outname="float4" if outty == "float" else "int4",args=", ".join("!%d" % (7 + i) for i in range(len(md))),argmd="\n".join("!%d = %s" % (7 + i, m) for i, m in enumerate(md)))@@ -131,7 +139,7 @@ def cases(decls):out.append(("%s (%d,%d)" % (name, x, y),build(have[name], body, False, "2d<float, access::read>","air.read"),- texel(x, y)))+ texel(x, y), False))# the same fetch with 16-bit coordinates, which is a separate declarationname = "air.read_texture_2d.i16.v4f32"@@ -143,7 +151,7 @@ def cases(decls):out.append(("%s (%d,%d)" % (name, x, y),build(have[name], body, False, "2d<float, access::read>","air.read"),- texel(x, y)))+ texel(x, y), False))# -- the size queries ---------------------------------------------------for name, want in (("air.get_width_texture_2d", TEX),@@ -158,7 +166,7 @@ def cases(decls):' %%v = insertelement <4 x float> %%v2, float %%f, i32 3' % name)out.append((name, build(have[name], body, False,"2d<float, access::read>", "air.read"),- [want] * 4))+ [want] * 4, False))# -- sampling, at the centre of several texels --------------------------name = "air.sample_texture_2d.v4f32"@@ -173,7 +181,7 @@ def cases(decls):out.append(("%s (%d,%d)" % (name, x, y),build(have[name], body, True, "2d<float, sample>","air.sample"),- texel(x, y)))+ texel(x, y), False))name = "air.sample_texture_3d.v4f32"if name in have:@@ -187,7 +195,127 @@ def cases(decls):out.append(("%s (%d,%d,%d)" % (name, x, y, z),build(have[name], body, True, "3d<float, sample>","air.sample"),- texel(x, y, z)))+ texel(x, y, z), False))+ # -- the integer-typed reads and samples ---------------------------------+ #+ # air_run picks the texture's format from what the shader samples into, so+ # these get an R32G32B32A32_SINT or _UINT texture holding the same numbers+ # cast to int. The render target has to follow too, or the bits come back+ # read the wrong way.+ for name in sorted(have):+ m = re.match(r"^air\.(read|sample)_texture_(2d|3d)"+ r"(\.i16)?\.([su])\.v4i(16|32)$", name)+ if not m:+ continue+ kindop, nd, narrow_coord, sign, width = m.groups()+ n = 3 if nd == "3d" else 2+ rty = "<4 x i%s>" % width+ decl = have[name]+ for coord in ([(1, 1, 1), (2, 3, 1)] if nd == "3d" else [(1, 1), (2, 3)]):+ if kindop == "read":+ cty = "i16" if narrow_coord else "i32"+ cv = ", ".join("%s %d" % (cty, v) for v in coord[:n])+ body = (' %%r = tail call { %s, i8 } @%s(ptr addrspace(1) %%t, '+ '<%d x %s> <%s>, %s 0, i32 0)\n'+ % (rty, name, n, cty, cv, cty))+ access, kind = "air.read", "%s<%s, access::read>" % (+ nd, "int" if sign == "s" else "uint")+ else:+ cv = ", ".join("float %.9e" % ((v + 0.5) / TEX) for v in coord[:n])+ body = (' %%r = tail call { %s, i8 } @%s(ptr addrspace(1) %%t, '+ 'ptr addrspace(2) %%s, <%d x float> <%s>, i1 true, '+ '<%d x i32> zeroinitializer, i1 false, float 0.0, '+ 'float 0.0, i32 0)\n' % (rty, name, n, cv, n))+ access, kind = "air.sample", "%s<%s, sample>" % (+ nd, "int" if sign == "s" else "uint")+ body += ' %%t0 = extractvalue { %s, i8 } %%r, 0\n' % rty+ if width == "16":+ # widen to the render target, by the declaration's own sign+ body += (' %%v = %s <4 x i16> %%t0 to <4 x i32>'+ % ("sext" if sign == "s" else "zext"))+ else:+ body += ' %v = bitcast <4 x i32> %t0 to <4 x i32>'+ out.append(("%s %s" % (name, coord[:n]),+ build(decl, body, kindop == "sample", kind, access, "i32"),+ texel(*coord[:3] if nd == "3d" else (coord[0], coord[1])),+ False))++ # -- the gather ----------------------------------------------------------+ #+ # A gather returns one channel of the four texels around the coordinate.+ # Metal fixes the order they come back in and this does not know it, so the+ # four values are compared as a multiset: that checks the footprint and the+ # channel selection without committing to an order convention, and every+ # channel gives a different multiset, so a wrong channel still shows.+ #+ # The coordinate is 1.75/4, so in texel space it is 1.25 and the footprint+ # is x,y in {1, 2} with nothing ambiguous about the rounding.+ for name in sorted(have):+ m = re.match(r"^air\.gather_texture_2d(\.[su])?\.v4(f32|f16|i32|i16)$", name)+ if not m:+ continue+ sign, width = m.group(1), m.group(2)+ rty = "<4 x %s>" % {"f32": "float", "f16": "half",+ "i32": "i32", "i16": "i16"}[width]+ outty = "float" if width.startswith("f") else "i32"+ base = {"f32": "float", "f16": "half",+ "i32": "uint" if sign == ".u" else "int",+ "i16": "uint" if sign == ".u" else "int"}[width]+ u = 1.75 / TEX+ for comp in range(4):+ want = sorted(texel(x, y)[comp] for x in (1, 2) for y in (1, 2))+ body = (' %%r = tail call { %s, i8 } @%s(ptr addrspace(1) %%t, '+ 'ptr addrspace(2) %%s, <2 x float> <float %.9e, float %.9e>, '+ 'i1 true, <2 x i32> zeroinitializer, i32 %d, i32 0)\n'+ % (rty, name, u, u, comp))+ body += ' %%g = extractvalue { %s, i8 } %%r, 0\n' % rty+ if width == "f16":+ body += ' %v = fpext <4 x half> %g to <4 x float>'+ elif width == "i16":+ body += (' %%v = %s <4 x i16> %%g to <4 x i32>'+ % ("zext" if sign == ".u" else "sext"))+ elif width == "i32":+ body += ' %v = bitcast <4 x i32> %g to <4 x i32>'+ else:+ body += ' %v = bitcast <4 x float> %g to <4 x float>'+ out.append(("%s channel %d" % (name, comp),+ build(have[name], body, True, "2d<%s, sample>" % base,+ "air.sample", outty),+ want, True))++ # -- the half-typed reads and samples ------------------------------------+ #+ # These belong with the rest rather than in a list of things with no case:+ # they are generated, they are correct as far as this can tell, and+ # softpipe sets them aside for the one reason it sets aside every 16-bit+ # operation. On ir3 they are a real question.+ for name in sorted(have):+ m = re.match(r"^air\.(read|sample)_texture_(2d|3d)\.v4f16$", name)+ if not m:+ continue+ kindop, nd = m.groups()+ n = 3 if nd == "3d" else 2+ coord = (1, 1, 1) if nd == "3d" else (1, 1)+ if kindop == "read":+ cv = ", ".join("i32 %d" % v for v in coord[:n])+ body = (' %%r = tail call { <4 x half>, i8 } @%s(ptr addrspace(1) %%t, '+ '<%d x i32> <%s>, i32 0, i32 0)\n' % (name, n, cv))+ access, kind = "air.read", "%s<half, access::read>" % nd+ sampler = False+ else:+ cv = ", ".join("float %.9e" % ((v + 0.5) / TEX) for v in coord[:n])+ body = (' %%r = tail call { <4 x half>, i8 } @%s(ptr addrspace(1) %%t, '+ 'ptr addrspace(2) %%s, <%d x float> <%s>, i1 true, '+ '<%d x i32> zeroinitializer, i1 false, float 0.0, float 0.0, '+ 'i32 0)\n' % (name, n, cv, n))+ access, kind = "air.sample", "%s<half, sample>" % nd+ sampler = True+ body += ' %g = extractvalue { <4 x half>, i8 } %r, 0\n'+ body += ' %v = fpext <4 x half> %g to <4 x float>'+ out.append(("%s %s" % (name, coord[:n]),+ build(have[name], body, sampler, kind, access),+ texel(*coord[:3]) if nd == "3d" else texel(*coord[:2]), False))+return out@@ -252,28 +380,48 @@ def main(argv):d = keep or tempfile.mkdtemp(prefix="texopcheck.")os.makedirs(d, exist_ok=True)exact = other = refused = 0- screen = None- for i, (label, source, want) in enumerate(todo):+ narrow = set()+ for i, (label, source, want, unordered) in enumerate(todo):got, err = run(d, i, source)if got is None:+ # TGSI's registers are 32-bit, so softpipe cannot carry a 16-bit+ # texture result: nir_to_tgsi asserts. That is softpipe's limit and+ # not the frontend's, and it is the same one intrincheck.py sets+ # aside for the narrow arithmetic. ir3 has real 16-bit registers,+ # so these are a question for the phone.+ if "nir_to_tgsi" in err and re.search(r"v4i16|v4f16", label):+ narrow.add(label.split(" ")[0])+ continueprint("refused %-44s %s" % (label, err[:80]))refused += 1continue- if [float(w) for w in want] == got[:4]:+ g4 = sorted(got[:4]) if unordered else got[:4]+ w4 = sorted(float(w) for w in want) if unordered else [float(w) for w in want]+ if w4 == g4:exact += 1if verbose:print("EXACT %-44s %s" % (label, " ".join("%g" % g for g in got[:4])))else:other += 1- print("WRONG %-44s got %s want %s"- % (label, " ".join("%g" % g for g in got[:4]),- " ".join("%g" % w for w in want)))+ print("WRONG %-44s got %s want %s%s"+ % (label, " ".join("%g" % g for g in g4),+ " ".join("%g" % w for w in w4),+ " (in any order)" if unordered else ""))print()- print("texopcheck: %d texture calls run: %d returned the exact texel, %d did not, "- "%d refused" % (len(todo), exact, other, refused))+ print("texopcheck: %d texture calls run: %d returned exactly what was put there, "+ "%d did not, %d refused" % (len(todo) - sum(1 for l, _, _, _ in todo+ if l.split(" ")[0] in narrow),+ exact, other, refused))+ if narrow:+ print("texopcheck: %d declarations left out: a 16-bit result, and TGSI's registers"+ % len(narrow))+ print("texopcheck: are 32-bit, so softpipe asserts rather than answering wrongly")print("texopcheck: the coordinates differ between calls, so a lowering that ignored")print("texopcheck: them and always fetched (1,1) would fail here and not in texcheck.sh")+ print("texopcheck: a gather is compared as a multiset: it came back in GL's order,")+ print("texopcheck: (i0j1, i1j1, i1j0, i0j0), but Metal's own order is not established")+ print("texopcheck: here, so the footprint and the channel are checked and not the order")print("texopcheck: nothing here tests Metal's sampler state: the sampler is the API's")print("texopcheck: (nearest, clamped) and the 64-bit sampler word is not decoded")if keep: