Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
feat(xdna): add native GLM shared execution lane
Kenneth-Javier Ortigozaf8d618c54c26
diff --git a/GPU_BACKENDS.md b/GPU_BACKENDS.mdindex ca7d22a7..c71423e1 100644--- a/GPU_BACKENDS.md+++ b/GPU_BACKENDS.md@@ -431,5 +431,100 @@ but does not require it, and correctness and publication semantics come first; aConversion needs no helper, no artifact, no XRT and no device β it is enginerepresentation logic, and the helper knows nothing about nibble packing or group-scales. **There is still no XDNA execution**: connecting a real operation to the-device is the next slice's work.+scales.+### Native execution: the GLM shared gate/up lane++Colibri can now execute one real operation on the NPU: the **MoE shared-expert+gate and up projections**, and nothing else. This is the whole supported surface,+and the constraints below are hard gates rather than guidance.++```+family MoE shared-expert gate / up (sh_gate, sh_up)+K 6144+N 2048+stored format fmt=4 grouped int4, group size 64+logical M 1..64, zero-padded to the artifact's M=64+execution blocking, one operation at a time+```++`sh_down` is deliberately **not** accelerated: its orientation is `I=2048,+O=6144`, which is not what the qualified artifact computes. Nor is the generic+`matmul_qt` intercepted. The semantic family is passed explicitly by the call+site and never inferred from a shape, so an unrelated operation that happens to+be 6144x2048 cannot inherit this one's qualification.++Row padding is legitimate because `C = A x B` is row-independent: output row *i*+depends only on input row *i* and on B. Only the logical rows are copied out;+padded rows never reach anything downstream. Logical M above 64 declines to the+current path β row tiling is a qualified strategy but is not implemented here.++**There is no automatic selection.** No `--xdna` flag, no `COLI_XDNA`+environment variable and no economic policy exists. An ordinary build with a+helper, a device and valid artifacts present still runs exactly the path it runs+today; the lane is reachable only from an internal test control. Deciding *when*+XDNA is preferable is a separate concern with a separate owner.++Hard eligibility is evaluated in this order, cheapest and most semantic first,+and no later gate can excuse an earlier refusal:++```+family -> logical M -> K/N -> fmt -> group size -> registry row+ -> artifact qualification -> artifact present -> artifact SHA256+ -> helper ABI -> prepared weight VALID -> 4096-byte alignment+ -> device/runtime -> artifact runtime object -> userptr wrap -> execute+```++Any refusal, and any helper failure at any stage, returns "not handled" and the+caller runs its current `matmul_qt` path. The candidate never calls `matmul_qt`+itself, so there is no recursion and no double dispatch, and the caller's output+buffer is written only after the helper reports successful completion β a+failure cannot leave a half-written result behind.++### Building the optional helper++The helper is the only component that links XRT, and it is **opt-in**: an+ordinary build compiles without XRT headers, links without XRT, imports no XRT+DLL, and does not require `coli_xdna.dll` to exist.++Tested provenance:++```+compiler MSVC 14.44.35207 (Visual Studio 2022), Windows SDK 10.0.26100+XRT SDK 2.21.75 -- headers + xrt_coreutil.lib+XRT runtime 2.21.0 -- installed xrt_coreutil.dll+device AMD XDNA2 (Ryzen AI), driver 32.0.20102.3930+```++```+cl /nologo /LD /EHsc /std:c++17 /Zc:__cplusplus /O2 /MD /W3 ^+ /I <XRT_SDK>\include c\backend_xdna_helper.cpp ^+ /Fe:coli_xdna.dll /link <XRT_SDK>\lib\xrt_coreutil.lib+```++`/Zc:__cplusplus` is required: without it MSVC reports `__cplusplus` as+`199711L`, `xrt/detail/any.h` takes its Boost branch, and the build fails on a+header Colibri does not use.++The host looks for `coli_xdna.dll` **beside the executable, by absolute path+only** β no PATH search, no current directory, no application-directory+fallback. A helper that is not there is simply absent, which is a normal state.++The helper ABI is generation **2**. Generation 1 is refused outright rather than+partially bound: it exports two of the seven entry points generation 2 requires+and cannot execute anything, so binding it would produce a helper that reports+availability and then fails. Binding is all-or-nothing.++### Artifacts are not shipped++The qualified `.xclbin` and instruction-stream bytes are **not** distributed with+Colibri, and how they eventually should be is still an open question. The+registry carries their SHA256, and a missing, unreadable or hash-mismatched+artifact fails closed before the device is opened or any byte reaches the+helper. With no artifact root configured β the default, and the only production+value β every request declines.++### What is not claimed++No full-model acceleration, no token-throughput figure, no general XDNA backend,+no routed-expert support, no concurrency with the GPU, and no scheduler. One+family, one shape, one bucket, one dispatch at a time.diff --git a/c/.gitignore b/c/.gitignoreindex fbab5078..5b1b58af 100644--- a/c/.gitignore+++ b/c/.gitignore@@ -46,3 +46,6 @@ tests/test_inkling_serve_framing*.dllcublasLt64*.dllcudart64*.dll+# Optional XDNA test scaffolding that the *.o / *.dll globs above do not cover.+tests/xdna_exec_fixtures/+tests/xdna_physical_probediff --git a/c/Makefile b/c/Makefileindex f17dd1f6..9cb1c58d 100644--- a/c/Makefile+++ b/c/Makefile@@ -1091,6 +1091,31 @@ tests/test_xdna_prepared_state$(EXE): tests/test_xdna_prepared_state.c backend_xtests/test_xdna_registry$(EXE): tests/test_xdna_registry.c backend_xdna.c backend_xdna.h$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)+# Synthetic ABI-2 helpers for the execution tests. Plain C, no XRT, no device:+# they let the whole execution control flow be qualified on a machine with no+# NPU. Three variants so the all-or-nothing binding contract is testable.+XDNA_FAKE_HELPERS = tests/xdna_fake_helper.dll \+ tests/xdna_fake_helper_abi1.dll \+ tests/xdna_fake_helper_partial.dll+tests/xdna_fake_helper.dll: tests/xdna_fake_helper.c+ $(CC) $(CFLAGS) -shared $< -o $@+tests/xdna_fake_helper_abi1.dll: tests/xdna_fake_helper.c+ $(CC) $(CFLAGS) -DFAKE_ABI1 -shared $< -o $@+tests/xdna_fake_helper_partial.dll: tests/xdna_fake_helper.c+ $(CC) $(CFLAGS) -DFAKE_PARTIAL -shared $< -o $@++tests/test_xdna_execution$(EXE): tests/test_xdna_execution.c backend_xdna.c backend_xdna.h compat.h $(XDNA_FAKE_HELPERS)+ $(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)++# NOT a test gate -- intentionally absent from TEST_BINS (its name does not match+# the tests/test_*$(EXE) pattern the gate list is derived from). It needs a real+# XDNA2 NPU, a working XRT, a built coli_xdna.dll and the qualified artifact+# bytes, none of which a build machine may assume. Build and run it explicitly:+# make tests/xdna_physical_probe$(EXE)+# tests/xdna_physical_probe <artifact-root> <path-to-coli_xdna.dll>+tests/xdna_physical_probe$(EXE): tests/xdna_physical_probe.c colibri.c backend_xdna.c backend_xdna.h st.h uring.h json.h tok.h tok_unicode.h compat.h grammar.h tier.h quant.h sample.h kv_persist.h telemetry.h route_trace.h+ $(CC) $(CFLAGS) -DCOLI_XDNA $< backend_xdna.c -o $@ $(LDFLAGS)+tests/test_json$(EXE): tests/test_json.c json.h$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)diff --git a/c/backend_xdna.c b/c/backend_xdna.cindex d472d342..659e20b6 100644--- a/c/backend_xdna.c+++ b/c/backend_xdna.c@@ -32,34 +32,78 @@#include <windows.h>#endif-/* Entry points required of a helper at this ABI generation. Kept minimal on- * purpose: a version handshake, and the release call this file's own lifetime- * contract needs. Later slices add device/artifact/execute entry points and- * bump COLI_XDNA_ABI_VERSION with them. */+/* Entry points required of a helper at ABI generation 2. Still minimal: exactly+ * the calls the first blocking operation needs, and nothing speculative.+ *+ * Everything crossing this boundary is a fixed-width integer, a size_t, a C+ * string, or a raw pointer with an explicit byte count. No engine type, no XRT+ * type and no C++ type appears in any signature, so the helper cannot be handed+ * -- and cannot acquire -- any knowledge of tensors, experts, layers, routing or+ * models. It executes what it is given. */typedef unsigned int (*coli_xdna_fn_abi_version)(void);+/* device + runtime + artifact program, for one (M,K,N). */+typedef int (*coli_xdna_fn_open)(const char *xclbin, const char *insts,+ uint32_t m, uint32_t k, uint32_t n);+/* wrap caller-owned prepared BF16 through the qualified userptr path. */+typedef int (*coli_xdna_fn_wrap_weight)(void *bf16, uint64_t bytes);+/* blocking: submit, wait, copy the artifact-M output out. */+typedef int (*coli_xdna_fn_execute)(const void *a_bf16, uint64_t a_bytes,+ void *c_f32, uint64_t c_bytes);+typedef int (*coli_xdna_fn_release_weight)(void);typedef void (*coli_xdna_fn_shutdown)(void);+typedef const char *(*coli_xdna_fn_last_error)(void);static struct {ColiXdnaBinding state;int load_attempts;- char path[512]; /* test override; empty = default lookup */+ char path[1024]; /* test override; empty = default lookup */#ifdef _WIN32HMODULE dll;#endif- coli_xdna_fn_abi_version abi_version;- coli_xdna_fn_shutdown shutdown;-} g_xdna = { COLI_XDNA_UNPROBED, 0, {0},-#ifdef _WIN32- NULL,-#endif- NULL, NULL };+ coli_xdna_fn_abi_version abi_version;+ coli_xdna_fn_open open;+ coli_xdna_fn_wrap_weight wrap_weight;+ coli_xdna_fn_execute execute;+ coli_xdna_fn_release_weight release_weight;+ coli_xdna_fn_shutdown shutdown;+ coli_xdna_fn_last_error last_error;+} g_xdna;/* Drop every callable pointer. Always runs BEFORE the module is released, so a* rejected or shut-down helper can never leave a pointer into an unmapped* module behind -- the all-or-nothing half of the binding contract. */static void coli_xdna_clear_entry_points(void){- g_xdna.abi_version = NULL;- g_xdna.shutdown = NULL;+ g_xdna.abi_version = NULL;+ g_xdna.open = NULL;+ g_xdna.wrap_weight = NULL;+ g_xdna.execute = NULL;+ g_xdna.release_weight = NULL;+ g_xdna.shutdown = NULL;+ g_xdna.last_error = NULL;+}++/* Every helper call goes through one of these. They are the only places a+ * helper pointer is dereferenced, and each refuses when unbound rather than+ * trusting a caller to have checked. */+static int coli_xdna_helper_open_call(const char *xb, const char *ib,+ unsigned m, unsigned k, unsigned n){+ if(!g_xdna.open) return -1;+ return g_xdna.open(xb, ib, (uint32_t)m, (uint32_t)k, (uint32_t)n);+}+static int coli_xdna_helper_wrap_call(void *p, uint64_t bytes){+ if(!g_xdna.wrap_weight) return -1;+ return g_xdna.wrap_weight(p, bytes);+}+static int coli_xdna_helper_execute_call(const void *a, uint64_t ab, void *c, uint64_t cb){+ if(!g_xdna.execute) return -1;+ return g_xdna.execute(a, ab, c, cb);+}+static int coli_xdna_helper_release_weight_call(void){+ if(!g_xdna.release_weight) return -1;+ return g_xdna.release_weight();+}+static void coli_xdna_helper_shutdown_call(void){+ if(g_xdna.shutdown) g_xdna.shutdown();}const char *coli_xdna_binding_text(ColiXdnaBinding state){@@ -136,8 +180,18 @@ static void coli_xdna_probe(void){#endifcoli_xdna_fn_abi_version abi_version =(coli_xdna_fn_abi_version)GetProcAddress(dll, "coli_xdna_helper_abi_version");+ coli_xdna_fn_open open_fn =+ (coli_xdna_fn_open)GetProcAddress(dll, "coli_xdna_helper_open");+ coli_xdna_fn_wrap_weight wrap_fn =+ (coli_xdna_fn_wrap_weight)GetProcAddress(dll, "coli_xdna_helper_wrap_weight");+ coli_xdna_fn_execute exec_fn =+ (coli_xdna_fn_execute)GetProcAddress(dll, "coli_xdna_helper_execute");+ coli_xdna_fn_release_weight relw_fn =+ (coli_xdna_fn_release_weight)GetProcAddress(dll, "coli_xdna_helper_release_weight");coli_xdna_fn_shutdown shutdown =(coli_xdna_fn_shutdown)GetProcAddress(dll, "coli_xdna_helper_shutdown");+ coli_xdna_fn_last_error lasterr_fn =+ (coli_xdna_fn_last_error)GetProcAddress(dll, "coli_xdna_helper_last_error");#if defined(__GNUC__)#pragma GCC diagnostic pop#endif@@ -152,20 +206,31 @@ static void coli_xdna_probe(void){g_xdna.state = COLI_XDNA_ABI_INCOMPATIBLE;return;}- if(!shutdown){+ /* All-or-nothing: one missing entry point rejects the whole helper. A+ * partially bound helper would report availability and then fail somewhere+ * further in, where the failure is far harder to attribute. */+ if(!open_fn || !wrap_fn || !exec_fn || !relw_fn || !shutdown || !lasterr_fn){FreeLibrary(dll);g_xdna.state = COLI_XDNA_SYMBOL_INCOMPLETE;return;}- g_xdna.dll = dll;- g_xdna.abi_version = abi_version;- g_xdna.shutdown = shutdown;- g_xdna.state = COLI_XDNA_AVAILABLE;+ g_xdna.dll = dll;+ g_xdna.abi_version = abi_version;+ g_xdna.open = open_fn;+ g_xdna.wrap_weight = wrap_fn;+ g_xdna.execute = exec_fn;+ g_xdna.release_weight = relw_fn;+ g_xdna.shutdown = shutdown;+ g_xdna.last_error = lasterr_fn;+ g_xdna.state = COLI_XDNA_AVAILABLE;}void coli_xdna_shutdown(void){- if(g_xdna.shutdown) g_xdna.shutdown();+ /* Release lane state (which itself calls the helper) BEFORE the module goes+ * away: no helper-owned object may outlive the module that created it, and+ * no host pointer into it may survive the FreeLibrary. */+ coli_xdna_execution_shutdown();coli_xdna_clear_entry_points();if(g_xdna.dll){FreeLibrary(g_xdna.dll);@@ -185,6 +250,7 @@ static void coli_xdna_probe(void){}void coli_xdna_shutdown(void){+ coli_xdna_execution_shutdown();coli_xdna_clear_entry_points();}@@ -364,7 +430,13 @@ static const int g_xdna_production_count =(int)(sizeof g_xdna_production_rows / sizeof g_xdna_production_rows[0]);static const ColiXdnaArtifact *g_rows = g_xdna_production_rows;-static int g_nrows = 0;+/* Initialised to the production count, NOT to 0. Zero here would leave the+ * production table installed but empty, so every production lookup would answer+ * UNKNOWN_FAMILY until a test happened to install a registry -- invisible while+ * the only callers were tests that always install one, and wrong the moment a+ * production caller arrived. I5 is that first production caller. */+static int g_nrows =+ (int)(sizeof g_xdna_production_rows / sizeof g_xdna_production_rows[0]);const char *coli_xdna_static_text(ColiXdnaStatic s){switch(s){@@ -464,7 +536,7 @@ static int coli_xdna_join(char *out, size_t cap, const char *root, const char *n* bytes that were qualified". */static ColiXdnaStatic coli_xdna_check_file(const char *root, const char *name,const char *expect_hex){- char path[1024];+ char path[2048];if(!coli_xdna_join(path, sizeof path, root, name))return COLI_XDNA_STATIC_ARTIFACT_UNAVAILABLE;unsigned char got[32];@@ -808,10 +880,345 @@ ColiXdnaPrepResult coli_xdna_prepare_from_fmt4(ColiXdnaPrepared *p,return COLI_XDNA_PREP_OK;}-/* This slice opens no device, calls no helper entry point and wraps no pointer.- * The counters make that assertable rather than merely stated. Conversion is now- * implemented, so its counter is no longer pinned at zero. */-int coli_xdna_test_device_opens(void){ return 0; }-int coli_xdna_test_helper_calls(void){ return 0; }-int coli_xdna_test_userptr_wraps(void){ return 0; }+/* ======================================================================+ * Full hard eligibility and native execution (W2-N7-I5)+ *+ * The engine keeps every decision that requires knowing what the operation+ * MEANS: which family it is, which artifact answers it, whether the bytes are+ * trustworthy, how the weight is represented, whether the request is in range.+ * The helper is handed an already-selected, already-verified artifact and an+ * already-prepared buffer, and executes. It is told nothing about tensors,+ * experts, layers, routing or models, and it decides nothing.+ * ====================================================================== */++const char *coli_xdna_hard_text(ColiXdnaHard h){+ switch(h){+ case COLI_XDNA_HARD_ELIGIBLE: return "HARD_ELIGIBLE";+ case COLI_XDNA_HARD_FAMILY_UNSUPPORTED: return "FAMILY_UNSUPPORTED";+ case COLI_XDNA_HARD_M_OUT_OF_RANGE: return "M_OUT_OF_RANGE";+ case COLI_XDNA_HARD_SHAPE_UNSUPPORTED: return "SHAPE_UNSUPPORTED";+ case COLI_XDNA_HARD_FORMAT_UNSUPPORTED: return "FORMAT_UNSUPPORTED";+ case COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED: return "GROUP_SIZE_UNSUPPORTED";+ case COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED: return "ARTIFACT_NOT_QUALIFIED";+ case COLI_XDNA_HARD_PREPARED_INVALID: return "PREPARED_INVALID";+ case COLI_XDNA_HARD_ALIGNMENT_INVALID: return "ALIGNMENT_INVALID";+ case COLI_XDNA_HARD_HELPER_UNAVAILABLE: return "HELPER_UNAVAILABLE";+ case COLI_XDNA_HARD_DEVICE_UNAVAILABLE: return "DEVICE_UNAVAILABLE";+ case COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE:return "ARTIFACT_RUNTIME_UNAVAILABLE";+ case COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE: return "WEIGHT_WRAP_UNAVAILABLE";+ }+ return "UNKNOWN";+}++const char *coli_xdna_exec_text(ColiXdnaExec e){+ switch(e){+ case COLI_XDNA_EXEC_OK: return "OK";+ case COLI_XDNA_EXEC_DECLINED: return "DECLINED";+ case COLI_XDNA_EXEC_DEVICE_INIT_FAILED: return "DEVICE_INIT_FAILED";+ case COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED: return "ARTIFACT_OPEN_FAILED";+ case COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED: return "WEIGHT_PREPARE_FAILED";+ case COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED: return "WEIGHT_WRAP_FAILED";+ case COLI_XDNA_EXEC_ACTIVATION_FAILED: return "ACTIVATION_FAILED";+ case COLI_XDNA_EXEC_EXECUTE_FAILED: return "EXECUTE_FAILED";+ case COLI_XDNA_EXEC_COMPLETION_FAILED: return "COMPLETION_FAILED";+ }+ return "UNKNOWN";+}++/* ---- execution-lane state ------------------------------------------------+ *+ * Transient per-operation staging (activation, output) and the identity of the+ * artifact currently open in the helper. The prepared WEIGHT is not here: it+ * belongs to the tensor, which outlives any of this. */+static int g_xdna_force; /* internal test control only */+static char g_xdna_root[1024]; /* artifact root; empty = none */+static ColiXdnaHard g_xdna_last_hard = COLI_XDNA_HARD_ELIGIBLE;+static ColiXdnaExec g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;+static int g_xdna_dispatches, g_xdna_completions, g_xdna_artifact_opens;+static int g_xdna_act_preps, g_xdna_padded_ops, g_xdna_fallbacks;+static int g_xdna_device_opens, g_xdna_helper_calls, g_xdna_userptr_wraps;++/* Which artifact the helper currently holds open, so a run of operations on the+ * same shape does not reopen the device, the context or the program. This is a+ * RUNTIME object cache, not an identity cache: it can only ever hold what the+ * engine already selected, and a different selection closes it and reopens. */+static struct {+ int open;+ unsigned m, k, n;+ /* the wrapped weight, identified by the exact pointer the engine handed us */+ const void *wrapped;+ unsigned short *act; size_t act_cap; /* [artifact_m, K] BF16 staging */+ float *out; size_t out_cap; /* [artifact_m, N] F32 staging */+} g_lane;++/* ---- helper entry points (ABI generation 2) ------------------------------+ *+ * Everything crossing this boundary is a fixed-width integer, a size_t, a raw+ * pointer with an explicit byte count, or a C string. No C++ type, no XRT type,+ * no engine type, and no exception: the helper catches everything and converts+ * it to one of these status codes. */+enum {+ COLI_XDNA_H_OK = 0,+ COLI_XDNA_H_E_DEVICE = -1,+ COLI_XDNA_H_E_ARTIFACT = -2,+ COLI_XDNA_H_E_NOT_OPEN = -3,+ COLI_XDNA_H_E_WRAP = -4,+ COLI_XDNA_H_E_SIZE = -5,+ COLI_XDNA_H_E_DISPATCH = -6,+ COLI_XDNA_H_E_COMPLETION = -7,+ COLI_XDNA_H_E_EXCEPTION = -8+};++int coli_xdna_test_dispatches(void){ return g_xdna_dispatches; }+int coli_xdna_test_completions(void){ return g_xdna_completions; }+int coli_xdna_test_artifact_opens(void){ return g_xdna_artifact_opens; }+int coli_xdna_test_activation_preparations(void){ return g_xdna_act_preps; }+int coli_xdna_test_padded_operations(void){ return g_xdna_padded_ops; }+int coli_xdna_test_fallbacks(void){ return g_xdna_fallbacks; }+ColiXdnaHard coli_xdna_test_last_hard(void){ return g_xdna_last_hard; }+ColiXdnaExec coli_xdna_test_last_exec(void){ return g_xdna_last_exec; }++void coli_xdna_test_set_force_execution(int on){ g_xdna_force = on ? 1 : 0; }++void coli_xdna_test_set_artifact_root(const char *root){+ if(root && *root) snprintf(g_xdna_root, sizeof g_xdna_root, "%s", root);+ else g_xdna_root[0] = '\0';+}++int coli_xdna_test_device_opens(void){ return g_xdna_device_opens; }+int coli_xdna_test_helper_calls(void){ return g_xdna_helper_calls; }+int coli_xdna_test_userptr_wraps(void){ return g_xdna_userptr_wraps; }int coli_xdna_test_conversions(void){ return g_xdna_conversions; }++/* ---- the candidate --------------------------------------------------------+ *+ * Gate order is the point of this function, not an accident of it. Everything+ * the engine can answer from its own state is answered first, cheapest and most+ * semantic first, so an operation that could never run costs a handful of+ * integer comparisons and never touches the filesystem, the helper or the+ * device. Nothing later can retroactively excuse an earlier refusal: every gate+ * returns, none of them merely records. */++static ColiXdnaHard coli_xdna_engine_gates(ColiXdnaFamily family,+ int fmt, int I, int O, int gs, int S,+ const ColiXdnaArtifact **row_out){+ /* 1 semantic family -- passed in by the call site, never inferred */+ if(family != COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP)+ return COLI_XDNA_HARD_FAMILY_UNSUPPORTED;++ /* 2 logical M in the range this slice implements */+ if(S < COLI_XDNA_I5_LOGICAL_M_MIN || S > COLI_XDNA_I5_LOGICAL_M_MAX)+ return COLI_XDNA_HARD_M_OUT_OF_RANGE;++ /* 3 K / N representable and positive */+ if(I <= 0 || O <= 0) return COLI_XDNA_HARD_SHAPE_UNSUPPORTED;++ /* 4 stored format */+ if(fmt != 4) return COLI_XDNA_HARD_FORMAT_UNSUPPORTED;++ /* 5 group size -- the converter is more permissive than the device */+ if(gs != COLI_XDNA_QUALIFIED_GROUP_SIZE)+ return COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED;++ /* 6 a qualified artifact for exactly this family/bucket/shape/dtypes.+ * The bucket is the artifact's M, which is where logical M is mapped. */+ ColiXdnaRequest q;+ q.family = family;+ q.m = (unsigned)COLI_XDNA_I5_LOGICAL_M_MAX; /* the F3 M64 bucket */+ q.k = (unsigned)I; q.n = (unsigned)O;+ q.in_dtype = COLI_XDNA_DT_BF16;+ q.weight_dtype = COLI_XDNA_DT_BF16;+ q.out_dtype = COLI_XDNA_DT_F32;++ const ColiXdnaArtifact *row = coli_xdna_registry_lookup(&q);+ if(!row) return COLI_XDNA_HARD_SHAPE_UNSUPPORTED;++ /* 7-9 qualification flags, artifact presence, artifact integrity. All+ * three live behind coli_xdna_artifact_status, which fails closed. */+ if(coli_xdna_artifact_status(&q, g_xdna_root[0] ? g_xdna_root : NULL)+ != COLI_XDNA_STATIC_QUALIFIED)+ return COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED;++ *row_out = row;+ return COLI_XDNA_HARD_ELIGIBLE;+}++/* Grow a staging buffer to at least `need` bytes. Transient operation state:+ * reused across operations so a steady stream of the same shape allocates once,+ * but carrying no validity and no authority of any kind. */+static int coli_xdna_stage(void **base, size_t *cap, size_t need){+ if(*cap >= need) return 1;+ void *p = malloc(need);+ if(!p) return 0;+ free(*base);+ *base = p; *cap = need;+ return 1;+}++int coli_xdna_try_matmul(ColiXdnaFamily family,+ ColiXdnaPrepared **slot,+ int fmt, const unsigned char *q4, const float *scale,+ int I, int O, int gs,+ float *y, const float *x, int S)+{+ /* Gate 0: the internal control. Without it this function is inert, which is+ * what keeps I5 from being an automatic scheduler. */+ if(!g_xdna_force){ g_xdna_fallbacks++; g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED; return 0; }++ if(!slot || !q4 || !scale || !y || !x){+ g_xdna_last_hard = COLI_XDNA_HARD_SHAPE_UNSUPPORTED;+ g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;+ g_xdna_fallbacks++; return 0;+ }++ const ColiXdnaArtifact *row = NULL;+ ColiXdnaHard h = coli_xdna_engine_gates(family, fmt, I, O, gs, S, &row);+ if(h != COLI_XDNA_HARD_ELIGIBLE){+ g_xdna_last_hard = h; g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;+ g_xdna_fallbacks++; return 0;+ }++ /* 10 helper ABI. Checked before the weight is prepared: preparing 24 MiB+ * for a lane that cannot execute would be pure waste. */+ if(coli_xdna_binding() != COLI_XDNA_AVAILABLE){+ g_xdna_last_hard = COLI_XDNA_HARD_HELPER_UNAVAILABLE;+ g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;+ g_xdna_fallbacks++; return 0;+ }++ /* 11 prepared weight. Lazy: prepared on first use of this tensor and+ * reused while it stays VALID for the same dimensions. There is no+ * eviction or cache policy here -- that is a later slice's decision. */+ if(!*slot){+ *slot = coli_xdna_prepared_create();+ if(!*slot){+ g_xdna_last_hard = COLI_XDNA_HARD_PREPARED_INVALID;+ g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED;+ g_xdna_fallbacks++; return 0;+ }+ }+ ColiXdnaPrepared *prep = *slot;+ if(coli_xdna_prepared_state(prep) != COLI_XDNA_PREP_VALID+ || coli_xdna_prepared_k(prep) != (unsigned)I+ || coli_xdna_prepared_n(prep) != (unsigned)O){+ if(coli_xdna_prepare_from_fmt4(prep, fmt, q4, scale, I, O, gs) != COLI_XDNA_PREP_OK){+ g_xdna_last_hard = COLI_XDNA_HARD_PREPARED_INVALID;+ g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED;+ g_xdna_fallbacks++; return 0;+ }+ }+ const void *wimg = coli_xdna_prepared_image(prep); /* NULL unless VALID */+ if(!wimg){+ g_xdna_last_hard = COLI_XDNA_HARD_PREPARED_INVALID;+ g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED;+ g_xdna_fallbacks++; return 0;+ }++ /* 12 alignment. The allocator guarantees it; checking anyway is cheap and+ * turns a confusing XRT "insufficient free video memory" into a local,+ * accurate refusal. */+ if(!coli_xdna_pointer_alignment_ok(wimg)){+ g_xdna_last_hard = COLI_XDNA_HARD_ALIGNMENT_INVALID;+ g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;+ g_xdna_fallbacks++; return 0;+ }++ const unsigned am = row->artifact_m;++ /* 13 device / runtime / artifact runtime object. Opened lazily, only now+ * that every engine-side gate has passed. */+ if(!g_lane.open || g_lane.m != am || g_lane.k != (unsigned)I || g_lane.n != (unsigned)O){+ char xb[2048], ib[2048];+ if(!coli_xdna_join(xb, sizeof xb, g_xdna_root, row->xclbin_name)+ || !coli_xdna_join(ib, sizeof ib, g_xdna_root, row->insts_name)){+ g_xdna_last_hard = COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE;+ g_xdna_last_exec = COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED;+ g_xdna_fallbacks++; return 0;+ }+ if(g_lane.open){ coli_xdna_helper_release_weight_call(); coli_xdna_helper_shutdown_call(); }+ g_xdna_helper_calls++; g_xdna_device_opens++;+ int rc = coli_xdna_helper_open_call(xb, ib, am, (unsigned)I, (unsigned)O);+ if(rc != COLI_XDNA_H_OK){+ g_xdna_last_hard = (rc == COLI_XDNA_H_E_DEVICE)+ ? COLI_XDNA_HARD_DEVICE_UNAVAILABLE+ : COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE;+ g_xdna_last_exec = (rc == COLI_XDNA_H_E_DEVICE)+ ? COLI_XDNA_EXEC_DEVICE_INIT_FAILED+ : COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED;+ g_xdna_fallbacks++; return 0;+ }+ g_lane.open = 1; g_lane.m = am; g_lane.k = (unsigned)I; g_lane.n = (unsigned)O;+ g_lane.wrapped = NULL;+ g_xdna_artifact_opens++;+ }++ /* 14 userptr wrap of the engine-owned prepared image. The helper wraps+ * THIS memory; it never allocates a second copy of the weight. */+ if(g_lane.wrapped != wimg){+ g_xdna_helper_calls++; g_xdna_userptr_wraps++;+ if(coli_xdna_helper_wrap_call((void *)wimg,+ (uint64_t)coli_xdna_prepared_bytes(prep)) != COLI_XDNA_H_OK){+ g_xdna_last_hard = COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE;+ g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED;+ g_xdna_fallbacks++; return 0;+ }+ g_lane.wrapped = wimg;+ }++ g_xdna_last_hard = COLI_XDNA_HARD_ELIGIBLE; /* FULL_XDNA_HARD_ELIGIBLE */++ /* ---- activation staging -------------------------------------------------+ * Caller activation is f32 [S, K] row-major. The artifact consumes BF16+ * [artifact_m, K]. The conversion uses the SAME round-to-nearest-even rule+ * as the weight, because both were qualified on that rule together.+ *+ * Rows S..artifact_m-1 are zeroed. Zero is the mathematically neutral row+ * for a pure GEMM: it contributes a zero output row and cannot perturb any+ * logical row, because C = A x B is row-independent. */+ size_t abytes = (size_t)am * (size_t)I * 2u;+ size_t obytes = (size_t)am * (size_t)O * 4u;+ if(!coli_xdna_stage((void **)&g_lane.act, &g_lane.act_cap, abytes)+ || !coli_xdna_stage((void **)&g_lane.out, &g_lane.out_cap, obytes)){+ g_xdna_last_exec = COLI_XDNA_EXEC_ACTIVATION_FAILED;+ g_xdna_fallbacks++; return 0;+ }+ for(size_t i = 0; i < (size_t)S * (size_t)I; i++)+ g_lane.act[i] = coli_xdna_f2b(x[i]);+ if((unsigned)S < am){+ memset(g_lane.act + (size_t)S * (size_t)I, 0,+ ((size_t)am - (size_t)S) * (size_t)I * 2u);+ g_xdna_padded_ops++;+ }+ g_xdna_act_preps++;++ /* ---- blocking execution -------------------------------------------------+ * One operation, one dispatch, one wait. No worker thread, no queue, no+ * second outstanding operation. */+ g_xdna_helper_calls++;+ int rc = coli_xdna_helper_execute_call(g_lane.act, (uint64_t)abytes,+ g_lane.out, (uint64_t)obytes);+ if(rc != COLI_XDNA_H_OK){+ g_xdna_last_exec = (rc == COLI_XDNA_H_E_COMPLETION)+ ? COLI_XDNA_EXEC_COMPLETION_FAILED+ : COLI_XDNA_EXEC_EXECUTE_FAILED;+ g_xdna_fallbacks++;+ return 0; /* y untouched: the caller's path owns it */+ }+ g_xdna_dispatches++; g_xdna_completions++;++ /* Only the logical rows become output. The padded rows are never copied and+ * so can never reach anything downstream, whatever the device wrote there. */+ memcpy(y, g_lane.out, (size_t)S * (size_t)O * 4u);+ g_xdna_last_exec = COLI_XDNA_EXEC_OK;+ return 1;+}++void coli_xdna_execution_shutdown(void){+ if(g_lane.open){+ coli_xdna_helper_release_weight_call();+ coli_xdna_helper_shutdown_call();+ }+ g_lane.open = 0; g_lane.wrapped = NULL;+ free(g_lane.act); g_lane.act = NULL; g_lane.act_cap = 0;+ free(g_lane.out); g_lane.out = NULL; g_lane.out_cap = 0;+}diff --git a/c/backend_xdna.h b/c/backend_xdna.hindex c0c87352..10b320e7 100644--- a/c/backend_xdna.h+++ b/c/backend_xdna.h@@ -29,8 +29,15 @@ extern "C" {/* Host-side ABI generation. A helper reporting a different value is refused* outright: there is no partial acceptance and no speculative forward* compatibility, because neither could be tested here. Bump this when the- * required entry-point set or any signature changes. */-#define COLI_XDNA_ABI_VERSION 1u+ * required entry-point set or any signature changes.+ *+ * Generation 2 (W2-N7-I5) is a DELIBERATE COMPATIBILITY BREAK. Generation 1+ * required two entry points and could only answer "a helper exists"; generation+ * 2 requires seven and can open a device, wrap a weight and execute. A+ * generation-1 helper is refused outright rather than partially used: it cannot+ * perform any of the new work, and silently binding it would produce a helper+ * that reports availability and then cannot execute. */+#define COLI_XDNA_ABI_VERSION 2u/* Symbols the helper must export, all prefixed coli_xdna_helper_ so they can* never collide with the host-side coli_xdna_ names in this header. */@@ -320,9 +327,138 @@ int coli_xdna_pointer_alignment_ok(const void *p);size_t coli_xdna_prepared_total_bytes(void);int coli_xdna_prepared_live_objects(void);+/* -- full hard eligibility and native execution ---------------------------+ *+ * I2 could only answer STATIC_ARTIFACT_QUALIFIED: does a trustworthy artifact+ * exist? This is the first slice that can answer the whole question:+ *+ * CAN this operation safely run on XDNA, right now, on this machine?+ *+ * It still does NOT answer "should it". Hard eligibility and economic+ * preference are separate decisions with separate owners, and nothing here+ * consults, computes or caches a cost. Automatic selection belongs to a later+ * slice; this one executes only when an internal test control asks it to. */++typedef enum {+ COLI_XDNA_HARD_ELIGIBLE = 0,+ /* engine-side semantic gates, evaluated before anything is opened */+ COLI_XDNA_HARD_FAMILY_UNSUPPORTED,+ COLI_XDNA_HARD_M_OUT_OF_RANGE,+ COLI_XDNA_HARD_SHAPE_UNSUPPORTED,+ COLI_XDNA_HARD_FORMAT_UNSUPPORTED, /* stored tensor is not fmt=4 */+ COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED, /* fmt=4 but gs is not the qualified 64 */+ /* artifact gates */+ COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED, /* any non-QUALIFIED static verdict */+ /* representation gates */+ COLI_XDNA_HARD_PREPARED_INVALID,+ COLI_XDNA_HARD_ALIGNMENT_INVALID,+ /* runtime gates */+ COLI_XDNA_HARD_HELPER_UNAVAILABLE,+ COLI_XDNA_HARD_DEVICE_UNAVAILABLE,+ COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE,+ COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE+} ColiXdnaHard;++const char *coli_xdna_hard_text(ColiXdnaHard h);++/* The qualified source group size. I4's converter accepts any gs>0 as a generic+ * representation transform; that breadth is a property of the CONVERTER, not a+ * statement about what the device was qualified on. A PREPARED_VALID image+ * built from another gs is a perfectly good host image and is still refused+ * here, because no artifact was ever correctness-qualified against one. */+#define COLI_XDNA_QUALIFIED_GROUP_SIZE 64++/* The logical-M range this slice implements, against the F3 M64 artifact.+ *+ * Row padding is legitimate because C = A x B is row-independent: output row i+ * depends only on input row i and on B. N6-A2-A1A qualified the strategy+ * physically -- 13 logical M values against an unmodified fixed-M artifact, all+ * bit-exact, with zeros written into every padded row. That argument is a+ * property of PURE GEMM and must never be extended to attention, normalisation+ * or routing.+ *+ * Row TILING (logical M above the bucket) is equally qualified as a strategy+ * and deliberately NOT implemented here: the smallest first seam is one+ * dispatch. Anything above the range declines to the current path. */+#define COLI_XDNA_I5_LOGICAL_M_MIN 1+#define COLI_XDNA_I5_LOGICAL_M_MAX 64++typedef enum {+ COLI_XDNA_EXEC_OK = 0,+ COLI_XDNA_EXEC_DECLINED, /* not hard-eligible; not an error */+ COLI_XDNA_EXEC_DEVICE_INIT_FAILED,+ COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED,+ COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED,+ COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED,+ COLI_XDNA_EXEC_ACTIVATION_FAILED,+ COLI_XDNA_EXEC_EXECUTE_FAILED, /* dispatch refused or threw */+ COLI_XDNA_EXEC_COMPLETION_FAILED /* dispatched, did not complete cleanly */+} ColiXdnaExec;++const char *coli_xdna_exec_text(ColiXdnaExec e);++/* The production candidate, called from the GLM shared gate/up sites and+ * nowhere else. Deliberately shaped like the existing optional-lane idiom+ * (vk_matmul_qt): returns 1 when it has fully written y, 0 when the caller must+ * run its current path.+ *+ * if(!coli_xdna_try_matmul(...)) matmul_qt(y, x, w, S);+ *+ * It NEVER calls matmul_qt itself, so there is no recursion, no double dispatch+ * and no path on which a failure here suppresses the current path.+ *+ * `family` is passed explicitly by the call site. It is never inferred from the+ * shape: two operations with identical M/K/N are different operations, and one+ * may not inherit the other's qualification.+ *+ * `slot` is the engine-owned prepared-state handle (QT::xdna), created lazily on+ * first use. The tensor's authoritative fmt=4 bytes are read and never modified.+ *+ * y is written ONLY after the helper reports successful completion, so a failure+ * at any stage leaves the caller's output buffer untouched for matmul_qt to+ * overwrite. */+int coli_xdna_try_matmul(ColiXdnaFamily family,+ ColiXdnaPrepared **slot,+ int fmt, const unsigned char *q4, const float *scale,+ int I, int O, int gs,+ float *y, const float *x, int S);++/* Release helper-owned runtime state and the transient staging buffers. Safe+ * when nothing was ever initialised, and safe to repeat. */+void coli_xdna_execution_shutdown(void);+/* Install a registry for tests; NULL restores the production table. */void coli_xdna_test_set_registry(const ColiXdnaArtifact *rows, int count);+/* -- internal execution controls ------------------------------------------+ *+ * These are the ONLY way to reach the native path, and they are deliberately+ * test seams rather than user-facing controls: there is no --xdna flag, no+ * COLI_XDNA environment variable and no automatic policy anywhere in this+ * slice, so an ordinary build with a helper, a device and valid artifacts+ * present still runs exactly the path it runs today.+ *+ * Forcing bypasses ECONOMIC preference only -- which does not exist yet, so+ * today it bypasses nothing at all. It cannot bypass any hard gate: family,+ * format, group size, shape, bucket, artifact integrity, qualification,+ * prepared validity, alignment, helper ABI or device availability. */+void coli_xdna_test_set_force_execution(int on);+/* Where logical artifact names resolve. NULL (the default, and the only+ * production value) means no artifact is reachable and every request declines.+ * Artifact distribution is an open question this slice does not close. */+void coli_xdna_test_set_artifact_root(const char *root);++/* The last hard-eligibility verdict, for tests and diagnostics. */+ColiXdnaHard coli_xdna_test_last_hard(void);+/* The last execution verdict. */+ColiXdnaExec coli_xdna_test_last_exec(void);+int coli_xdna_test_dispatches(void); /* successful helper executions */+int coli_xdna_test_completions(void); /* of those, cleanly completed */+int coli_xdna_test_artifact_opens(void);+int coli_xdna_test_activation_preparations(void);+int coli_xdna_test_padded_operations(void);+int coli_xdna_test_fallbacks(void); /* candidate returned 0 */+/* Deterministic mid-conversion fault injection, for tests only. 0 disables it;* 1..99 fails after approximately that percentage of the conversion's rows.* Named coli_xdna_test_* like every other seam here, and never referenced by@@ -333,7 +469,9 @@ void coli_xdna_test_set_convert_fail_pct(int pct);* a failed conversion left behind. Production code uses* coli_xdna_prepared_image(), which returns NULL unless the image is VALID. */const void *coli_xdna_prepared_image_unchecked(const ColiXdnaPrepared *p);-/* All four must stay 0 for the whole of this slice. */+/* Device opens, helper calls and userptr wraps became reachable in W2-N7-I5 and+ * are 0 only on paths that never reach the native lane. Conversions became+ * reachable in I4. */int coli_xdna_test_device_opens(void);int coli_xdna_test_helper_calls(void);int coli_xdna_test_userptr_wraps(void);diff --git a/c/backend_xdna_helper.cpp b/c/backend_xdna_helper.cppnew file mode 100644index 00000000..fb65b1e3--- /dev/null+++ b/c/backend_xdna_helper.cpp@@ -0,0 +1,249 @@+// Optional AMD XDNA2 helper -- the ONLY component in Colibri that links XRT.+//+// This file is NOT part of an ordinary Colibri build. It compiles to+// coli_xdna.dll, which c/backend_xdna.c loads at runtime and may refuse. A+// machine with no NPU, no XRT and no helper is a normal machine; the engine+// simply keeps doing what it does today.+//+// WHAT THIS FILE OWNS+// XRT runtime objects, the device, the hardware context, the kernel, the+// artifact program, the device-visible activation and output buffers, the+// userptr wrapper around the caller's prepared weight, the blocking submit+// and wait, and the release of all of the above.+//+// WHAT THIS FILE MUST NEVER OWN+// Anything that requires knowing what the operation MEANS. It is not told the+// semantic family, the tensor, the expert, the layer, the model, the routing+// decision, the quantisation format or the group size, and it makes no+// selection, no fallback decision and no economic judgement. It is handed an+// already-selected, already-integrity-verified artifact and an+// already-prepared buffer, and it executes.+//+// Every export catches std::exception AND unknown exceptions and converts them+// to a status code. No exception may cross the C ABI.+//+// Build: see GPU_BACKENDS.md. Requires the XRT SDK headers and+// xrt_coreutil.lib; MSVC needs /Zc:__cplusplus for xrt/detail/any.h to select+// <any> instead of its boost fallback.++#include <cstdint>+#include <cstring>+#include <cstdio>+#include <string>+#include <vector>+#include <fstream>++#include "xrt/xrt_device.h"+#include "xrt/xrt_kernel.h"+#include "xrt/xrt_bo.h"+#include "xrt/experimental/xrt_xclbin.h"+#include "xrt/experimental/xrt_ext.h"++#define COLI_XDNA_HELPER_API extern "C" __declspec(dllexport)++// Must equal COLI_XDNA_ABI_VERSION in c/backend_xdna.h. Exact equality: the+// host refuses any other generation outright.+#define COLI_XDNA_HELPER_ABI 2u++// Must match the enum in backend_xdna.c. Kept as plain ints so the boundary+// carries no type of its own.+enum {+ H_OK = 0,+ H_E_DEVICE = -1,+ H_E_ARTIFACT = -2,+ H_E_NOT_OPEN = -3,+ H_E_WRAP = -4,+ H_E_SIZE = -5,+ H_E_DISPATCH = -6,+ H_E_COMPLETION = -7,+ H_E_EXCEPTION = -8+};++namespace {++struct Lane {+ bool open = false;+ xrt::device *dev = nullptr;+ xrt::xclbin *xcl = nullptr;+ xrt::hw_context *ctx = nullptr;+ xrt::kernel *ker = nullptr;+ xrt::bo *bo_i = nullptr; // instruction stream+ xrt::bo *bo_a = nullptr; // activation, device-visible+ xrt::bo *bo_c = nullptr; // output, device-visible+ xrt::ext::bo *bo_w = nullptr; // userptr wrapper over CALLER memory+ std::vector<uint32_t> instr;+ size_t m = 0, k = 0, n = 0;+};++Lane g;+char g_err[512] = {0};++void set_err(const char *s) { std::snprintf(g_err, sizeof g_err, "%s", s ? s : ""); }++std::vector<uint32_t> load_instr(const char *p) {+ std::ifstream f(p, std::ios::binary | std::ios::ate);+ if (!f) return {};+ std::streamsize n = f.tellg();+ if (n <= 0 || (n % 4) != 0) return {};+ f.seekg(0);+ std::vector<uint32_t> v(static_cast<size_t>(n) / 4);+ f.read(reinterpret_cast<char *>(v.data()), n);+ if (!f) return {};+ return v;+}++// Release everything, in reverse construction order, exactly once each. Safe+// when nothing was ever constructed, and safe to repeat: every pointer is+// nulled as it is deleted.+void teardown() {+ delete g.bo_w; g.bo_w = nullptr; // wrapper first: it borrows caller memory+ delete g.bo_c; g.bo_c = nullptr;+ delete g.bo_a; g.bo_a = nullptr;+ delete g.bo_i; g.bo_i = nullptr;+ delete g.ker; g.ker = nullptr;+ delete g.ctx; g.ctx = nullptr;+ delete g.xcl; g.xcl = nullptr;+ delete g.dev; g.dev = nullptr;+ g.instr.clear();+ g.m = g.k = g.n = 0;+ g.open = false;+}++} // namespace++COLI_XDNA_HELPER_API unsigned int coli_xdna_helper_abi_version(void) {+ return COLI_XDNA_HELPER_ABI;+}++COLI_XDNA_HELPER_API const char *coli_xdna_helper_last_error(void) {+ return g_err;+}++// Open the device and the already-selected artifact for one (m, k, n).+//+// The host has already verified that these bytes hash to the values the+// research programme qualified; this function does not re-decide which artifact+// to use, and could not, since it is never told what the operation is.+COLI_XDNA_HELPER_API int coli_xdna_helper_open(const char *xclbin, const char *insts,+ uint32_t m, uint32_t k, uint32_t n) {+ if (!xclbin || !insts || m == 0 || k == 0 || n == 0) { set_err("bad open arguments"); return H_E_SIZE; }+ try {+ teardown(); // a re-open never layers onto old state++ g.instr = load_instr(insts);+ if (g.instr.empty()) { set_err("instruction stream missing or malformed"); return H_E_ARTIFACT; }++ g.dev = new xrt::device(static_cast<unsigned int>(0));+ g.xcl = new xrt::xclbin(std::string(xclbin));+ g.dev->register_xclbin(*g.xcl);+ g.ctx = new xrt::hw_context(*g.dev, g.xcl->get_uuid());+ g.ker = new xrt::kernel(*g.ctx, "MLIR_AIE");++ g.m = m; g.k = k; g.n = n;+ g.bo_i = new xrt::bo(*g.dev, g.instr.size() * 4, XCL_BO_FLAGS_CACHEABLE, g.ker->group_id(1));+ g.bo_a = new xrt::bo(*g.dev, static_cast<size_t>(m) * k * 2, XRT_BO_FLAGS_HOST_ONLY, g.ker->group_id(3));+ g.bo_c = new xrt::bo(*g.dev, static_cast<size_t>(m) * n * 4, XRT_BO_FLAGS_HOST_ONLY, g.ker->group_id(5));++ std::memcpy(g.bo_i->map<void *>(), g.instr.data(), g.instr.size() * 4);+ g.bo_i->sync(XCL_BO_SYNC_BO_TO_DEVICE);++ g.open = true;+ set_err("");+ return H_OK;+ } catch (const std::exception &e) {+ set_err(e.what()); teardown();+ // A device that will not open and an artifact that will not load are+ // different failures for the host: one disables the lane, the other+ // only this shape. XRT reports both as exceptions, so this cannot be+ // distinguished perfectly here; the host treats an open failure as+ // artifact-scoped, which is the narrower and therefore safer claim.+ return H_E_ARTIFACT;+ } catch (...) {+ set_err("unknown exception in open"); teardown();+ return H_E_EXCEPTION;+ }+}++// Wrap the caller's prepared BF16 weight image through the qualified XRT+// userptr path. No copy and no allocation of a second weight image: the device+// reads the engine's own buffer.+//+// The pointer must be 4096-byte aligned. The host guarantees and re-checks that+// before calling; a misaligned pointer fails here with an XRT message about+// video memory, which points at entirely the wrong subsystem.+COLI_XDNA_HELPER_API int coli_xdna_helper_wrap_weight(void *bf16, uint64_t bytes) {+ if (!g.open) { set_err("wrap before open"); return H_E_NOT_OPEN; }+ if (!bf16 || bytes == 0) { set_err("bad wrap arguments"); return H_E_SIZE; }+ if (bytes != static_cast<uint64_t>(g.k) * g.n * 2) { set_err("weight size does not match K*N*2"); return H_E_SIZE; }+ try {+ delete g.bo_w; g.bo_w = nullptr;+ g.bo_w = new xrt::ext::bo(*g.dev, bf16, static_cast<size_t>(bytes));+ g.bo_w->sync(XCL_BO_SYNC_BO_TO_DEVICE);+ set_err("");+ return H_OK;+ } catch (const std::exception &e) {+ set_err(e.what()); delete g.bo_w; g.bo_w = nullptr; return H_E_WRAP;+ } catch (...) {+ set_err("unknown exception in wrap"); delete g.bo_w; g.bo_w = nullptr; return H_E_EXCEPTION;+ }+}++// One blocking operation: stage the activation, submit, wait, read the output+// back. No worker thread, no queue, and never more than one operation+// outstanding -- the serialized V1 posture the concurrency research froze.+//+// c_f32 receives all artifact-M rows. Deciding which of them are logical output+// is the host's job, and the host copies only those.+COLI_XDNA_HELPER_API int coli_xdna_helper_execute(const void *a_bf16, uint64_t a_bytes,+ void *c_f32, uint64_t c_bytes) {+ if (!g.open || !g.bo_w) { set_err("execute before open/wrap"); return H_E_NOT_OPEN; }+ if (!a_bf16 || !c_f32) { set_err("null execute buffer"); return H_E_SIZE; }+ if (a_bytes != static_cast<uint64_t>(g.m) * g.k * 2) { set_err("activation size mismatch"); return H_E_SIZE; }+ if (c_bytes != static_cast<uint64_t>(g.m) * g.n * 4) { set_err("output size mismatch"); return H_E_SIZE; }+ try {+ std::memcpy(g.bo_a->map<void *>(), a_bf16, static_cast<size_t>(a_bytes));+ g.bo_a->sync(XCL_BO_SYNC_BO_TO_DEVICE);++ auto r = xrt::run(*g.ker);+ r.set_arg(0, static_cast<unsigned int>(3));+ r.set_arg(1, *g.bo_i);+ r.set_arg(2, g.instr.size());+ r.set_arg(3, *g.bo_a);+ // MUST bind through xrt::bo&. Binding an xrt::ext::bo by its own type+ // selects the scalar set_arg overload, which the runtime accepts and+ // then reports ERT_CMD_STATE_COMPLETED while every output is NaN.+ xrt::bo &wref = *g.bo_w;+ r.set_arg(4, wref);+ r.set_arg(5, *g.bo_c);++ r.start();+ auto st = r.wait();+ if (st != ERT_CMD_STATE_COMPLETED) {+ set_err("command did not complete");+ return H_E_COMPLETION;+ }+ g.bo_c->sync(XCL_BO_SYNC_BO_FROM_DEVICE);+ std::memcpy(c_f32, g.bo_c->map<const void *>(), static_cast<size_t>(c_bytes));+ set_err("");+ return H_OK;+ } catch (const std::exception &e) {+ set_err(e.what()); return H_E_DISPATCH;+ } catch (...) {+ set_err("unknown exception in execute"); return H_E_EXCEPTION;+ }+}++// Drop the userptr wrapper while keeping the artifact runtime open. The caller+// memory it borrowed is untouched: the engine owns it and always did.+COLI_XDNA_HELPER_API int coli_xdna_helper_release_weight(void) {+ try {+ delete g.bo_w; g.bo_w = nullptr;+ return H_OK;+ } catch (...) {+ g.bo_w = nullptr; set_err("unknown exception in release_weight"); return H_E_EXCEPTION;+ }+}++COLI_XDNA_HELPER_API void coli_xdna_helper_shutdown(void) {+ try { teardown(); } catch (...) { /* nothing useful remains to report */ }+}diff --git a/c/colibri.c b/c/colibri.cindex 4aad3b23..454b4c8c 100644--- a/c/colibri.c+++ b/c/colibri.c@@ -6243,10 +6243,29 @@ static void moe(Model *m, Layer *l, int layer, float *x, int S, float *out, intif(!shared_cuda){#endifsg=falloc((int64_t)S*sI);su=falloc((int64_t)S*sI);+ /* Optional XDNA2 lane, gate/up only. Same idiom as vk_matmul_qt: 0 means+ * "not handled, run the current path", and the candidate never calls+ * matmul_qt itself, so there is no recursion and no double dispatch.+ * The semantic family is passed EXPLICITLY -- it is never inferred from+ * M/K/N, so an unrelated operation of the same shape cannot inherit this+ * one's qualification. sh_down is deliberately absent below: its+ * orientation (I=sI, O=D) is not what the qualified F3 artifact+ * computes. Inert without the internal test control; there is no public+ * switch and no automatic policy in this slice. */+#ifdef COLI_XDNA+ if(!coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &l->sh_gate.xdna,+ l->sh_gate.fmt, l->sh_gate.q4, l->sh_gate.s,+ l->sh_gate.I, l->sh_gate.O, l->sh_gate.gs, sg, x, S))+#endif#ifdef COLI_VULKANif(!vk_matmul_qt(&l->sh_gate, sg, x, S))#endifmatmul_qt(sg, x, &l->sh_gate, S);+#ifdef COLI_XDNA+ if(!coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &l->sh_up.xdna,+ l->sh_up.fmt, l->sh_up.q4, l->sh_up.s,+ l->sh_up.I, l->sh_up.O, l->sh_up.gs, su, x, S))+#endif#ifdef COLI_VULKANif(!vk_matmul_qt(&l->sh_up, su, x, S))#endifdiff --git a/c/tests/test_backend_loader.py b/c/tests/test_backend_loader.pyindex 3a5cbab6..b19ddacf 100644--- a/c/tests/test_backend_loader.py+++ b/c/tests/test_backend_loader.py@@ -2437,13 +2437,31 @@ _XDNA_FAKE_HELPER_C = r'''#include <windows.h>#ifndef FAKE_ABI_VERSION-#define FAKE_ABI_VERSION 1u+#define FAKE_ABI_VERSION 2u#endif__declspec(dllexport) unsigned int coli_xdna_helper_abi_version(void){return (unsigned int)FAKE_ABI_VERSION;}+/* ABI generation 2 requires the whole execution set, not just a handshake. None+ * of these does anything: the contract under test is the BINDING. */+__declspec(dllexport) int coli_xdna_helper_open(const char *x, const char *i,+ unsigned m, unsigned k, unsigned n){+ (void)x;(void)i;(void)m;(void)k;(void)n; return 0;+}+__declspec(dllexport) int coli_xdna_helper_wrap_weight(void *p, unsigned long long b){+ (void)p;(void)b; return 0;+}+#ifndef FAKE_OMIT_EXECUTE+__declspec(dllexport) int coli_xdna_helper_execute(const void *a, unsigned long long ab,+ void *c, unsigned long long cb){+ (void)a;(void)ab;(void)c;(void)cb; return 0;+}+#endif+__declspec(dllexport) int coli_xdna_helper_release_weight(void){ return 0; }+__declspec(dllexport) const char *coli_xdna_helper_last_error(void){ return ""; }+#ifndef FAKE_OMIT_SHUTDOWN__declspec(dllexport) void coli_xdna_helper_shutdown(void){ }#endif@@ -2479,8 +2497,8 @@ _XDNA_HARNESS_C = r'''static void report(const char *tag){ColiXdnaBinding s = coli_xdna_binding();- printf("%s=%s attempts=%d\n", tag, coli_xdna_binding_text(s),- coli_xdna_test_load_attempts());+ printf("%s=%s attempts=%d bound=%d\n", tag, coli_xdna_binding_text(s),+ coli_xdna_test_load_attempts(), coli_xdna_test_entry_points_bound());}int main(int argc, char **argv){@@ -2600,12 +2618,28 @@ class XdnaOptionalBindingTest(unittest.TestCase):gcc(["-O0", "-shared", "-DFAKE_ABI_VERSION=999u", str(helper_src),"-o", str(cls.bad_abi)], "building the wrong-ABI synthetic helper")+ # The previous ABI generation. W2-N7-I5 bumped the host to generation 2,+ # which is a deliberate compatibility break: a generation-1 helper+ # exports two of the seven entry points now required and cannot execute+ # anything, so it must be refused outright rather than partially bound.+ cls.abi1 = root / "abi1" / _XDNA_HELPER_BASENAME+ cls.abi1.parent.mkdir(parents=True, exist_ok=True)+ gcc(["-O0", "-shared", "-DFAKE_ABI_VERSION=1u", str(helper_src),+ "-o", str(cls.abi1)], "building the previous-generation synthetic helper")+# Missing a required entry point.cls.incomplete = root / "incomplete" / _XDNA_HELPER_BASENAMEcls.incomplete.parent.mkdir(parents=True, exist_ok=True)gcc(["-O0", "-shared", "-DFAKE_OMIT_SHUTDOWN", str(helper_src),"-o", str(cls.incomplete)], "building the incomplete synthetic helper")+ # Missing one of the entry points ABI 2 introduced. Binding is+ # all-or-nothing across the WHOLE set, not just the parts it inherited.+ cls.no_execute = root / "no_execute" / _XDNA_HELPER_BASENAME+ cls.no_execute.parent.mkdir(parents=True, exist_ok=True)+ gcc(["-O0", "-shared", "-DFAKE_OMIT_EXECUTE", str(helper_src),+ "-o", str(cls.no_execute)], "building the execute-less synthetic helper")+# Present but unloadable: imports a dependency that is then deleted.cls.unloadable = root / "unloadable" / _XDNA_HELPER_BASENAMEcls.unloadable.parent.mkdir(parents=True, exist_ok=True)@@ -2662,6 +2696,21 @@ class XdnaOptionalBindingTest(unittest.TestCase):out = self._run("probe", self.bad_abi)self.assertEqual(self._state(out), "ABI_INCOMPATIBLE")+ def test_previous_abi_generation_is_rejected(self):+ # The I1 -> I5 generation bump is a compatibility break by design. A+ # stale generation-1 helper must never become callable under the+ # generation-2 contract, however many symbol names it happens to share.+ out = self._run("probe", self.abi1)+ self.assertEqual(self._state(out), "ABI_INCOMPATIBLE")+ self.assertIn("bound=0", out)++ def test_missing_execute_entry_point_is_rejected(self):+ # All-or-nothing applies to the entry points ABI 2 added, not only to+ # the ones it inherited from ABI 1.+ out = self._run("probe", self.no_execute)+ self.assertEqual(self._state(out), "SYMBOL_INCOMPLETE")+ self.assertIn("bound=0", out)+def test_missing_required_entry_point_is_rejected(self):out = self._run("probe", self.incomplete)self.assertEqual(self._state(out), "SYMBOL_INCOMPLETE")@@ -2692,7 +2741,7 @@ class XdnaOptionalBindingTest(unittest.TestCase):out = self._run("repeat", path, 8)# attempts is reported on the same line as state=line = [l for l in out.splitlines() if l.startswith("state=")][0]- attempts = int(line.split("attempts=")[1])+ attempts = int(line.split("attempts=")[1].split(" ")[0])self.assertLessEqual(attempts, 1,"loader re-probed %d times: %s" % (attempts, out))diff --git a/c/tests/test_xdna_execution.c b/c/tests/test_xdna_execution.cnew file mode 100644index 00000000..7c1585f6--- /dev/null+++ b/c/tests/test_xdna_execution.c@@ -0,0 +1,507 @@+/* W2-N7-I5 -- XDNA execution seam, qualified WITHOUT hardware.+ *+ * This is the owner of the execution control flow: gate order, artifact open,+ * userptr wrap, activation staging and padding, logical-row extraction, failure+ * classification, and the guarantee that every non-success path falls back to+ * the caller's current operation with its output buffer untouched.+ *+ * It runs against a synthetic ABI-2 helper (tests/xdna_fake_helper.c) that+ * contains no XRT and computes the GEMM on the CPU, so a machine with no NPU+ * qualifies all of it. The NUMERICS of real device execution are qualified+ * separately and explicitly on hardware; nothing here stands in for that.+ */++#include <stdio.h>+#include <stdlib.h>+#include <string.h>+#include <math.h>++#include "../backend_xdna.c"++static int g_fail = 0;+static void ck(int cond, const char *what){+ if(!cond){ printf(" FAIL %s\n", what); g_fail = 1; }+ else printf(" ok %s\n", what);+}++/* ---- fixtures ----------------------------------------------------------- */++#define TK 256 /* K -- small so the CPU GEMM in the fake stays quick */+#define TN 128 /* N */+#define TM 64 /* artifact M, the bucket this slice implements */++static char g_root[1024];+static char g_helper[1024], g_helper_abi1[1024], g_helper_partial[1024];+static char g_xclbin[2048], g_insts[2048];+static char g_xhex[65], g_ihex[65];+static ColiXdnaArtifact g_test_rows[1];++static void hexify(const unsigned char h[32], char out[65]){+ static const char *H = "0123456789abcdef";+ for(int i = 0; i < 32; i++){ out[i*2] = H[h[i]>>4]; out[i*2+1] = H[h[i]&15]; }+ out[64] = '\0';+}++static int write_blob(const char *path, unsigned seed, size_t n){+ FILE *f = fopen(path, "wb");+ if(!f) return 0;+ unsigned st = seed;+ for(size_t i = 0; i < n; i++){ st = st*1664525u+1013904223u; fputc((int)(st>>24) & 0xFF, f); }+ fclose(f);+ return 1;+}++/* Build a registry whose hashes are the REAL hashes of the fixture files, so the+ * integrity gate is exercised for what it is rather than stubbed out. */+static void build_registry(void){+ unsigned char h[32];+ if(!coli_xdna_sha256_file(g_xclbin, h)){ printf(" FAIL fixture xclbin unreadable\n"); g_fail=1; return; }+ hexify(h, g_xhex);+ if(!coli_xdna_sha256_file(g_insts, h)){ printf(" FAIL fixture insts unreadable\n"); g_fail=1; return; }+ hexify(h, g_ihex);++ g_test_rows[0].family = COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP;+ g_test_rows[0].research_family = "F3";+ g_test_rows[0].artifact_m = TM;+ g_test_rows[0].k = TK; g_test_rows[0].n = TN;+ g_test_rows[0].in_dtype = COLI_XDNA_DT_BF16;+ g_test_rows[0].weight_dtype = COLI_XDNA_DT_BF16;+ g_test_rows[0].out_dtype = COLI_XDNA_DT_F32;+ g_test_rows[0].target = COLI_XDNA_TARGET_XDNA2;+ g_test_rows[0].xclbin_name = "fake.xclbin";+ g_test_rows[0].xclbin_sha256 = g_xhex;+ g_test_rows[0].insts_name = "fake_insts.bin";+ g_test_rows[0].insts_sha256 = g_ihex;+ g_test_rows[0].runtime_weight_qualified = 1;+ g_test_rows[0].correctness_qualified = 1;+ g_test_rows[0].userptr_qualified = 1;+ g_test_rows[0].structural_qualified = 1;+ coli_xdna_test_set_registry(g_test_rows, 1);+}++/* A deterministic fmt4 tensor with I=K, O=N. */+typedef struct { unsigned char *q4; float *s; int I, O, gs; } Fmt4;++static void fmt4_make(Fmt4 *t, int I, int O, int gs, unsigned seed){+ int rb = (I+1)/2, ng = (I+gs-1)/gs;+ t->I = I; t->O = O; t->gs = gs;+ t->q4 = (unsigned char *)malloc((size_t)O*rb);+ t->s = (float *)malloc((size_t)O*ng*sizeof(float));+ unsigned st = seed;+ for(size_t i = 0; i < (size_t)O*rb; i++){ st = st*1664525u+1013904223u; t->q4[i] = (unsigned char)(st>>24); }+ for(size_t i = 0; i < (size_t)O*ng; i++){ st = st*1664525u+1013904223u;+ t->s[i] = 0.25f * (float)(1 + ((st>>26)&3)); }+}+static void fmt4_free(Fmt4 *t){ free(t->q4); free(t->s); t->q4=NULL; t->s=NULL; }++static float *mk_x(int S, int K, unsigned seed){+ float *x = (float *)malloc((size_t)S*K*sizeof(float));+ unsigned st = seed;+ for(size_t i = 0; i < (size_t)S*K; i++){ st = st*1664525u+1013904223u;+ x[i] = (float)((int)((st>>20)&255) - 128) * 0.0078125f; }+ return x;+}++static float b2f_t(unsigned short b){+ unsigned int u = (unsigned int)b << 16; float f; memcpy(&f,&u,4); return f;+}++/* Reference for the LAYOUT contract: BF16 activation x BF16 prepared weight,+ * float accumulation in k order -- exactly what the fake computes. This checks+ * that the right rows went in, the right rows came out, and nothing from the+ * padded region leaked; it is not a claim about device arithmetic. */+static void oracle(float *out, const float *x, const unsigned short *W,+ int S, int K, int N){+ for(int i = 0; i < S; i++)+ for(int j = 0; j < N; j++){+ float acc = 0.0f;+ for(int k = 0; k < K; k++)+ acc += b2f_t(coli_xdna_f2b(x[(size_t)i*K+k])) * b2f_t(W[(size_t)k*N+j]);+ out[(size_t)i*N+j] = acc;+ }+}++#define POISON (-123456.0f)+static void poison(float *y, size_t n){ for(size_t i=0;i<n;i++) y[i]=POISON; }+static int all_poison(const float *y, size_t n){+ for(size_t i=0;i<n;i++) if(y[i]!=POISON) return 0; return 1;+}++/* Reset every lane-visible counter and state between groups. */+static void lane_reset(void){+ coli_xdna_execution_shutdown();+ coli_xdna_test_set_force_execution(0);+ g_xdna_dispatches = g_xdna_completions = g_xdna_artifact_opens = 0;+ g_xdna_act_preps = g_xdna_padded_ops = g_xdna_fallbacks = 0;+ g_xdna_device_opens = g_xdna_helper_calls = g_xdna_userptr_wraps = 0;+}++/* Rebind the loader to a helper path and clear its sticky verdict. */+static void use_helper(const char *path){+ lane_reset();+ coli_xdna_test_set_helper_path(path);+}++int main(int argc, char **argv){+ /* Locate fixtures beside this executable's directory. */+ char dir[512]; snprintf(dir, sizeof dir, "%s", argc>0?argv[0]:"tests/x");+ char *sl = strrchr(dir,'\\'); char *fw = strrchr(dir,'/');+ if(fw && (!sl || fw>sl)) sl = fw;+ if(sl) *sl = '\0'; else snprintf(dir,sizeof dir,".");++ snprintf(g_root, sizeof g_root, "%s/xdna_exec_fixtures", dir);+ snprintf(g_helper, sizeof g_helper, "%s/xdna_fake_helper.dll", dir);+ snprintf(g_helper_abi1, sizeof g_helper_abi1, "%s/xdna_fake_helper_abi1.dll", dir);+ snprintf(g_helper_partial, sizeof g_helper_partial, "%s/xdna_fake_helper_partial.dll", dir);+#ifdef _WIN32+ { char cmd[1200]; snprintf(cmd,sizeof cmd,"mkdir \"%s\" 2>nul", g_root);+ for(char *p=cmd;*p;p++) if(*p=='/') *p='\\';+ if(system(cmd)){} }+#else+ { char cmd[1200]; snprintf(cmd,sizeof cmd,"mkdir -p \"%s\"", g_root); if(system(cmd)){} }+#endif+ snprintf(g_xclbin, sizeof g_xclbin, "%s/fake.xclbin", g_root);+ snprintf(g_insts, sizeof g_insts, "%s/fake_insts.bin", g_root);+ if(!write_blob(g_xclbin, 11u, 4096) || !write_blob(g_insts, 22u, 1024)){+ printf("FAIL could not create fixtures under %s\n", g_root); return 1; }+ build_registry();+ if(g_fail) return 1;++ Fmt4 W; fmt4_make(&W, TK, TN, 64, 7u);+ Fmt4 W2; fmt4_make(&W2, TK, TN, 64, 999u); /* a genuinely different weight */++ /* ================================================================== */+ printf("default behaviour: the seam is inert without the internal control\n");+ {+ use_helper(g_helper);+ coli_xdna_test_set_artifact_root(g_root);+ ColiXdnaPrepared *slot = NULL;+ float *x = mk_x(TM, TK, 1u);+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &slot,+ 4, W.q4, W.s, TK, TN, 64, y, x, TM);+ ck(handled == 0, "helper, artifact and device all available -- still not handled");+ ck(coli_xdna_test_dispatches() == 0, "and zero dispatches");+ ck(all_poison(y,(size_t)TM*TN), "caller output untouched");+ coli_xdna_prepared_release(&slot); free(x); free(y);+ }++ /* ================================================================== */+ printf("forced execution: the full path end to end\n");+ {+ use_helper(g_helper);+ coli_xdna_test_set_artifact_root(g_root);+ coli_xdna_test_set_force_execution(1);+ ColiXdnaPrepared *slot = NULL;+ float *x = mk_x(TM, TK, 2u);+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &slot,+ 4, W.q4, W.s, TK, TN, 64, y, x, TM);+ ck(handled == 1, "handled");+ ck(coli_xdna_test_last_hard() == COLI_XDNA_HARD_ELIGIBLE, "FULL_XDNA_HARD_ELIGIBLE");+ ck(coli_xdna_test_last_exec() == COLI_XDNA_EXEC_OK, "exec OK");+ ck(coli_xdna_test_dispatches() == 1, "one dispatch");+ ck(coli_xdna_test_completions() == 1, "one completion");+ ck(coli_xdna_test_userptr_wraps() == 1, "one userptr wrap");+ ck(coli_xdna_test_artifact_opens() == 1, "one artifact open");+ ck(coli_xdna_prepared_state(slot) == COLI_XDNA_PREP_VALID, "weight left VALID");+ ck(coli_xdna_pointer_alignment_ok(coli_xdna_prepared_image(slot)),+ "prepared image 4096-aligned");++ const unsigned short *Wi = (const unsigned short *)coli_xdna_prepared_image(slot);+ float *ref = (float *)malloc((size_t)TM*TN*4);+ oracle(ref, x, Wi, TM, TK, TN);+ size_t mism = 0;+ for(size_t i = 0; i < (size_t)TM*TN; i++) if(y[i] != ref[i]) mism++;+ ck(mism == 0, "every output element matches the layout reference");+ free(ref); coli_xdna_prepared_release(&slot); free(x); free(y);+ }++ /* ================================================================== */+ printf("logical-M padding inside the qualified range\n");+ {+ int cases[] = { 1, 2, 7, 31, 32, 33, 63, 64 };+ for(size_t c = 0; c < sizeof cases/sizeof cases[0]; c++){+ int S = cases[c];+ use_helper(g_helper);+ coli_xdna_test_set_artifact_root(g_root);+ coli_xdna_test_set_force_execution(1);+ ColiXdnaPrepared *slot = NULL;+ float *x = mk_x(S, TK, 100u + (unsigned)S);+ float *y = (float *)malloc((size_t)S*TN*4); poison(y,(size_t)S*TN);+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &slot,+ 4, W.q4, W.s, TK, TN, 64, y, x, S);+ char msg[128];+ snprintf(msg,sizeof msg,"S=%d handled",S); ck(handled==1,msg);+ snprintf(msg,sizeof msg,"S=%d padded_ops=%d",S,coli_xdna_test_padded_operations());+ ck(coli_xdna_test_padded_operations() == (S<TM?1:0), msg);++ const unsigned short *Wi = (const unsigned short *)coli_xdna_prepared_image(slot);+ float *ref = (float *)malloc((size_t)S*TN*4);+ oracle(ref, x, Wi, S, TK, TN);+ size_t mism = 0;+ for(size_t i = 0; i < (size_t)S*TN; i++) if(y[i] != ref[i]) mism++;+ snprintf(msg,sizeof msg,"S=%d all %d logical rows exact, no padded row exposed",S,S);+ ck(mism == 0, msg);+ free(ref); coli_xdna_prepared_release(&slot); free(x); free(y);+ }+ }++ /* ================================================================== */+ printf("negative controls -- none of these may ever dispatch\n");+ {+ struct { const char *what; ColiXdnaFamily fam; int fmt, I, O, gs, S; ColiXdnaHard want; } cases[] = {+ { "wrong semantic family (same shape)", COLI_XDNA_FAMILY_NONE, 4, TK, TN, 64, TM,+ COLI_XDNA_HARD_FAMILY_UNSUPPORTED },+ { "sh_down orientation (K/N transposed)", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TN, TK, 64, TM,+ COLI_XDNA_HARD_SHAPE_UNSUPPORTED },+ { "gs != 64 (preparable but not qualified)", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TK, TN, 32, TM,+ COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED },+ { "fmt != 4", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 2, TK, TN, 64, TM,+ COLI_XDNA_HARD_FORMAT_UNSUPPORTED },+ { "logical M above the qualified range", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TK, TN, 64, TM+1,+ COLI_XDNA_HARD_M_OUT_OF_RANGE },+ { "logical M below the qualified range", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TK, TN, 64, 0,+ COLI_XDNA_HARD_M_OUT_OF_RANGE },+ { "unqualified shape", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TK+8, TN, 64, TM,+ COLI_XDNA_HARD_SHAPE_UNSUPPORTED }+ };+ for(size_t c = 0; c < sizeof cases/sizeof cases[0]; c++){+ use_helper(g_helper);+ coli_xdna_test_set_artifact_root(g_root);+ coli_xdna_test_set_force_execution(1);+ Fmt4 T; fmt4_make(&T, cases[c].I, cases[c].O, cases[c].gs>0?cases[c].gs:64, 5u);+ ColiXdnaPrepared *slot = NULL;+ int S = cases[c].S > 0 ? cases[c].S : 1;+ float *x = mk_x(S, cases[c].I, 3u);+ float *y = (float *)malloc((size_t)S*cases[c].O*4); poison(y,(size_t)S*cases[c].O);+ int handled = coli_xdna_try_matmul(cases[c].fam, &slot, cases[c].fmt,+ T.q4, T.s, cases[c].I, cases[c].O,+ cases[c].gs, y, x, cases[c].S);+ char msg[160];+ snprintf(msg,sizeof msg,"%s -> not handled", cases[c].what); ck(handled==0,msg);+ snprintf(msg,sizeof msg,"%s -> dispatches 0", cases[c].what);+ ck(coli_xdna_test_dispatches()==0,msg);+ snprintf(msg,sizeof msg,"%s -> %s", cases[c].what,+ coli_xdna_hard_text(coli_xdna_test_last_hard()));+ ck(coli_xdna_test_last_hard()==cases[c].want,msg);+ snprintf(msg,sizeof msg,"%s -> output untouched", cases[c].what);+ ck(all_poison(y,(size_t)S*cases[c].O),msg);+ snprintf(msg,sizeof msg,"%s -> helper never called", cases[c].what);+ ck(coli_xdna_test_helper_calls()==0,msg);+ coli_xdna_prepared_release(&slot); free(x); free(y); fmt4_free(&T);+ }+ }++ /* ================================================================== */+ printf("gs!=64 is refused even though the converter can prepare it\n");+ {+ use_helper(g_helper);+ coli_xdna_test_set_artifact_root(g_root);+ Fmt4 T; fmt4_make(&T, TK, TN, 32, 8u);+ ColiXdnaPrepared *p = coli_xdna_prepared_create();+ ck(coli_xdna_prepare_from_fmt4(p,4,T.q4,T.s,TK,TN,32) == COLI_XDNA_PREP_OK,+ "gs=32 prepares successfully -- a valid derived host image");+ ck(coli_xdna_prepared_state(p) == COLI_XDNA_PREP_VALID, "and it is PREPARED_VALID");+ coli_xdna_prepared_release(&p);++ coli_xdna_test_set_force_execution(1);+ ColiXdnaPrepared *slot = NULL;+ float *x = mk_x(TM, TK, 4u);+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,4,T.q4,T.s,+ TK,TN,32,y,x,TM) == 0,+ "GENERIC_PREPARABLE is not INITIAL_XDNA_QUALIFIED");+ ck(coli_xdna_test_dispatches()==0, "dispatches 0");+ coli_xdna_prepared_release(&slot); free(x); free(y); fmt4_free(&T);+ }++ /* ================================================================== */+ printf("helper absent / ABI 1 / incomplete ABI 2\n");+ {+ struct { const char *path, *what; } hs[] = {+ { "tests/definitely_no_such_helper.dll", "helper absent" },+ { g_helper_abi1, "ABI generation 1 helper" },+ { g_helper_partial, "incomplete ABI 2 helper" }+ };+ for(size_t c = 0; c < 3; c++){+ use_helper(hs[c].path);+ coli_xdna_test_set_artifact_root(g_root);+ coli_xdna_test_set_force_execution(1);+ ColiXdnaPrepared *slot = NULL;+ float *x = mk_x(TM, TK, 5u);+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,+ 4,W.q4,W.s,TK,TN,64,y,x,TM);+ char msg[160];+ snprintf(msg,sizeof msg,"%s -> not handled", hs[c].what); ck(handled==0,msg);+ snprintf(msg,sizeof msg,"%s -> dispatches 0", hs[c].what);+ ck(coli_xdna_test_dispatches()==0,msg);+ snprintf(msg,sizeof msg,"%s -> HELPER_UNAVAILABLE", hs[c].what);+ ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_HELPER_UNAVAILABLE,msg);+ snprintf(msg,sizeof msg,"%s -> no entry point bound", hs[c].what);+ ck(coli_xdna_test_entry_points_bound()==0,msg);+ snprintf(msg,sizeof msg,"%s -> output untouched", hs[c].what);+ ck(all_poison(y,(size_t)TM*TN),msg);+ snprintf(msg,sizeof msg,"%s -> no weight prepared (gate precedes preparation)", hs[c].what);+ ck(slot==NULL,msg);+ coli_xdna_prepared_release(&slot); free(x); free(y);+ }+ }++ /* ================================================================== */+ printf("artifact absent / corrupt -- refused before any helper call\n");+ {+ char bad[2048]; snprintf(bad,sizeof bad,"%s/nonexistent_root", g_root);+ const char *roots[2]; roots[0] = bad; roots[1] = g_root;+ for(int c = 0; c < 2; c++){+ if(c == 1){+ /* corrupt a STAGING copy; canonical fixtures are never damaged */+ FILE *f = fopen(g_xclbin, "r+b");+ if(f){ fseek(f, 100, SEEK_SET); fputc(0xFF, f); fclose(f); }+ }+ use_helper(g_helper);+ coli_xdna_test_set_artifact_root(roots[c]);+ coli_xdna_test_set_force_execution(1);+ ColiXdnaPrepared *slot = NULL;+ float *x = mk_x(TM, TK, 6u);+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,+ 4,W.q4,W.s,TK,TN,64,y,x,TM);+ const char *what = c==0 ? "artifact absent" : "artifact hash mismatch";+ char msg[160];+ snprintf(msg,sizeof msg,"%s -> not handled", what); ck(handled==0,msg);+ snprintf(msg,sizeof msg,"%s -> ARTIFACT_NOT_QUALIFIED", what);+ ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED,msg);+ snprintf(msg,sizeof msg,"%s -> helper never called", what);+ ck(coli_xdna_test_helper_calls()==0,msg);+ snprintf(msg,sizeof msg,"%s -> unverified bytes never reached the helper", what);+ ck(coli_xdna_test_device_opens()==0,msg);+ snprintf(msg,sizeof msg,"%s -> output untouched", what);+ ck(all_poison(y,(size_t)TM*TN),msg);+ coli_xdna_prepared_release(&slot); free(x); free(y);+ }+ /* restore the fixture and the registry hash agreement */+ write_blob(g_xclbin, 11u, 4096);+ build_registry();+ }++ /* ================================================================== */+ printf("helper failure at every stage -- fallback intact, output untouched\n");+ {+ struct { int stage; const char *what; ColiXdnaExec want; } fs[] = {+ { 1, "device init", COLI_XDNA_EXEC_DEVICE_INIT_FAILED },+ { 2, "artifact open",COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED },+ { 3, "weight wrap", COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED },+ { 4, "dispatch", COLI_XDNA_EXEC_EXECUTE_FAILED },+ { 5, "completion (output already written by the helper)",+ COLI_XDNA_EXEC_COMPLETION_FAILED }+ };+ for(size_t c = 0; c < sizeof fs/sizeof fs[0]; c++){+ use_helper(g_helper);+ coli_xdna_test_set_artifact_root(g_root);+ coli_xdna_test_set_force_execution(1);+ /* reach into the loaded fake to arm the injection */+ ck(coli_xdna_binding()==COLI_XDNA_AVAILABLE, "fake helper bound");+ {+ void (*setf)(int) = (void(*)(int))(void*)GetProcAddress(g_xdna.dll,"fake_set_fail");+ void (*rst)(void) = (void(*)(void))(void*)GetProcAddress(g_xdna.dll,"fake_reset");+ if(rst) rst();+ if(setf) setf(fs[c].stage);+ }+ ColiXdnaPrepared *slot = NULL;+ float *x = mk_x(TM, TK, 7u);+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,+ 4,W.q4,W.s,TK,TN,64,y,x,TM);+ char msg[200];+ snprintf(msg,sizeof msg,"%s failure -> not handled", fs[c].what); ck(handled==0,msg);+ snprintf(msg,sizeof msg,"%s failure -> %s", fs[c].what,+ coli_xdna_exec_text(coli_xdna_test_last_exec()));+ ck(coli_xdna_test_last_exec()==fs[c].want,msg);+ snprintf(msg,sizeof msg,"%s failure -> caller output STILL untouched", fs[c].what);+ ck(all_poison(y,(size_t)TM*TN),msg);+ snprintf(msg,sizeof msg,"%s failure -> counted as a fallback", fs[c].what);+ ck(coli_xdna_test_fallbacks()>=1,msg);+ snprintf(msg,sizeof msg,"%s failure -> zero successful dispatches", fs[c].what);+ ck(coli_xdna_test_dispatches()==0,msg);+ snprintf(msg,sizeof msg,"%s failure -> fmt4 source unchanged", fs[c].what);+ ck(W.q4 != NULL && W.gs == 64, msg);+ coli_xdna_prepared_release(&slot); free(x); free(y);+ { void (*rst)(void) = (void(*)(void))(void*)GetProcAddress(g_xdna.dll,"fake_reset");+ if(rst) rst(); }+ }+ }++ /* ================================================================== */+ printf("weight reuse and genuinely distinct runtime weights\n");+ {+ use_helper(g_helper);+ coli_xdna_test_set_artifact_root(g_root);+ coli_xdna_test_set_force_execution(1);+ int conv0 = coli_xdna_test_conversions();++ ColiXdnaPrepared *sa = NULL, *sb = NULL;+ float *x = mk_x(TM, TK, 9u);+ float *ya = (float *)malloc((size_t)TM*TN*4);+ float *yb = (float *)malloc((size_t)TM*TN*4);+ float *ya2= (float *)malloc((size_t)TM*TN*4);++ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&sa,4,W.q4,W.s,+ TK,TN,64,ya,x,TM)==1, "gate-like weight executes");+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&sa,4,W.q4,W.s,+ TK,TN,64,ya2,x,TM)==1, "same weight again executes");+ ck(coli_xdna_test_conversions()-conv0 == 1, "prepared once, reused on the second call");+ ck(coli_xdna_test_userptr_wraps() == 1, "wrapped once for the same pointer");+ ck(coli_xdna_test_artifact_opens() == 1, "artifact runtime opened once, not per operation");+ ck(memcmp(ya,ya2,(size_t)TM*TN*4)==0, "identical inputs give identical outputs");++ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&sb,4,W2.q4,W2.s,+ TK,TN,64,yb,x,TM)==1, "up-like weight executes");+ ck(coli_xdna_test_userptr_wraps() == 2, "a different prepared image is wrapped again");+ ck(coli_xdna_test_artifact_opens() == 1, "and reuses the same artifact runtime");+ ck(memcmp(ya,yb,(size_t)TM*TN*4)!=0, "different runtime weights give different outputs");++ const unsigned short *Wb = (const unsigned short *)coli_xdna_prepared_image(sb);+ float *ref = (float *)malloc((size_t)TM*TN*4);+ oracle(ref, x, Wb, TM, TK, TN);+ size_t mism=0; for(size_t i=0;i<(size_t)TM*TN;i++) if(yb[i]!=ref[i]) mism++;+ ck(mism==0, "the second weight's output is exact too");+ free(ref);+ coli_xdna_prepared_release(&sa); coli_xdna_prepared_release(&sb);+ free(x); free(ya); free(yb); free(ya2);+ }++ /* ================================================================== */+ printf("shutdown lifetime\n");+ {+ use_helper(g_helper);+ coli_xdna_execution_shutdown();+ ck(1, "shutdown when never initialised is safe");+ coli_xdna_test_set_artifact_root(g_root);+ coli_xdna_test_set_force_execution(1);+ ColiXdnaPrepared *slot = NULL;+ float *x = mk_x(TM,TK,10u);+ float *y = (float*)malloc((size_t)TM*TN*4);+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,4,W.q4,W.s,+ TK,TN,64,y,x,TM)==1, "operation succeeds");+ coli_xdna_execution_shutdown();+ ck(1, "shutdown after a successful operation is safe");+ coli_xdna_execution_shutdown();+ ck(1, "repeated shutdown is safe");+ coli_xdna_shutdown();+ ck(coli_xdna_test_entry_points_bound()==0, "module released, no pointer survives it");+ coli_xdna_prepared_release(&slot); free(x); free(y);+ }++ coli_xdna_test_set_helper_path(NULL);+ coli_xdna_shutdown();+ coli_xdna_test_set_registry(NULL,0);+ fmt4_free(&W); fmt4_free(&W2);++ ck(coli_xdna_prepared_live_objects()==0, "no prepared object leaked");+ ck(coli_xdna_prepared_total_bytes()==0, "no prepared bytes leaked");++ printf("test_xdna_execution: %s\n", g_fail ? "FAIL" : "ok");+ return g_fail;+}diff --git a/c/tests/xdna_fake_helper.c b/c/tests/xdna_fake_helper.cnew file mode 100644index 00000000..a5def5c5--- /dev/null+++ b/c/tests/xdna_fake_helper.c@@ -0,0 +1,128 @@+/* Synthetic XDNA helper for hardware-free tests. Contains NO XRT and touches no+ * device: it implements the ABI generation 2 contract in plain C and computes+ * the GEMM on the CPU.+ *+ * This exists so the whole execution control flow -- gate order, artifact open,+ * userptr wrap, activation staging, padding, logical-row extraction, failure+ * classification and fallback -- is qualified on a machine with no NPU. The+ * physical qualification of the NUMERICS is a separate, explicitly hardware+ * run; nothing here substitutes for it.+ *+ * Build variants:+ * (default) complete, ABI 2+ * -DFAKE_ABI1 reports ABI generation 1 -- must be refused outright+ * -DFAKE_PARTIAL ABI 2 but missing one required export -- must be refused+ */++#include <stdint.h>+#include <string.h>+#include <stdlib.h>+#include <stdio.h>++#define API __declspec(dllexport)++#ifdef FAKE_ABI1+#define FAKE_ABI 1u+#else+#define FAKE_ABI 2u+#endif++enum {+ H_OK = 0,+ H_E_DEVICE = -1,+ H_E_ARTIFACT = -2,+ H_E_NOT_OPEN = -3,+ H_E_WRAP = -4,+ H_E_SIZE = -5,+ H_E_DISPATCH = -6,+ H_E_COMPLETION = -7,+ H_E_EXCEPTION = -8+};++/* Which stage the next call should fail at. 0 = none. */+enum { F_NONE = 0, F_DEVICE, F_ARTIFACT, F_WRAP, F_EXECUTE, F_COMPLETION };++static int g_fail;+static int g_open, g_wrap, g_exec, g_relw, g_shut;+static int g_is_open;+static const unsigned short *g_w; /* borrowed, never owned */+static size_t g_m, g_k, g_n;+static char g_err[256] = {0};++static float b2f(unsigned short b){+ unsigned int u = (unsigned int)b << 16;+ float f; memcpy(&f, &u, sizeof f); return f;+}++API unsigned int coli_xdna_helper_abi_version(void){ return FAKE_ABI; }+API const char *coli_xdna_helper_last_error(void){ return g_err; }++API int coli_xdna_helper_open(const char *xclbin, const char *insts,+ uint32_t m, uint32_t k, uint32_t n){+ g_open++;+ if(g_fail == F_DEVICE){ snprintf(g_err,sizeof g_err,"injected device failure"); return H_E_DEVICE; }+ if(g_fail == F_ARTIFACT){ snprintf(g_err,sizeof g_err,"injected artifact failure"); return H_E_ARTIFACT; }+ /* The host is supposed to hand us real, already-verified paths. Checking+ * they are at least openable keeps the fake honest about that contract. */+ if(!xclbin || !insts) return H_E_ARTIFACT;+ { FILE *f = fopen(xclbin, "rb"); if(!f) return H_E_ARTIFACT; fclose(f); }+ { FILE *f = fopen(insts, "rb"); if(!f) return H_E_ARTIFACT; fclose(f); }+ g_m = m; g_k = k; g_n = n; g_is_open = 1; g_w = NULL;+ return H_OK;+}++API int coli_xdna_helper_wrap_weight(void *bf16, uint64_t bytes){+ g_wrap++;+ if(!g_is_open) return H_E_NOT_OPEN;+ if(g_fail == F_WRAP){ snprintf(g_err,sizeof g_err,"injected wrap failure"); return H_E_WRAP; }+ if(!bf16 || bytes != (uint64_t)g_k * g_n * 2) return H_E_SIZE;+ /* The alignment the qualified userptr path requires. The fake enforces it+ * so a host regression that dropped the check would still be caught here. */+ if(((uintptr_t)bf16 % 4096u) != 0){ snprintf(g_err,sizeof g_err,"unaligned userptr"); return H_E_WRAP; }+ g_w = (const unsigned short *)bf16; /* borrowed; no copy is made */+ return H_OK;+}++#ifndef FAKE_PARTIAL+API int coli_xdna_helper_execute(const void *a_bf16, uint64_t a_bytes,+ void *c_f32, uint64_t c_bytes){+ g_exec++;+ if(!g_is_open || !g_w) return H_E_NOT_OPEN;+ if(g_fail == F_EXECUTE){ snprintf(g_err,sizeof g_err,"injected dispatch failure"); return H_E_DISPATCH; }+ if(a_bytes != (uint64_t)g_m * g_k * 2) return H_E_SIZE;+ if(c_bytes != (uint64_t)g_m * g_n * 4) return H_E_SIZE;+ {+ const unsigned short *A = (const unsigned short *)a_bf16;+ float *C = (float *)c_f32;+ size_t i, j, kk;+ for(i = 0; i < g_m; i++)+ for(j = 0; j < g_n; j++){+ float acc = 0.0f;+ for(kk = 0; kk < g_k; kk++)+ acc += b2f(A[i*g_k + kk]) * b2f(g_w[kk*g_n + j]);+ C[i*g_n + j] = acc;+ }+ }+ if(g_fail == F_COMPLETION){+ /* The hostile shape: the output buffer has already been written and we+ * then declare failure. The host must still not treat it as handled. */+ snprintf(g_err,sizeof g_err,"injected completion failure");+ return H_E_COMPLETION;+ }+ return H_OK;+}+#endif++API int coli_xdna_helper_release_weight(void){ g_relw++; g_w = NULL; return H_OK; }+API void coli_xdna_helper_shutdown(void){ g_shut++; g_is_open = 0; g_w = NULL; }++/* -- test controls, not part of the ABI ---------------------------------- */+API void fake_set_fail(int stage){ g_fail = stage; }+API void fake_reset(void){+ g_fail = 0; g_open = g_wrap = g_exec = g_relw = g_shut = 0;+ g_is_open = 0; g_w = NULL; g_err[0] = '\0';+}+API void fake_counts(int *open, int *wrap, int *exec, int *relw, int *shut){+ if(open) *open = g_open; if(wrap) *wrap = g_wrap; if(exec) *exec = g_exec;+ if(relw) *relw = g_relw; if(shut) *shut = g_shut;+}diff --git a/c/tests/xdna_physical_probe.c b/c/tests/xdna_physical_probe.cnew file mode 100644index 00000000..e3618d44--- /dev/null+++ b/c/tests/xdna_physical_probe.c@@ -0,0 +1,329 @@+/* W2-N7-I5 -- PHYSICAL qualification of the native XDNA lane.+ *+ * NOT a test gate: it requires an XDNA2 NPU, a working XRT installation, the+ * built coli_xdna.dll and the qualified F3 artifact bytes, none of which a+ * build machine is entitled to assume. It is built and run explicitly.+ *+ * Every step below is PRODUCTION code -- the production registry, the+ * production integrity check, the production loader, the production weight+ * preparation, the production hard-eligibility gates and the production+ * candidate function that the GLM shared gate/up call sites invoke. This file+ * supplies only a synthetic tensor and synthetic activations, and the oracles.+ *+ * usage: xdna_physical_probe <artifact-root> <helper-dll>+ */++#define main coli_glm_main_unused+#include "../colibri.c"+#undef main++#include "../backend_xdna.h"++#define PK 6144 /* the frozen F3 K */+#define PN 2048 /* the frozen F3 N */+#define PGS 64++static int g_bad = 0;++static float b2f(unsigned short b){+ unsigned int u = (unsigned int)b << 16; float f; memcpy(&f,&u,4); return f;+}+static unsigned short f2b_local(float f){+ unsigned int u; memcpy(&u,&f,4);+ u += 0x7FFFu + ((u>>16)&1u);+ return (unsigned short)(u>>16);+}++/* ---- the two BF16 oracles, and why there are two -------------------------+ *+ * ORACLE A -- ACCEPTANCE. The criterion frozen by N6-A1C-1 and reused unchanged+ * by A1A: take the exact BF16 inputs the lane submits, accumulate in DOUBLE,+ * round once to f32, and accept on absolute 0.5 OR relative 0.05. A1C-1's own+ * note on this oracle is that "the only error present is the device's+ * accumulation order", which is precisely what it is built to tolerate. This is+ * a pre-declared research criterion, not a threshold chosen after seeing+ * output.+ *+ * ORACLE B -- CONTINUITY. The same BF16 inputs accumulated sequentially in f32,+ * which is the comparator A5-R1 reported bit-exactness against.+ *+ * They are kept separate because bit-exactness against Oracle B is NOT a+ * universal property, and this slice is where that became visible. A5-R1's+ * fixture used activations that are exact multiples of 2^-7 and weights that+ * are small multiples of 0.25 -- every value exactly representable in BF16, and+ * every partial sum exact -- so accumulation order could not matter and any+ * order gave the identical result. Once activations carry a full f32 mantissa,+ * partial sums round, order matters, and the device legitimately differs from+ * one particular host summation order. Oracle B is therefore REPORTED at both+ * fixtures and GATES at neither; Oracle A is the acceptance criterion. */++static const float ABS_TOL = 0.5f, REL_TOL = 0.05f;+static int accept_a1c1(float expected, float actual){+ float d = fabsf(expected - actual);+ if(d <= ABS_TOL) return 1;+ float norm = fabsf(expected) + fabsf(actual);+ return d <= norm * REL_TOL;+}++static void oracle_double(float *out, const float *x, const unsigned short *W,+ int S, int K, int N){+ #pragma omp parallel for schedule(static)+ for(int i = 0; i < S; i++)+ for(int j = 0; j < N; j++){+ double acc = 0.0;+ for(int k = 0; k < K; k++)+ acc += (double)b2f(f2b_local(x[(size_t)i*K+k])) * (double)b2f(W[(size_t)k*N+j]);+ out[(size_t)i*N+j] = (float)acc;+ }+}++static void oracle_bf16(float *out, const float *x, const unsigned short *W,+ int S, int K, int N){+ #pragma omp parallel for schedule(static)+ for(int i = 0; i < S; i++)+ for(int j = 0; j < N; j++){+ float acc = 0.0f;+ for(int k = 0; k < K; k++)+ acc += b2f(f2b_local(x[(size_t)i*K+k])) * b2f(W[(size_t)k*N+j]);+ out[(size_t)i*N+j] = acc;+ }+}++/* Report both oracles for one result. Returns 1 when the ACCEPTANCE criterion+ * holds. */+static int report_oracles(const float *y, const float *x, const unsigned short *W,+ int S, int K, int N){+ float *refA = (float*)malloc((size_t)S*N*4);+ float *refB = (float*)malloc((size_t)S*N*4);+ oracle_double(refA, x, W, S, K, N);+ oracle_bf16(refB, x, W, S, K, N);+ size_t mism = 0, bitmis = 0, nan_n = 0, inf_n = 0;+ double maxabsA = 0, maxrelA = 0, maxabsB = 0;+ for(size_t i = 0; i < (size_t)S*N; i++){+ if(!accept_a1c1(refA[i], y[i])) mism++;+ double ae = fabs((double)y[i]-(double)refA[i]);+ if(ae > maxabsA) maxabsA = ae;+ double den = fabs((double)refA[i]);+ if(den > 1e-6 && ae/den > maxrelA) maxrelA = ae/den;+ if(y[i] != refB[i]) bitmis++;+ double be = fabs((double)y[i]-(double)refB[i]);+ if(be > maxabsB) maxabsB = be;+ if(y[i] != y[i]) nan_n++;+ else if(y[i] > 3.0e38f || y[i] < -3.0e38f) inf_n++;+ }+ printf("ORACLE_A_ACCEPT mismatches=%llu (criterion abs<=0.5 OR rel<=0.05) max_abs=%.9f max_rel=%.9f\n",+ (unsigned long long)mism, maxabsA, maxrelA);+ printf("ORACLE_B_EXACT bit_mismatches=%llu max_abs=%.9f\n",+ (unsigned long long)bitmis, maxabsB);+ printf("NAN %llu\nINF %llu\n",+ (unsigned long long)nan_n, (unsigned long long)inf_n);+ free(refA); free(refB);+ return mism == 0;+}++int main(int argc, char **argv){+ if(argc < 3){ fprintf(stderr,"usage: xdna_physical_probe <artifact-root> <helper-dll>\n"); return 2; }+ const char *root = argv[1], *helper = argv[2];++ printf("W2-N7-I5 PHYSICAL XDNA QUALIFICATION\n");+ printf("PID %lu\n", (unsigned long)GetCurrentProcessId());+ printf("ARTIFACT_ROOT %s\n", root);+ printf("HELPER %s\n", helper);+ printf("REGISTRY production (not a test registry)\n");++ coli_xdna_test_set_helper_path(helper);+ coli_xdna_test_set_artifact_root(root);++ printf("BINDING %s\n", coli_xdna_binding_text(coli_xdna_binding()));+ if(coli_xdna_binding() != COLI_XDNA_AVAILABLE){+ printf("RESULT FAIL -- helper did not bind\n"); return 1; }+ printf("ABI host %u\n", (unsigned)COLI_XDNA_ABI_VERSION);++ /* Production static verdict, on the production registry and the real bytes. */+ ColiXdnaRequest q;+ q.family = COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP;+ q.m = 64; q.k = PK; q.n = PN;+ q.in_dtype = COLI_XDNA_DT_BF16; q.weight_dtype = COLI_XDNA_DT_BF16;+ q.out_dtype = COLI_XDNA_DT_F32;+ ColiXdnaStatic st = coli_xdna_static_eligibility(&q, root);+ printf("STATIC %s\n", coli_xdna_static_text(st));+ if(st != COLI_XDNA_STATIC_QUALIFIED){+ printf("RESULT FAIL -- artifact integrity gate refused\n"); return 1; }++ /* Two genuinely different runtime weights: gate-like and up-like. */+ const int rb = (PK+1)/2, ng = (PK+PGS-1)/PGS;+ QT G, U;+ memset(&G,0,sizeof G); memset(&U,0,sizeof U);+ QT *ts[2] = { &G, &U };+ unsigned seeds[2] = { 20260824u, 77771111u };+ for(int t = 0; t < 2; t++){+ QT *w = ts[t];+ w->fmt = 4; w->I = PK; w->O = PN; w->gs = PGS;+ w->q4 = (uint8_t*)malloc((size_t)PN*rb);+ w->s = (float*)malloc((size_t)PN*ng*sizeof(float));+ unsigned s = seeds[t];+ for(size_t i=0;i<(size_t)PN*rb;i++){ s=s*1664525u+1013904223u; w->q4[i]=(uint8_t)(s>>24); }+ for(size_t i=0;i<(size_t)PN*ng;i++){ s=s*1664525u+1013904223u;+ w->s[i] = 0.25f*(float)(1+((s>>26)&3)); }+ }++ coli_xdna_test_set_force_execution(1);++ int Ms[] = { 1, 32, 64 };+ const char *names[2] = { "sh_gate-like", "sh_up-like" };+ unsigned long long hashes[2][3];++ for(int t = 0; t < 2; t++){+ for(size_t c = 0; c < sizeof Ms/sizeof Ms[0]; c++){+ int S = Ms[c];+ QT *w = ts[t];+ float *x = (float*)malloc((size_t)S*PK*4);+ unsigned s = 31337u + (unsigned)S + 1000u*(unsigned)t;+ for(size_t i=0;i<(size_t)S*PK;i++){ s=s*1664525u+1013904223u;+ x[i] = (float)((int)((s>>20)&255)-128)*0.0078125f; }+ float *y = (float*)malloc((size_t)S*PN*4);+ for(size_t i=0;i<(size_t)S*PN;i++) y[i] = -123456.0f;++ int d0 = coli_xdna_test_dispatches();+ double t0 = now_s();+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,+ &w->xdna, w->fmt, w->q4, w->s,+ w->I, w->O, w->gs, y, x, S);+ double ms = (now_s()-t0)*1000.0;++ printf("\n-- %s M=%d --\n", names[t], S);+ printf("HANDLED %d\n", handled);+ printf("HARD %s\n", coli_xdna_hard_text(coli_xdna_test_last_hard()));+ printf("EXEC %s\n", coli_xdna_exec_text(coli_xdna_test_last_exec()));+ if(!handled){+ printf("HELPER_ERR (see loader)\n");+ printf("RESULT FAIL -- native execution declined or failed\n");+ g_bad = 1; free(x); free(y); continue;+ }+ printf("DISPATCHES %d (+%d)\n", coli_xdna_test_dispatches(),+ coli_xdna_test_dispatches()-d0);+ printf("ARTIFACT_OPENS %d\n", coli_xdna_test_artifact_opens());+ printf("USERPTR_WRAPS %d\n", coli_xdna_test_userptr_wraps());+ printf("PADDED_OPS %d\n", coli_xdna_test_padded_operations());+ printf("ELAPSED_MS %.3f\n", ms);++ const unsigned short *W = (const unsigned short*)coli_xdna_prepared_image(w->xdna);+ printf("PREPARED_PTR %p mod4096=%llu\n", (const void*)W,+ (unsigned long long)((uintptr_t)W % 4096u));+ printf("PREPARED_BYTES %llu\n",+ (unsigned long long)coli_xdna_prepared_bytes(w->xdna));++ if(!report_oracles(y, x, W, S, PK, PN)) g_bad = 1;++ /* ---- ORACLE B: the exact current path, for characterisation ---- */+ float *refB = (float*)malloc((size_t)S*PN*4);+ matmul_qt(refB, x, w, S);+ double maxabsB = 0, maxrelB = 0, sse = 0;+ for(size_t i=0;i<(size_t)S*PN;i++){+ double a = (double)y[i], b = (double)refB[i], d = fabs(a-b);+ if(d>maxabsB) maxabsB=d;+ double den = fabs(b);+ if(den > 1e-12 && d/den > maxrelB) maxrelB = d/den;+ sse += d*d;+ }+ printf("VS_MATMUL_QT max_abs=%.6f max_rel=%.3f rms=%.6f\n",+ maxabsB, maxrelB, sqrt(sse/((double)S*PN)));++ unsigned long long h = 1469598103934665603ULL;+ for(size_t i=0;i<(size_t)S*PN*4;i++){ h ^= ((const unsigned char*)y)[i];+ h *= 1099511628211ULL; }+ hashes[t][c] = h;+ printf("OUTPUT_HASH %016llx\n", h);++ free(refB); free(x); free(y);+ }+ }++ printf("\n-- dynamic runtime weights --\n");+ for(size_t c = 0; c < 3; c++)+ printf("M=%-3d gate=%016llx up=%016llx distinct=%s\n", Ms[c],+ hashes[0][c], hashes[1][c],+ hashes[0][c]!=hashes[1][c] ? "YES" : "NO");+ for(size_t c = 0; c < 3; c++) if(hashes[0][c]==hashes[1][c]) g_bad = 1;++ printf("\nARTIFACT_OPENS_TOTAL %d (one artifact serving every M and both weights)\n",+ coli_xdna_test_artifact_opens());+ printf("DISPATCHES_TOTAL %d\n", coli_xdna_test_dispatches());+ printf("COMPLETIONS_TOTAL %d\n", coli_xdna_test_completions());+ printf("DEVICE_OPENS %d\n", coli_xdna_test_device_opens());+ printf("CONVERSIONS %d\n", coli_xdna_test_conversions());++ /* ---- characterising the difference against the CURRENT path -------------+ *+ * The activations above are all exact multiples of 2^-7 and the weights all+ * small multiples of 0.25, so every value is exactly representable in BF16+ * and the f32->BF16 conversion is lossless. That makes the current path and+ * the lane agree exactly -- a true result, but one that characterises+ * nothing, because it never exercises the conversion the two paths differ+ * over.+ *+ * This case uses activations with a full f32 mantissa, which BF16 genuinely+ * rounds. The lane must STILL be bit-exact against its own BF16 oracle; the+ * difference against matmul_qt is the real, expected representational gap,+ * reported and not tested against any invented threshold. */+ {+ int S = 64;+ float *x = (float*)malloc((size_t)S*PK*4);+ unsigned s = 0xC0FFEEu;+ for(size_t i=0;i<(size_t)S*PK;i++){+ s = s*1664525u+1013904223u;+ /* a generic f32 in [-1,1): 24 significant bits, not BF16-exact */+ x[i] = ((float)(s >> 8) / (float)(1u<<24)) * 2.0f - 1.0f;+ }+ float *y = (float*)malloc((size_t)S*PN*4);+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,+ &G.xdna, G.fmt, G.q4, G.s,+ G.I, G.O, G.gs, y, x, S);+ printf("\n-- generic f32 activations (BF16 rounding is LOSSY here), M=%d --\n", S);+ printf("HANDLED %d\n", handled);+ if(!handled) g_bad = 1;+ else {+ const unsigned short *W = (const unsigned short*)coli_xdna_prepared_image(G.xdna);+ if(!report_oracles(y, x, W, S, PK, PN)) g_bad = 1;++ float *refB = (float*)malloc((size_t)S*PN*4);+ matmul_qt(refB, x, &G, S);+ double maxabsB=0, maxrelB=0, sse=0, refmax=0;+ for(size_t i=0;i<(size_t)S*PN;i++){+ double a=(double)y[i], b=(double)refB[i], d=fabs(a-b);+ if(d>maxabsB) maxabsB=d;+ if(fabs(b)>refmax) refmax=fabs(b);+ if(fabs(b)>1e-12 && d/fabs(b) > maxrelB) maxrelB = d/fabs(b);+ sse += d*d;+ }+ printf("VS_MATMUL_QT max_abs=%.6f max_rel=%.3f rms=%.6f ref_max_abs=%.6f\n",+ maxabsB, maxrelB, sqrt(sse/((double)S*PN)), refmax);+ printf("INTERPRETATION expected representational gap: matmul_qt accumulates f32\n");+ printf(" activations against dequantised int4; the lane is BF16\n");+ printf(" throughout. max_rel is inflated by near-zero reference\n");+ printf(" dot products and is NOT an error measure.\n");+ free(refB);+ }+ free(x); free(y);+ }++ /* Negative control on live hardware: same shape, wrong family. */+ {+ float *x = (float*)malloc((size_t)8*PK*4); memset(x,0,(size_t)8*PK*4);+ float *y = (float*)malloc((size_t)8*PN*4);+ int before = coli_xdna_test_dispatches();+ int h2 = coli_xdna_try_matmul(COLI_XDNA_FAMILY_NONE, &G.xdna, G.fmt, G.q4, G.s,+ G.I, G.O, G.gs, y, x, 8);+ printf("\nWRONG_FAMILY_HANDLED %d dispatch_delta %d (%s)\n", h2,+ coli_xdna_test_dispatches()-before,+ coli_xdna_hard_text(coli_xdna_test_last_hard()));+ if(h2 || coli_xdna_test_dispatches()!=before) g_bad = 1;+ free(x); free(y);+ }++ coli_xdna_execution_shutdown();+ coli_xdna_shutdown();+ printf("\nSHUTDOWN clean\n");+ printf("RESULT %s\n", g_bad ? "FAIL" : "PASS");+ return g_bad;+}