Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

feat(xdna): add native GLM shared execution lane

Kenneth-Javier Ortigozaf8d618c54c26
diff --git a/GPU_BACKENDS.md b/GPU_BACKENDS.md
index ca7d22a7..c71423e1 100644
--- a/GPU_BACKENDS.md
+++ b/GPU_BACKENDS.md
@@ -431,5 +431,100 @@ but does not require it, and correctness and publication semantics come first; a
Conversion needs no helper, no artifact, no XRT and no device β€” it is engine
representation logic, and the helper knows nothing about nibble packing or group
-scales. **There is still no XDNA execution**: connecting a real operation to the
-device is the next slice's work.
+scales.
+### Native execution: the GLM shared gate/up lane
+
+Colibri can now execute one real operation on the NPU: the **MoE shared-expert
+gate and up projections**, and nothing else. This is the whole supported surface,
+and the constraints below are hard gates rather than guidance.
+
+```
+family MoE shared-expert gate / up (sh_gate, sh_up)
+K 6144
+N 2048
+stored format fmt=4 grouped int4, group size 64
+logical M 1..64, zero-padded to the artifact's M=64
+execution blocking, one operation at a time
+```
+
+`sh_down` is deliberately **not** accelerated: its orientation is `I=2048,
+O=6144`, which is not what the qualified artifact computes. Nor is the generic
+`matmul_qt` intercepted. The semantic family is passed explicitly by the call
+site and never inferred from a shape, so an unrelated operation that happens to
+be 6144x2048 cannot inherit this one's qualification.
+
+Row padding is legitimate because `C = A x B` is row-independent: output row *i*
+depends only on input row *i* and on B. Only the logical rows are copied out;
+padded rows never reach anything downstream. Logical M above 64 declines to the
+current path β€” row tiling is a qualified strategy but is not implemented here.
+
+**There is no automatic selection.** No `--xdna` flag, no `COLI_XDNA`
+environment variable and no economic policy exists. An ordinary build with a
+helper, a device and valid artifacts present still runs exactly the path it runs
+today; the lane is reachable only from an internal test control. Deciding *when*
+XDNA is preferable is a separate concern with a separate owner.
+
+Hard eligibility is evaluated in this order, cheapest and most semantic first,
+and no later gate can excuse an earlier refusal:
+
+```
+family -> logical M -> K/N -> fmt -> group size -> registry row
+ -> artifact qualification -> artifact present -> artifact SHA256
+ -> helper ABI -> prepared weight VALID -> 4096-byte alignment
+ -> device/runtime -> artifact runtime object -> userptr wrap -> execute
+```
+
+Any refusal, and any helper failure at any stage, returns "not handled" and the
+caller runs its current `matmul_qt` path. The candidate never calls `matmul_qt`
+itself, so there is no recursion and no double dispatch, and the caller's output
+buffer is written only after the helper reports successful completion β€” a
+failure cannot leave a half-written result behind.
+
+### Building the optional helper
+
+The helper is the only component that links XRT, and it is **opt-in**: an
+ordinary build compiles without XRT headers, links without XRT, imports no XRT
+DLL, and does not require `coli_xdna.dll` to exist.
+
+Tested provenance:
+
+```
+compiler MSVC 14.44.35207 (Visual Studio 2022), Windows SDK 10.0.26100
+XRT SDK 2.21.75 -- headers + xrt_coreutil.lib
+XRT runtime 2.21.0 -- installed xrt_coreutil.dll
+device AMD XDNA2 (Ryzen AI), driver 32.0.20102.3930
+```
+
+```
+cl /nologo /LD /EHsc /std:c++17 /Zc:__cplusplus /O2 /MD /W3 ^
+ /I <XRT_SDK>\include c\backend_xdna_helper.cpp ^
+ /Fe:coli_xdna.dll /link <XRT_SDK>\lib\xrt_coreutil.lib
+```
+
+`/Zc:__cplusplus` is required: without it MSVC reports `__cplusplus` as
+`199711L`, `xrt/detail/any.h` takes its Boost branch, and the build fails on a
+header Colibri does not use.
+
+The host looks for `coli_xdna.dll` **beside the executable, by absolute path
+only** β€” no PATH search, no current directory, no application-directory
+fallback. A helper that is not there is simply absent, which is a normal state.
+
+The helper ABI is generation **2**. Generation 1 is refused outright rather than
+partially bound: it exports two of the seven entry points generation 2 requires
+and cannot execute anything, so binding it would produce a helper that reports
+availability and then fails. Binding is all-or-nothing.
+
+### Artifacts are not shipped
+
+The qualified `.xclbin` and instruction-stream bytes are **not** distributed with
+Colibri, and how they eventually should be is still an open question. The
+registry carries their SHA256, and a missing, unreadable or hash-mismatched
+artifact fails closed before the device is opened or any byte reaches the
+helper. With no artifact root configured β€” the default, and the only production
+value β€” every request declines.
+
+### What is not claimed
+
+No full-model acceleration, no token-throughput figure, no general XDNA backend,
+no routed-expert support, no concurrency with the GPU, and no scheduler. One
+family, one shape, one bucket, one dispatch at a time.
diff --git a/c/.gitignore b/c/.gitignore
index fbab5078..5b1b58af 100644
--- a/c/.gitignore
+++ b/c/.gitignore
@@ -46,3 +46,6 @@ tests/test_inkling_serve_framing
*.dll
cublasLt64*.dll
cudart64*.dll
+# Optional XDNA test scaffolding that the *.o / *.dll globs above do not cover.
+tests/xdna_exec_fixtures/
+tests/xdna_physical_probe
diff --git a/c/Makefile b/c/Makefile
index f17dd1f6..9cb1c58d 100644
--- a/c/Makefile
+++ b/c/Makefile
@@ -1091,6 +1091,31 @@ tests/test_xdna_prepared_state$(EXE): tests/test_xdna_prepared_state.c backend_x
tests/test_xdna_registry$(EXE): tests/test_xdna_registry.c backend_xdna.c backend_xdna.h
$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)
+# Synthetic ABI-2 helpers for the execution tests. Plain C, no XRT, no device:
+# they let the whole execution control flow be qualified on a machine with no
+# NPU. Three variants so the all-or-nothing binding contract is testable.
+XDNA_FAKE_HELPERS = tests/xdna_fake_helper.dll \
+ tests/xdna_fake_helper_abi1.dll \
+ tests/xdna_fake_helper_partial.dll
+tests/xdna_fake_helper.dll: tests/xdna_fake_helper.c
+ $(CC) $(CFLAGS) -shared $< -o $@
+tests/xdna_fake_helper_abi1.dll: tests/xdna_fake_helper.c
+ $(CC) $(CFLAGS) -DFAKE_ABI1 -shared $< -o $@
+tests/xdna_fake_helper_partial.dll: tests/xdna_fake_helper.c
+ $(CC) $(CFLAGS) -DFAKE_PARTIAL -shared $< -o $@
+
+tests/test_xdna_execution$(EXE): tests/test_xdna_execution.c backend_xdna.c backend_xdna.h compat.h $(XDNA_FAKE_HELPERS)
+ $(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)
+
+# NOT a test gate -- intentionally absent from TEST_BINS (its name does not match
+# the tests/test_*$(EXE) pattern the gate list is derived from). It needs a real
+# XDNA2 NPU, a working XRT, a built coli_xdna.dll and the qualified artifact
+# bytes, none of which a build machine may assume. Build and run it explicitly:
+# make tests/xdna_physical_probe$(EXE)
+# tests/xdna_physical_probe <artifact-root> <path-to-coli_xdna.dll>
+tests/xdna_physical_probe$(EXE): tests/xdna_physical_probe.c colibri.c backend_xdna.c backend_xdna.h st.h uring.h json.h tok.h tok_unicode.h compat.h grammar.h tier.h quant.h sample.h kv_persist.h telemetry.h route_trace.h
+ $(CC) $(CFLAGS) -DCOLI_XDNA $< backend_xdna.c -o $@ $(LDFLAGS)
+
tests/test_json$(EXE): tests/test_json.c json.h
$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)
diff --git a/c/backend_xdna.c b/c/backend_xdna.c
index d472d342..659e20b6 100644
--- a/c/backend_xdna.c
+++ b/c/backend_xdna.c
@@ -32,34 +32,78 @@
#include <windows.h>
#endif
-/* Entry points required of a helper at this ABI generation. Kept minimal on
- * purpose: a version handshake, and the release call this file's own lifetime
- * contract needs. Later slices add device/artifact/execute entry points and
- * bump COLI_XDNA_ABI_VERSION with them. */
+/* Entry points required of a helper at ABI generation 2. Still minimal: exactly
+ * the calls the first blocking operation needs, and nothing speculative.
+ *
+ * Everything crossing this boundary is a fixed-width integer, a size_t, a C
+ * string, or a raw pointer with an explicit byte count. No engine type, no XRT
+ * type and no C++ type appears in any signature, so the helper cannot be handed
+ * -- and cannot acquire -- any knowledge of tensors, experts, layers, routing or
+ * models. It executes what it is given. */
typedef unsigned int (*coli_xdna_fn_abi_version)(void);
+/* device + runtime + artifact program, for one (M,K,N). */
+typedef int (*coli_xdna_fn_open)(const char *xclbin, const char *insts,
+ uint32_t m, uint32_t k, uint32_t n);
+/* wrap caller-owned prepared BF16 through the qualified userptr path. */
+typedef int (*coli_xdna_fn_wrap_weight)(void *bf16, uint64_t bytes);
+/* blocking: submit, wait, copy the artifact-M output out. */
+typedef int (*coli_xdna_fn_execute)(const void *a_bf16, uint64_t a_bytes,
+ void *c_f32, uint64_t c_bytes);
+typedef int (*coli_xdna_fn_release_weight)(void);
typedef void (*coli_xdna_fn_shutdown)(void);
+typedef const char *(*coli_xdna_fn_last_error)(void);
static struct {
ColiXdnaBinding state;
int load_attempts;
- char path[512]; /* test override; empty = default lookup */
+ char path[1024]; /* test override; empty = default lookup */
#ifdef _WIN32
HMODULE dll;
#endif
- coli_xdna_fn_abi_version abi_version;
- coli_xdna_fn_shutdown shutdown;
-} g_xdna = { COLI_XDNA_UNPROBED, 0, {0},
-#ifdef _WIN32
- NULL,
-#endif
- NULL, NULL };
+ coli_xdna_fn_abi_version abi_version;
+ coli_xdna_fn_open open;
+ coli_xdna_fn_wrap_weight wrap_weight;
+ coli_xdna_fn_execute execute;
+ coli_xdna_fn_release_weight release_weight;
+ coli_xdna_fn_shutdown shutdown;
+ coli_xdna_fn_last_error last_error;
+} g_xdna;
/* Drop every callable pointer. Always runs BEFORE the module is released, so a
* rejected or shut-down helper can never leave a pointer into an unmapped
* module behind -- the all-or-nothing half of the binding contract. */
static void coli_xdna_clear_entry_points(void){
- g_xdna.abi_version = NULL;
- g_xdna.shutdown = NULL;
+ g_xdna.abi_version = NULL;
+ g_xdna.open = NULL;
+ g_xdna.wrap_weight = NULL;
+ g_xdna.execute = NULL;
+ g_xdna.release_weight = NULL;
+ g_xdna.shutdown = NULL;
+ g_xdna.last_error = NULL;
+}
+
+/* Every helper call goes through one of these. They are the only places a
+ * helper pointer is dereferenced, and each refuses when unbound rather than
+ * trusting a caller to have checked. */
+static int coli_xdna_helper_open_call(const char *xb, const char *ib,
+ unsigned m, unsigned k, unsigned n){
+ if(!g_xdna.open) return -1;
+ return g_xdna.open(xb, ib, (uint32_t)m, (uint32_t)k, (uint32_t)n);
+}
+static int coli_xdna_helper_wrap_call(void *p, uint64_t bytes){
+ if(!g_xdna.wrap_weight) return -1;
+ return g_xdna.wrap_weight(p, bytes);
+}
+static int coli_xdna_helper_execute_call(const void *a, uint64_t ab, void *c, uint64_t cb){
+ if(!g_xdna.execute) return -1;
+ return g_xdna.execute(a, ab, c, cb);
+}
+static int coli_xdna_helper_release_weight_call(void){
+ if(!g_xdna.release_weight) return -1;
+ return g_xdna.release_weight();
+}
+static void coli_xdna_helper_shutdown_call(void){
+ if(g_xdna.shutdown) g_xdna.shutdown();
}
const char *coli_xdna_binding_text(ColiXdnaBinding state){
@@ -136,8 +180,18 @@ static void coli_xdna_probe(void){
#endif
coli_xdna_fn_abi_version abi_version =
(coli_xdna_fn_abi_version)GetProcAddress(dll, "coli_xdna_helper_abi_version");
+ coli_xdna_fn_open open_fn =
+ (coli_xdna_fn_open)GetProcAddress(dll, "coli_xdna_helper_open");
+ coli_xdna_fn_wrap_weight wrap_fn =
+ (coli_xdna_fn_wrap_weight)GetProcAddress(dll, "coli_xdna_helper_wrap_weight");
+ coli_xdna_fn_execute exec_fn =
+ (coli_xdna_fn_execute)GetProcAddress(dll, "coli_xdna_helper_execute");
+ coli_xdna_fn_release_weight relw_fn =
+ (coli_xdna_fn_release_weight)GetProcAddress(dll, "coli_xdna_helper_release_weight");
coli_xdna_fn_shutdown shutdown =
(coli_xdna_fn_shutdown)GetProcAddress(dll, "coli_xdna_helper_shutdown");
+ coli_xdna_fn_last_error lasterr_fn =
+ (coli_xdna_fn_last_error)GetProcAddress(dll, "coli_xdna_helper_last_error");
#if defined(__GNUC__)
#pragma GCC diagnostic pop
#endif
@@ -152,20 +206,31 @@ static void coli_xdna_probe(void){
g_xdna.state = COLI_XDNA_ABI_INCOMPATIBLE;
return;
}
- if(!shutdown){
+ /* All-or-nothing: one missing entry point rejects the whole helper. A
+ * partially bound helper would report availability and then fail somewhere
+ * further in, where the failure is far harder to attribute. */
+ if(!open_fn || !wrap_fn || !exec_fn || !relw_fn || !shutdown || !lasterr_fn){
FreeLibrary(dll);
g_xdna.state = COLI_XDNA_SYMBOL_INCOMPLETE;
return;
}
- g_xdna.dll = dll;
- g_xdna.abi_version = abi_version;
- g_xdna.shutdown = shutdown;
- g_xdna.state = COLI_XDNA_AVAILABLE;
+ g_xdna.dll = dll;
+ g_xdna.abi_version = abi_version;
+ g_xdna.open = open_fn;
+ g_xdna.wrap_weight = wrap_fn;
+ g_xdna.execute = exec_fn;
+ g_xdna.release_weight = relw_fn;
+ g_xdna.shutdown = shutdown;
+ g_xdna.last_error = lasterr_fn;
+ g_xdna.state = COLI_XDNA_AVAILABLE;
}
void coli_xdna_shutdown(void){
- if(g_xdna.shutdown) g_xdna.shutdown();
+ /* Release lane state (which itself calls the helper) BEFORE the module goes
+ * away: no helper-owned object may outlive the module that created it, and
+ * no host pointer into it may survive the FreeLibrary. */
+ coli_xdna_execution_shutdown();
coli_xdna_clear_entry_points();
if(g_xdna.dll){
FreeLibrary(g_xdna.dll);
@@ -185,6 +250,7 @@ static void coli_xdna_probe(void){
}
void coli_xdna_shutdown(void){
+ coli_xdna_execution_shutdown();
coli_xdna_clear_entry_points();
}
@@ -364,7 +430,13 @@ static const int g_xdna_production_count =
(int)(sizeof g_xdna_production_rows / sizeof g_xdna_production_rows[0]);
static const ColiXdnaArtifact *g_rows = g_xdna_production_rows;
-static int g_nrows = 0;
+/* Initialised to the production count, NOT to 0. Zero here would leave the
+ * production table installed but empty, so every production lookup would answer
+ * UNKNOWN_FAMILY until a test happened to install a registry -- invisible while
+ * the only callers were tests that always install one, and wrong the moment a
+ * production caller arrived. I5 is that first production caller. */
+static int g_nrows =
+ (int)(sizeof g_xdna_production_rows / sizeof g_xdna_production_rows[0]);
const char *coli_xdna_static_text(ColiXdnaStatic s){
switch(s){
@@ -464,7 +536,7 @@ static int coli_xdna_join(char *out, size_t cap, const char *root, const char *n
* bytes that were qualified". */
static ColiXdnaStatic coli_xdna_check_file(const char *root, const char *name,
const char *expect_hex){
- char path[1024];
+ char path[2048];
if(!coli_xdna_join(path, sizeof path, root, name))
return COLI_XDNA_STATIC_ARTIFACT_UNAVAILABLE;
unsigned char got[32];
@@ -808,10 +880,345 @@ ColiXdnaPrepResult coli_xdna_prepare_from_fmt4(ColiXdnaPrepared *p,
return COLI_XDNA_PREP_OK;
}
-/* This slice opens no device, calls no helper entry point and wraps no pointer.
- * The counters make that assertable rather than merely stated. Conversion is now
- * implemented, so its counter is no longer pinned at zero. */
-int coli_xdna_test_device_opens(void){ return 0; }
-int coli_xdna_test_helper_calls(void){ return 0; }
-int coli_xdna_test_userptr_wraps(void){ return 0; }
+/* ======================================================================
+ * Full hard eligibility and native execution (W2-N7-I5)
+ *
+ * The engine keeps every decision that requires knowing what the operation
+ * MEANS: which family it is, which artifact answers it, whether the bytes are
+ * trustworthy, how the weight is represented, whether the request is in range.
+ * The helper is handed an already-selected, already-verified artifact and an
+ * already-prepared buffer, and executes. It is told nothing about tensors,
+ * experts, layers, routing or models, and it decides nothing.
+ * ====================================================================== */
+
+const char *coli_xdna_hard_text(ColiXdnaHard h){
+ switch(h){
+ case COLI_XDNA_HARD_ELIGIBLE: return "HARD_ELIGIBLE";
+ case COLI_XDNA_HARD_FAMILY_UNSUPPORTED: return "FAMILY_UNSUPPORTED";
+ case COLI_XDNA_HARD_M_OUT_OF_RANGE: return "M_OUT_OF_RANGE";
+ case COLI_XDNA_HARD_SHAPE_UNSUPPORTED: return "SHAPE_UNSUPPORTED";
+ case COLI_XDNA_HARD_FORMAT_UNSUPPORTED: return "FORMAT_UNSUPPORTED";
+ case COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED: return "GROUP_SIZE_UNSUPPORTED";
+ case COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED: return "ARTIFACT_NOT_QUALIFIED";
+ case COLI_XDNA_HARD_PREPARED_INVALID: return "PREPARED_INVALID";
+ case COLI_XDNA_HARD_ALIGNMENT_INVALID: return "ALIGNMENT_INVALID";
+ case COLI_XDNA_HARD_HELPER_UNAVAILABLE: return "HELPER_UNAVAILABLE";
+ case COLI_XDNA_HARD_DEVICE_UNAVAILABLE: return "DEVICE_UNAVAILABLE";
+ case COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE:return "ARTIFACT_RUNTIME_UNAVAILABLE";
+ case COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE: return "WEIGHT_WRAP_UNAVAILABLE";
+ }
+ return "UNKNOWN";
+}
+
+const char *coli_xdna_exec_text(ColiXdnaExec e){
+ switch(e){
+ case COLI_XDNA_EXEC_OK: return "OK";
+ case COLI_XDNA_EXEC_DECLINED: return "DECLINED";
+ case COLI_XDNA_EXEC_DEVICE_INIT_FAILED: return "DEVICE_INIT_FAILED";
+ case COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED: return "ARTIFACT_OPEN_FAILED";
+ case COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED: return "WEIGHT_PREPARE_FAILED";
+ case COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED: return "WEIGHT_WRAP_FAILED";
+ case COLI_XDNA_EXEC_ACTIVATION_FAILED: return "ACTIVATION_FAILED";
+ case COLI_XDNA_EXEC_EXECUTE_FAILED: return "EXECUTE_FAILED";
+ case COLI_XDNA_EXEC_COMPLETION_FAILED: return "COMPLETION_FAILED";
+ }
+ return "UNKNOWN";
+}
+
+/* ---- execution-lane state ------------------------------------------------
+ *
+ * Transient per-operation staging (activation, output) and the identity of the
+ * artifact currently open in the helper. The prepared WEIGHT is not here: it
+ * belongs to the tensor, which outlives any of this. */
+static int g_xdna_force; /* internal test control only */
+static char g_xdna_root[1024]; /* artifact root; empty = none */
+static ColiXdnaHard g_xdna_last_hard = COLI_XDNA_HARD_ELIGIBLE;
+static ColiXdnaExec g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
+static int g_xdna_dispatches, g_xdna_completions, g_xdna_artifact_opens;
+static int g_xdna_act_preps, g_xdna_padded_ops, g_xdna_fallbacks;
+static int g_xdna_device_opens, g_xdna_helper_calls, g_xdna_userptr_wraps;
+
+/* Which artifact the helper currently holds open, so a run of operations on the
+ * same shape does not reopen the device, the context or the program. This is a
+ * RUNTIME object cache, not an identity cache: it can only ever hold what the
+ * engine already selected, and a different selection closes it and reopens. */
+static struct {
+ int open;
+ unsigned m, k, n;
+ /* the wrapped weight, identified by the exact pointer the engine handed us */
+ const void *wrapped;
+ unsigned short *act; size_t act_cap; /* [artifact_m, K] BF16 staging */
+ float *out; size_t out_cap; /* [artifact_m, N] F32 staging */
+} g_lane;
+
+/* ---- helper entry points (ABI generation 2) ------------------------------
+ *
+ * Everything crossing this boundary is a fixed-width integer, a size_t, a raw
+ * pointer with an explicit byte count, or a C string. No C++ type, no XRT type,
+ * no engine type, and no exception: the helper catches everything and converts
+ * it to one of these status codes. */
+enum {
+ COLI_XDNA_H_OK = 0,
+ COLI_XDNA_H_E_DEVICE = -1,
+ COLI_XDNA_H_E_ARTIFACT = -2,
+ COLI_XDNA_H_E_NOT_OPEN = -3,
+ COLI_XDNA_H_E_WRAP = -4,
+ COLI_XDNA_H_E_SIZE = -5,
+ COLI_XDNA_H_E_DISPATCH = -6,
+ COLI_XDNA_H_E_COMPLETION = -7,
+ COLI_XDNA_H_E_EXCEPTION = -8
+};
+
+int coli_xdna_test_dispatches(void){ return g_xdna_dispatches; }
+int coli_xdna_test_completions(void){ return g_xdna_completions; }
+int coli_xdna_test_artifact_opens(void){ return g_xdna_artifact_opens; }
+int coli_xdna_test_activation_preparations(void){ return g_xdna_act_preps; }
+int coli_xdna_test_padded_operations(void){ return g_xdna_padded_ops; }
+int coli_xdna_test_fallbacks(void){ return g_xdna_fallbacks; }
+ColiXdnaHard coli_xdna_test_last_hard(void){ return g_xdna_last_hard; }
+ColiXdnaExec coli_xdna_test_last_exec(void){ return g_xdna_last_exec; }
+
+void coli_xdna_test_set_force_execution(int on){ g_xdna_force = on ? 1 : 0; }
+
+void coli_xdna_test_set_artifact_root(const char *root){
+ if(root && *root) snprintf(g_xdna_root, sizeof g_xdna_root, "%s", root);
+ else g_xdna_root[0] = '\0';
+}
+
+int coli_xdna_test_device_opens(void){ return g_xdna_device_opens; }
+int coli_xdna_test_helper_calls(void){ return g_xdna_helper_calls; }
+int coli_xdna_test_userptr_wraps(void){ return g_xdna_userptr_wraps; }
int coli_xdna_test_conversions(void){ return g_xdna_conversions; }
+
+/* ---- the candidate --------------------------------------------------------
+ *
+ * Gate order is the point of this function, not an accident of it. Everything
+ * the engine can answer from its own state is answered first, cheapest and most
+ * semantic first, so an operation that could never run costs a handful of
+ * integer comparisons and never touches the filesystem, the helper or the
+ * device. Nothing later can retroactively excuse an earlier refusal: every gate
+ * returns, none of them merely records. */
+
+static ColiXdnaHard coli_xdna_engine_gates(ColiXdnaFamily family,
+ int fmt, int I, int O, int gs, int S,
+ const ColiXdnaArtifact **row_out){
+ /* 1 semantic family -- passed in by the call site, never inferred */
+ if(family != COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP)
+ return COLI_XDNA_HARD_FAMILY_UNSUPPORTED;
+
+ /* 2 logical M in the range this slice implements */
+ if(S < COLI_XDNA_I5_LOGICAL_M_MIN || S > COLI_XDNA_I5_LOGICAL_M_MAX)
+ return COLI_XDNA_HARD_M_OUT_OF_RANGE;
+
+ /* 3 K / N representable and positive */
+ if(I <= 0 || O <= 0) return COLI_XDNA_HARD_SHAPE_UNSUPPORTED;
+
+ /* 4 stored format */
+ if(fmt != 4) return COLI_XDNA_HARD_FORMAT_UNSUPPORTED;
+
+ /* 5 group size -- the converter is more permissive than the device */
+ if(gs != COLI_XDNA_QUALIFIED_GROUP_SIZE)
+ return COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED;
+
+ /* 6 a qualified artifact for exactly this family/bucket/shape/dtypes.
+ * The bucket is the artifact's M, which is where logical M is mapped. */
+ ColiXdnaRequest q;
+ q.family = family;
+ q.m = (unsigned)COLI_XDNA_I5_LOGICAL_M_MAX; /* the F3 M64 bucket */
+ q.k = (unsigned)I; q.n = (unsigned)O;
+ q.in_dtype = COLI_XDNA_DT_BF16;
+ q.weight_dtype = COLI_XDNA_DT_BF16;
+ q.out_dtype = COLI_XDNA_DT_F32;
+
+ const ColiXdnaArtifact *row = coli_xdna_registry_lookup(&q);
+ if(!row) return COLI_XDNA_HARD_SHAPE_UNSUPPORTED;
+
+ /* 7-9 qualification flags, artifact presence, artifact integrity. All
+ * three live behind coli_xdna_artifact_status, which fails closed. */
+ if(coli_xdna_artifact_status(&q, g_xdna_root[0] ? g_xdna_root : NULL)
+ != COLI_XDNA_STATIC_QUALIFIED)
+ return COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED;
+
+ *row_out = row;
+ return COLI_XDNA_HARD_ELIGIBLE;
+}
+
+/* Grow a staging buffer to at least `need` bytes. Transient operation state:
+ * reused across operations so a steady stream of the same shape allocates once,
+ * but carrying no validity and no authority of any kind. */
+static int coli_xdna_stage(void **base, size_t *cap, size_t need){
+ if(*cap >= need) return 1;
+ void *p = malloc(need);
+ if(!p) return 0;
+ free(*base);
+ *base = p; *cap = need;
+ return 1;
+}
+
+int coli_xdna_try_matmul(ColiXdnaFamily family,
+ ColiXdnaPrepared **slot,
+ int fmt, const unsigned char *q4, const float *scale,
+ int I, int O, int gs,
+ float *y, const float *x, int S)
+{
+ /* Gate 0: the internal control. Without it this function is inert, which is
+ * what keeps I5 from being an automatic scheduler. */
+ if(!g_xdna_force){ g_xdna_fallbacks++; g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED; return 0; }
+
+ if(!slot || !q4 || !scale || !y || !x){
+ g_xdna_last_hard = COLI_XDNA_HARD_SHAPE_UNSUPPORTED;
+ g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
+ g_xdna_fallbacks++; return 0;
+ }
+
+ const ColiXdnaArtifact *row = NULL;
+ ColiXdnaHard h = coli_xdna_engine_gates(family, fmt, I, O, gs, S, &row);
+ if(h != COLI_XDNA_HARD_ELIGIBLE){
+ g_xdna_last_hard = h; g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
+ g_xdna_fallbacks++; return 0;
+ }
+
+ /* 10 helper ABI. Checked before the weight is prepared: preparing 24 MiB
+ * for a lane that cannot execute would be pure waste. */
+ if(coli_xdna_binding() != COLI_XDNA_AVAILABLE){
+ g_xdna_last_hard = COLI_XDNA_HARD_HELPER_UNAVAILABLE;
+ g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
+ g_xdna_fallbacks++; return 0;
+ }
+
+ /* 11 prepared weight. Lazy: prepared on first use of this tensor and
+ * reused while it stays VALID for the same dimensions. There is no
+ * eviction or cache policy here -- that is a later slice's decision. */
+ if(!*slot){
+ *slot = coli_xdna_prepared_create();
+ if(!*slot){
+ g_xdna_last_hard = COLI_XDNA_HARD_PREPARED_INVALID;
+ g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED;
+ g_xdna_fallbacks++; return 0;
+ }
+ }
+ ColiXdnaPrepared *prep = *slot;
+ if(coli_xdna_prepared_state(prep) != COLI_XDNA_PREP_VALID
+ || coli_xdna_prepared_k(prep) != (unsigned)I
+ || coli_xdna_prepared_n(prep) != (unsigned)O){
+ if(coli_xdna_prepare_from_fmt4(prep, fmt, q4, scale, I, O, gs) != COLI_XDNA_PREP_OK){
+ g_xdna_last_hard = COLI_XDNA_HARD_PREPARED_INVALID;
+ g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED;
+ g_xdna_fallbacks++; return 0;
+ }
+ }
+ const void *wimg = coli_xdna_prepared_image(prep); /* NULL unless VALID */
+ if(!wimg){
+ g_xdna_last_hard = COLI_XDNA_HARD_PREPARED_INVALID;
+ g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED;
+ g_xdna_fallbacks++; return 0;
+ }
+
+ /* 12 alignment. The allocator guarantees it; checking anyway is cheap and
+ * turns a confusing XRT "insufficient free video memory" into a local,
+ * accurate refusal. */
+ if(!coli_xdna_pointer_alignment_ok(wimg)){
+ g_xdna_last_hard = COLI_XDNA_HARD_ALIGNMENT_INVALID;
+ g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
+ g_xdna_fallbacks++; return 0;
+ }
+
+ const unsigned am = row->artifact_m;
+
+ /* 13 device / runtime / artifact runtime object. Opened lazily, only now
+ * that every engine-side gate has passed. */
+ if(!g_lane.open || g_lane.m != am || g_lane.k != (unsigned)I || g_lane.n != (unsigned)O){
+ char xb[2048], ib[2048];
+ if(!coli_xdna_join(xb, sizeof xb, g_xdna_root, row->xclbin_name)
+ || !coli_xdna_join(ib, sizeof ib, g_xdna_root, row->insts_name)){
+ g_xdna_last_hard = COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE;
+ g_xdna_last_exec = COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED;
+ g_xdna_fallbacks++; return 0;
+ }
+ if(g_lane.open){ coli_xdna_helper_release_weight_call(); coli_xdna_helper_shutdown_call(); }
+ g_xdna_helper_calls++; g_xdna_device_opens++;
+ int rc = coli_xdna_helper_open_call(xb, ib, am, (unsigned)I, (unsigned)O);
+ if(rc != COLI_XDNA_H_OK){
+ g_xdna_last_hard = (rc == COLI_XDNA_H_E_DEVICE)
+ ? COLI_XDNA_HARD_DEVICE_UNAVAILABLE
+ : COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE;
+ g_xdna_last_exec = (rc == COLI_XDNA_H_E_DEVICE)
+ ? COLI_XDNA_EXEC_DEVICE_INIT_FAILED
+ : COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED;
+ g_xdna_fallbacks++; return 0;
+ }
+ g_lane.open = 1; g_lane.m = am; g_lane.k = (unsigned)I; g_lane.n = (unsigned)O;
+ g_lane.wrapped = NULL;
+ g_xdna_artifact_opens++;
+ }
+
+ /* 14 userptr wrap of the engine-owned prepared image. The helper wraps
+ * THIS memory; it never allocates a second copy of the weight. */
+ if(g_lane.wrapped != wimg){
+ g_xdna_helper_calls++; g_xdna_userptr_wraps++;
+ if(coli_xdna_helper_wrap_call((void *)wimg,
+ (uint64_t)coli_xdna_prepared_bytes(prep)) != COLI_XDNA_H_OK){
+ g_xdna_last_hard = COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE;
+ g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED;
+ g_xdna_fallbacks++; return 0;
+ }
+ g_lane.wrapped = wimg;
+ }
+
+ g_xdna_last_hard = COLI_XDNA_HARD_ELIGIBLE; /* FULL_XDNA_HARD_ELIGIBLE */
+
+ /* ---- activation staging -------------------------------------------------
+ * Caller activation is f32 [S, K] row-major. The artifact consumes BF16
+ * [artifact_m, K]. The conversion uses the SAME round-to-nearest-even rule
+ * as the weight, because both were qualified on that rule together.
+ *
+ * Rows S..artifact_m-1 are zeroed. Zero is the mathematically neutral row
+ * for a pure GEMM: it contributes a zero output row and cannot perturb any
+ * logical row, because C = A x B is row-independent. */
+ size_t abytes = (size_t)am * (size_t)I * 2u;
+ size_t obytes = (size_t)am * (size_t)O * 4u;
+ if(!coli_xdna_stage((void **)&g_lane.act, &g_lane.act_cap, abytes)
+ || !coli_xdna_stage((void **)&g_lane.out, &g_lane.out_cap, obytes)){
+ g_xdna_last_exec = COLI_XDNA_EXEC_ACTIVATION_FAILED;
+ g_xdna_fallbacks++; return 0;
+ }
+ for(size_t i = 0; i < (size_t)S * (size_t)I; i++)
+ g_lane.act[i] = coli_xdna_f2b(x[i]);
+ if((unsigned)S < am){
+ memset(g_lane.act + (size_t)S * (size_t)I, 0,
+ ((size_t)am - (size_t)S) * (size_t)I * 2u);
+ g_xdna_padded_ops++;
+ }
+ g_xdna_act_preps++;
+
+ /* ---- blocking execution -------------------------------------------------
+ * One operation, one dispatch, one wait. No worker thread, no queue, no
+ * second outstanding operation. */
+ g_xdna_helper_calls++;
+ int rc = coli_xdna_helper_execute_call(g_lane.act, (uint64_t)abytes,
+ g_lane.out, (uint64_t)obytes);
+ if(rc != COLI_XDNA_H_OK){
+ g_xdna_last_exec = (rc == COLI_XDNA_H_E_COMPLETION)
+ ? COLI_XDNA_EXEC_COMPLETION_FAILED
+ : COLI_XDNA_EXEC_EXECUTE_FAILED;
+ g_xdna_fallbacks++;
+ return 0; /* y untouched: the caller's path owns it */
+ }
+ g_xdna_dispatches++; g_xdna_completions++;
+
+ /* Only the logical rows become output. The padded rows are never copied and
+ * so can never reach anything downstream, whatever the device wrote there. */
+ memcpy(y, g_lane.out, (size_t)S * (size_t)O * 4u);
+ g_xdna_last_exec = COLI_XDNA_EXEC_OK;
+ return 1;
+}
+
+void coli_xdna_execution_shutdown(void){
+ if(g_lane.open){
+ coli_xdna_helper_release_weight_call();
+ coli_xdna_helper_shutdown_call();
+ }
+ g_lane.open = 0; g_lane.wrapped = NULL;
+ free(g_lane.act); g_lane.act = NULL; g_lane.act_cap = 0;
+ free(g_lane.out); g_lane.out = NULL; g_lane.out_cap = 0;
+}
diff --git a/c/backend_xdna.h b/c/backend_xdna.h
index c0c87352..10b320e7 100644
--- a/c/backend_xdna.h
+++ b/c/backend_xdna.h
@@ -29,8 +29,15 @@ extern "C" {
/* Host-side ABI generation. A helper reporting a different value is refused
* outright: there is no partial acceptance and no speculative forward
* compatibility, because neither could be tested here. Bump this when the
- * required entry-point set or any signature changes. */
-#define COLI_XDNA_ABI_VERSION 1u
+ * required entry-point set or any signature changes.
+ *
+ * Generation 2 (W2-N7-I5) is a DELIBERATE COMPATIBILITY BREAK. Generation 1
+ * required two entry points and could only answer "a helper exists"; generation
+ * 2 requires seven and can open a device, wrap a weight and execute. A
+ * generation-1 helper is refused outright rather than partially used: it cannot
+ * perform any of the new work, and silently binding it would produce a helper
+ * that reports availability and then cannot execute. */
+#define COLI_XDNA_ABI_VERSION 2u
/* Symbols the helper must export, all prefixed coli_xdna_helper_ so they can
* never collide with the host-side coli_xdna_ names in this header. */
@@ -320,9 +327,138 @@ int coli_xdna_pointer_alignment_ok(const void *p);
size_t coli_xdna_prepared_total_bytes(void);
int coli_xdna_prepared_live_objects(void);
+/* -- full hard eligibility and native execution ---------------------------
+ *
+ * I2 could only answer STATIC_ARTIFACT_QUALIFIED: does a trustworthy artifact
+ * exist? This is the first slice that can answer the whole question:
+ *
+ * CAN this operation safely run on XDNA, right now, on this machine?
+ *
+ * It still does NOT answer "should it". Hard eligibility and economic
+ * preference are separate decisions with separate owners, and nothing here
+ * consults, computes or caches a cost. Automatic selection belongs to a later
+ * slice; this one executes only when an internal test control asks it to. */
+
+typedef enum {
+ COLI_XDNA_HARD_ELIGIBLE = 0,
+ /* engine-side semantic gates, evaluated before anything is opened */
+ COLI_XDNA_HARD_FAMILY_UNSUPPORTED,
+ COLI_XDNA_HARD_M_OUT_OF_RANGE,
+ COLI_XDNA_HARD_SHAPE_UNSUPPORTED,
+ COLI_XDNA_HARD_FORMAT_UNSUPPORTED, /* stored tensor is not fmt=4 */
+ COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED, /* fmt=4 but gs is not the qualified 64 */
+ /* artifact gates */
+ COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED, /* any non-QUALIFIED static verdict */
+ /* representation gates */
+ COLI_XDNA_HARD_PREPARED_INVALID,
+ COLI_XDNA_HARD_ALIGNMENT_INVALID,
+ /* runtime gates */
+ COLI_XDNA_HARD_HELPER_UNAVAILABLE,
+ COLI_XDNA_HARD_DEVICE_UNAVAILABLE,
+ COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE,
+ COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE
+} ColiXdnaHard;
+
+const char *coli_xdna_hard_text(ColiXdnaHard h);
+
+/* The qualified source group size. I4's converter accepts any gs>0 as a generic
+ * representation transform; that breadth is a property of the CONVERTER, not a
+ * statement about what the device was qualified on. A PREPARED_VALID image
+ * built from another gs is a perfectly good host image and is still refused
+ * here, because no artifact was ever correctness-qualified against one. */
+#define COLI_XDNA_QUALIFIED_GROUP_SIZE 64
+
+/* The logical-M range this slice implements, against the F3 M64 artifact.
+ *
+ * Row padding is legitimate because C = A x B is row-independent: output row i
+ * depends only on input row i and on B. N6-A2-A1A qualified the strategy
+ * physically -- 13 logical M values against an unmodified fixed-M artifact, all
+ * bit-exact, with zeros written into every padded row. That argument is a
+ * property of PURE GEMM and must never be extended to attention, normalisation
+ * or routing.
+ *
+ * Row TILING (logical M above the bucket) is equally qualified as a strategy
+ * and deliberately NOT implemented here: the smallest first seam is one
+ * dispatch. Anything above the range declines to the current path. */
+#define COLI_XDNA_I5_LOGICAL_M_MIN 1
+#define COLI_XDNA_I5_LOGICAL_M_MAX 64
+
+typedef enum {
+ COLI_XDNA_EXEC_OK = 0,
+ COLI_XDNA_EXEC_DECLINED, /* not hard-eligible; not an error */
+ COLI_XDNA_EXEC_DEVICE_INIT_FAILED,
+ COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED,
+ COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED,
+ COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED,
+ COLI_XDNA_EXEC_ACTIVATION_FAILED,
+ COLI_XDNA_EXEC_EXECUTE_FAILED, /* dispatch refused or threw */
+ COLI_XDNA_EXEC_COMPLETION_FAILED /* dispatched, did not complete cleanly */
+} ColiXdnaExec;
+
+const char *coli_xdna_exec_text(ColiXdnaExec e);
+
+/* The production candidate, called from the GLM shared gate/up sites and
+ * nowhere else. Deliberately shaped like the existing optional-lane idiom
+ * (vk_matmul_qt): returns 1 when it has fully written y, 0 when the caller must
+ * run its current path.
+ *
+ * if(!coli_xdna_try_matmul(...)) matmul_qt(y, x, w, S);
+ *
+ * It NEVER calls matmul_qt itself, so there is no recursion, no double dispatch
+ * and no path on which a failure here suppresses the current path.
+ *
+ * `family` is passed explicitly by the call site. It is never inferred from the
+ * shape: two operations with identical M/K/N are different operations, and one
+ * may not inherit the other's qualification.
+ *
+ * `slot` is the engine-owned prepared-state handle (QT::xdna), created lazily on
+ * first use. The tensor's authoritative fmt=4 bytes are read and never modified.
+ *
+ * y is written ONLY after the helper reports successful completion, so a failure
+ * at any stage leaves the caller's output buffer untouched for matmul_qt to
+ * overwrite. */
+int coli_xdna_try_matmul(ColiXdnaFamily family,
+ ColiXdnaPrepared **slot,
+ int fmt, const unsigned char *q4, const float *scale,
+ int I, int O, int gs,
+ float *y, const float *x, int S);
+
+/* Release helper-owned runtime state and the transient staging buffers. Safe
+ * when nothing was ever initialised, and safe to repeat. */
+void coli_xdna_execution_shutdown(void);
+
/* Install a registry for tests; NULL restores the production table. */
void coli_xdna_test_set_registry(const ColiXdnaArtifact *rows, int count);
+/* -- internal execution controls ------------------------------------------
+ *
+ * These are the ONLY way to reach the native path, and they are deliberately
+ * test seams rather than user-facing controls: there is no --xdna flag, no
+ * COLI_XDNA environment variable and no automatic policy anywhere in this
+ * slice, so an ordinary build with a helper, a device and valid artifacts
+ * present still runs exactly the path it runs today.
+ *
+ * Forcing bypasses ECONOMIC preference only -- which does not exist yet, so
+ * today it bypasses nothing at all. It cannot bypass any hard gate: family,
+ * format, group size, shape, bucket, artifact integrity, qualification,
+ * prepared validity, alignment, helper ABI or device availability. */
+void coli_xdna_test_set_force_execution(int on);
+/* Where logical artifact names resolve. NULL (the default, and the only
+ * production value) means no artifact is reachable and every request declines.
+ * Artifact distribution is an open question this slice does not close. */
+void coli_xdna_test_set_artifact_root(const char *root);
+
+/* The last hard-eligibility verdict, for tests and diagnostics. */
+ColiXdnaHard coli_xdna_test_last_hard(void);
+/* The last execution verdict. */
+ColiXdnaExec coli_xdna_test_last_exec(void);
+int coli_xdna_test_dispatches(void); /* successful helper executions */
+int coli_xdna_test_completions(void); /* of those, cleanly completed */
+int coli_xdna_test_artifact_opens(void);
+int coli_xdna_test_activation_preparations(void);
+int coli_xdna_test_padded_operations(void);
+int coli_xdna_test_fallbacks(void); /* candidate returned 0 */
+
/* Deterministic mid-conversion fault injection, for tests only. 0 disables it;
* 1..99 fails after approximately that percentage of the conversion's rows.
* Named coli_xdna_test_* like every other seam here, and never referenced by
@@ -333,7 +469,9 @@ void coli_xdna_test_set_convert_fail_pct(int pct);
* a failed conversion left behind. Production code uses
* coli_xdna_prepared_image(), which returns NULL unless the image is VALID. */
const void *coli_xdna_prepared_image_unchecked(const ColiXdnaPrepared *p);
-/* All four must stay 0 for the whole of this slice. */
+/* Device opens, helper calls and userptr wraps became reachable in W2-N7-I5 and
+ * are 0 only on paths that never reach the native lane. Conversions became
+ * reachable in I4. */
int coli_xdna_test_device_opens(void);
int coli_xdna_test_helper_calls(void);
int coli_xdna_test_userptr_wraps(void);
diff --git a/c/backend_xdna_helper.cpp b/c/backend_xdna_helper.cpp
new file mode 100644
index 00000000..fb65b1e3
--- /dev/null
+++ b/c/backend_xdna_helper.cpp
@@ -0,0 +1,249 @@
+// Optional AMD XDNA2 helper -- the ONLY component in Colibri that links XRT.
+//
+// This file is NOT part of an ordinary Colibri build. It compiles to
+// coli_xdna.dll, which c/backend_xdna.c loads at runtime and may refuse. A
+// machine with no NPU, no XRT and no helper is a normal machine; the engine
+// simply keeps doing what it does today.
+//
+// WHAT THIS FILE OWNS
+// XRT runtime objects, the device, the hardware context, the kernel, the
+// artifact program, the device-visible activation and output buffers, the
+// userptr wrapper around the caller's prepared weight, the blocking submit
+// and wait, and the release of all of the above.
+//
+// WHAT THIS FILE MUST NEVER OWN
+// Anything that requires knowing what the operation MEANS. It is not told the
+// semantic family, the tensor, the expert, the layer, the model, the routing
+// decision, the quantisation format or the group size, and it makes no
+// selection, no fallback decision and no economic judgement. It is handed an
+// already-selected, already-integrity-verified artifact and an
+// already-prepared buffer, and it executes.
+//
+// Every export catches std::exception AND unknown exceptions and converts them
+// to a status code. No exception may cross the C ABI.
+//
+// Build: see GPU_BACKENDS.md. Requires the XRT SDK headers and
+// xrt_coreutil.lib; MSVC needs /Zc:__cplusplus for xrt/detail/any.h to select
+// <any> instead of its boost fallback.
+
+#include <cstdint>
+#include <cstring>
+#include <cstdio>
+#include <string>
+#include <vector>
+#include <fstream>
+
+#include "xrt/xrt_device.h"
+#include "xrt/xrt_kernel.h"
+#include "xrt/xrt_bo.h"
+#include "xrt/experimental/xrt_xclbin.h"
+#include "xrt/experimental/xrt_ext.h"
+
+#define COLI_XDNA_HELPER_API extern "C" __declspec(dllexport)
+
+// Must equal COLI_XDNA_ABI_VERSION in c/backend_xdna.h. Exact equality: the
+// host refuses any other generation outright.
+#define COLI_XDNA_HELPER_ABI 2u
+
+// Must match the enum in backend_xdna.c. Kept as plain ints so the boundary
+// carries no type of its own.
+enum {
+ H_OK = 0,
+ H_E_DEVICE = -1,
+ H_E_ARTIFACT = -2,
+ H_E_NOT_OPEN = -3,
+ H_E_WRAP = -4,
+ H_E_SIZE = -5,
+ H_E_DISPATCH = -6,
+ H_E_COMPLETION = -7,
+ H_E_EXCEPTION = -8
+};
+
+namespace {
+
+struct Lane {
+ bool open = false;
+ xrt::device *dev = nullptr;
+ xrt::xclbin *xcl = nullptr;
+ xrt::hw_context *ctx = nullptr;
+ xrt::kernel *ker = nullptr;
+ xrt::bo *bo_i = nullptr; // instruction stream
+ xrt::bo *bo_a = nullptr; // activation, device-visible
+ xrt::bo *bo_c = nullptr; // output, device-visible
+ xrt::ext::bo *bo_w = nullptr; // userptr wrapper over CALLER memory
+ std::vector<uint32_t> instr;
+ size_t m = 0, k = 0, n = 0;
+};
+
+Lane g;
+char g_err[512] = {0};
+
+void set_err(const char *s) { std::snprintf(g_err, sizeof g_err, "%s", s ? s : ""); }
+
+std::vector<uint32_t> load_instr(const char *p) {
+ std::ifstream f(p, std::ios::binary | std::ios::ate);
+ if (!f) return {};
+ std::streamsize n = f.tellg();
+ if (n <= 0 || (n % 4) != 0) return {};
+ f.seekg(0);
+ std::vector<uint32_t> v(static_cast<size_t>(n) / 4);
+ f.read(reinterpret_cast<char *>(v.data()), n);
+ if (!f) return {};
+ return v;
+}
+
+// Release everything, in reverse construction order, exactly once each. Safe
+// when nothing was ever constructed, and safe to repeat: every pointer is
+// nulled as it is deleted.
+void teardown() {
+ delete g.bo_w; g.bo_w = nullptr; // wrapper first: it borrows caller memory
+ delete g.bo_c; g.bo_c = nullptr;
+ delete g.bo_a; g.bo_a = nullptr;
+ delete g.bo_i; g.bo_i = nullptr;
+ delete g.ker; g.ker = nullptr;
+ delete g.ctx; g.ctx = nullptr;
+ delete g.xcl; g.xcl = nullptr;
+ delete g.dev; g.dev = nullptr;
+ g.instr.clear();
+ g.m = g.k = g.n = 0;
+ g.open = false;
+}
+
+} // namespace
+
+COLI_XDNA_HELPER_API unsigned int coli_xdna_helper_abi_version(void) {
+ return COLI_XDNA_HELPER_ABI;
+}
+
+COLI_XDNA_HELPER_API const char *coli_xdna_helper_last_error(void) {
+ return g_err;
+}
+
+// Open the device and the already-selected artifact for one (m, k, n).
+//
+// The host has already verified that these bytes hash to the values the
+// research programme qualified; this function does not re-decide which artifact
+// to use, and could not, since it is never told what the operation is.
+COLI_XDNA_HELPER_API int coli_xdna_helper_open(const char *xclbin, const char *insts,
+ uint32_t m, uint32_t k, uint32_t n) {
+ if (!xclbin || !insts || m == 0 || k == 0 || n == 0) { set_err("bad open arguments"); return H_E_SIZE; }
+ try {
+ teardown(); // a re-open never layers onto old state
+
+ g.instr = load_instr(insts);
+ if (g.instr.empty()) { set_err("instruction stream missing or malformed"); return H_E_ARTIFACT; }
+
+ g.dev = new xrt::device(static_cast<unsigned int>(0));
+ g.xcl = new xrt::xclbin(std::string(xclbin));
+ g.dev->register_xclbin(*g.xcl);
+ g.ctx = new xrt::hw_context(*g.dev, g.xcl->get_uuid());
+ g.ker = new xrt::kernel(*g.ctx, "MLIR_AIE");
+
+ g.m = m; g.k = k; g.n = n;
+ g.bo_i = new xrt::bo(*g.dev, g.instr.size() * 4, XCL_BO_FLAGS_CACHEABLE, g.ker->group_id(1));
+ g.bo_a = new xrt::bo(*g.dev, static_cast<size_t>(m) * k * 2, XRT_BO_FLAGS_HOST_ONLY, g.ker->group_id(3));
+ g.bo_c = new xrt::bo(*g.dev, static_cast<size_t>(m) * n * 4, XRT_BO_FLAGS_HOST_ONLY, g.ker->group_id(5));
+
+ std::memcpy(g.bo_i->map<void *>(), g.instr.data(), g.instr.size() * 4);
+ g.bo_i->sync(XCL_BO_SYNC_BO_TO_DEVICE);
+
+ g.open = true;
+ set_err("");
+ return H_OK;
+ } catch (const std::exception &e) {
+ set_err(e.what()); teardown();
+ // A device that will not open and an artifact that will not load are
+ // different failures for the host: one disables the lane, the other
+ // only this shape. XRT reports both as exceptions, so this cannot be
+ // distinguished perfectly here; the host treats an open failure as
+ // artifact-scoped, which is the narrower and therefore safer claim.
+ return H_E_ARTIFACT;
+ } catch (...) {
+ set_err("unknown exception in open"); teardown();
+ return H_E_EXCEPTION;
+ }
+}
+
+// Wrap the caller's prepared BF16 weight image through the qualified XRT
+// userptr path. No copy and no allocation of a second weight image: the device
+// reads the engine's own buffer.
+//
+// The pointer must be 4096-byte aligned. The host guarantees and re-checks that
+// before calling; a misaligned pointer fails here with an XRT message about
+// video memory, which points at entirely the wrong subsystem.
+COLI_XDNA_HELPER_API int coli_xdna_helper_wrap_weight(void *bf16, uint64_t bytes) {
+ if (!g.open) { set_err("wrap before open"); return H_E_NOT_OPEN; }
+ if (!bf16 || bytes == 0) { set_err("bad wrap arguments"); return H_E_SIZE; }
+ if (bytes != static_cast<uint64_t>(g.k) * g.n * 2) { set_err("weight size does not match K*N*2"); return H_E_SIZE; }
+ try {
+ delete g.bo_w; g.bo_w = nullptr;
+ g.bo_w = new xrt::ext::bo(*g.dev, bf16, static_cast<size_t>(bytes));
+ g.bo_w->sync(XCL_BO_SYNC_BO_TO_DEVICE);
+ set_err("");
+ return H_OK;
+ } catch (const std::exception &e) {
+ set_err(e.what()); delete g.bo_w; g.bo_w = nullptr; return H_E_WRAP;
+ } catch (...) {
+ set_err("unknown exception in wrap"); delete g.bo_w; g.bo_w = nullptr; return H_E_EXCEPTION;
+ }
+}
+
+// One blocking operation: stage the activation, submit, wait, read the output
+// back. No worker thread, no queue, and never more than one operation
+// outstanding -- the serialized V1 posture the concurrency research froze.
+//
+// c_f32 receives all artifact-M rows. Deciding which of them are logical output
+// is the host's job, and the host copies only those.
+COLI_XDNA_HELPER_API int coli_xdna_helper_execute(const void *a_bf16, uint64_t a_bytes,
+ void *c_f32, uint64_t c_bytes) {
+ if (!g.open || !g.bo_w) { set_err("execute before open/wrap"); return H_E_NOT_OPEN; }
+ if (!a_bf16 || !c_f32) { set_err("null execute buffer"); return H_E_SIZE; }
+ if (a_bytes != static_cast<uint64_t>(g.m) * g.k * 2) { set_err("activation size mismatch"); return H_E_SIZE; }
+ if (c_bytes != static_cast<uint64_t>(g.m) * g.n * 4) { set_err("output size mismatch"); return H_E_SIZE; }
+ try {
+ std::memcpy(g.bo_a->map<void *>(), a_bf16, static_cast<size_t>(a_bytes));
+ g.bo_a->sync(XCL_BO_SYNC_BO_TO_DEVICE);
+
+ auto r = xrt::run(*g.ker);
+ r.set_arg(0, static_cast<unsigned int>(3));
+ r.set_arg(1, *g.bo_i);
+ r.set_arg(2, g.instr.size());
+ r.set_arg(3, *g.bo_a);
+ // MUST bind through xrt::bo&. Binding an xrt::ext::bo by its own type
+ // selects the scalar set_arg overload, which the runtime accepts and
+ // then reports ERT_CMD_STATE_COMPLETED while every output is NaN.
+ xrt::bo &wref = *g.bo_w;
+ r.set_arg(4, wref);
+ r.set_arg(5, *g.bo_c);
+
+ r.start();
+ auto st = r.wait();
+ if (st != ERT_CMD_STATE_COMPLETED) {
+ set_err("command did not complete");
+ return H_E_COMPLETION;
+ }
+ g.bo_c->sync(XCL_BO_SYNC_BO_FROM_DEVICE);
+ std::memcpy(c_f32, g.bo_c->map<const void *>(), static_cast<size_t>(c_bytes));
+ set_err("");
+ return H_OK;
+ } catch (const std::exception &e) {
+ set_err(e.what()); return H_E_DISPATCH;
+ } catch (...) {
+ set_err("unknown exception in execute"); return H_E_EXCEPTION;
+ }
+}
+
+// Drop the userptr wrapper while keeping the artifact runtime open. The caller
+// memory it borrowed is untouched: the engine owns it and always did.
+COLI_XDNA_HELPER_API int coli_xdna_helper_release_weight(void) {
+ try {
+ delete g.bo_w; g.bo_w = nullptr;
+ return H_OK;
+ } catch (...) {
+ g.bo_w = nullptr; set_err("unknown exception in release_weight"); return H_E_EXCEPTION;
+ }
+}
+
+COLI_XDNA_HELPER_API void coli_xdna_helper_shutdown(void) {
+ try { teardown(); } catch (...) { /* nothing useful remains to report */ }
+}
diff --git a/c/colibri.c b/c/colibri.c
index 4aad3b23..454b4c8c 100644
--- a/c/colibri.c
+++ b/c/colibri.c
@@ -6243,10 +6243,29 @@ static void moe(Model *m, Layer *l, int layer, float *x, int S, float *out, int
if(!shared_cuda){
#endif
sg=falloc((int64_t)S*sI);su=falloc((int64_t)S*sI);
+ /* Optional XDNA2 lane, gate/up only. Same idiom as vk_matmul_qt: 0 means
+ * "not handled, run the current path", and the candidate never calls
+ * matmul_qt itself, so there is no recursion and no double dispatch.
+ * The semantic family is passed EXPLICITLY -- it is never inferred from
+ * M/K/N, so an unrelated operation of the same shape cannot inherit this
+ * one's qualification. sh_down is deliberately absent below: its
+ * orientation (I=sI, O=D) is not what the qualified F3 artifact
+ * computes. Inert without the internal test control; there is no public
+ * switch and no automatic policy in this slice. */
+#ifdef COLI_XDNA
+ if(!coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &l->sh_gate.xdna,
+ l->sh_gate.fmt, l->sh_gate.q4, l->sh_gate.s,
+ l->sh_gate.I, l->sh_gate.O, l->sh_gate.gs, sg, x, S))
+#endif
#ifdef COLI_VULKAN
if(!vk_matmul_qt(&l->sh_gate, sg, x, S))
#endif
matmul_qt(sg, x, &l->sh_gate, S);
+#ifdef COLI_XDNA
+ if(!coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &l->sh_up.xdna,
+ l->sh_up.fmt, l->sh_up.q4, l->sh_up.s,
+ l->sh_up.I, l->sh_up.O, l->sh_up.gs, su, x, S))
+#endif
#ifdef COLI_VULKAN
if(!vk_matmul_qt(&l->sh_up, su, x, S))
#endif
diff --git a/c/tests/test_backend_loader.py b/c/tests/test_backend_loader.py
index 3a5cbab6..b19ddacf 100644
--- a/c/tests/test_backend_loader.py
+++ b/c/tests/test_backend_loader.py
@@ -2437,13 +2437,31 @@ _XDNA_FAKE_HELPER_C = r'''
#include <windows.h>
#ifndef FAKE_ABI_VERSION
-#define FAKE_ABI_VERSION 1u
+#define FAKE_ABI_VERSION 2u
#endif
__declspec(dllexport) unsigned int coli_xdna_helper_abi_version(void){
return (unsigned int)FAKE_ABI_VERSION;
}
+/* ABI generation 2 requires the whole execution set, not just a handshake. None
+ * of these does anything: the contract under test is the BINDING. */
+__declspec(dllexport) int coli_xdna_helper_open(const char *x, const char *i,
+ unsigned m, unsigned k, unsigned n){
+ (void)x;(void)i;(void)m;(void)k;(void)n; return 0;
+}
+__declspec(dllexport) int coli_xdna_helper_wrap_weight(void *p, unsigned long long b){
+ (void)p;(void)b; return 0;
+}
+#ifndef FAKE_OMIT_EXECUTE
+__declspec(dllexport) int coli_xdna_helper_execute(const void *a, unsigned long long ab,
+ void *c, unsigned long long cb){
+ (void)a;(void)ab;(void)c;(void)cb; return 0;
+}
+#endif
+__declspec(dllexport) int coli_xdna_helper_release_weight(void){ return 0; }
+__declspec(dllexport) const char *coli_xdna_helper_last_error(void){ return ""; }
+
#ifndef FAKE_OMIT_SHUTDOWN
__declspec(dllexport) void coli_xdna_helper_shutdown(void){ }
#endif
@@ -2479,8 +2497,8 @@ _XDNA_HARNESS_C = r'''
static void report(const char *tag){
ColiXdnaBinding s = coli_xdna_binding();
- printf("%s=%s attempts=%d\n", tag, coli_xdna_binding_text(s),
- coli_xdna_test_load_attempts());
+ printf("%s=%s attempts=%d bound=%d\n", tag, coli_xdna_binding_text(s),
+ coli_xdna_test_load_attempts(), coli_xdna_test_entry_points_bound());
}
int main(int argc, char **argv){
@@ -2600,12 +2618,28 @@ class XdnaOptionalBindingTest(unittest.TestCase):
gcc(["-O0", "-shared", "-DFAKE_ABI_VERSION=999u", str(helper_src),
"-o", str(cls.bad_abi)], "building the wrong-ABI synthetic helper")
+ # The previous ABI generation. W2-N7-I5 bumped the host to generation 2,
+ # which is a deliberate compatibility break: a generation-1 helper
+ # exports two of the seven entry points now required and cannot execute
+ # anything, so it must be refused outright rather than partially bound.
+ cls.abi1 = root / "abi1" / _XDNA_HELPER_BASENAME
+ cls.abi1.parent.mkdir(parents=True, exist_ok=True)
+ gcc(["-O0", "-shared", "-DFAKE_ABI_VERSION=1u", str(helper_src),
+ "-o", str(cls.abi1)], "building the previous-generation synthetic helper")
+
# Missing a required entry point.
cls.incomplete = root / "incomplete" / _XDNA_HELPER_BASENAME
cls.incomplete.parent.mkdir(parents=True, exist_ok=True)
gcc(["-O0", "-shared", "-DFAKE_OMIT_SHUTDOWN", str(helper_src),
"-o", str(cls.incomplete)], "building the incomplete synthetic helper")
+ # Missing one of the entry points ABI 2 introduced. Binding is
+ # all-or-nothing across the WHOLE set, not just the parts it inherited.
+ cls.no_execute = root / "no_execute" / _XDNA_HELPER_BASENAME
+ cls.no_execute.parent.mkdir(parents=True, exist_ok=True)
+ gcc(["-O0", "-shared", "-DFAKE_OMIT_EXECUTE", str(helper_src),
+ "-o", str(cls.no_execute)], "building the execute-less synthetic helper")
+
# Present but unloadable: imports a dependency that is then deleted.
cls.unloadable = root / "unloadable" / _XDNA_HELPER_BASENAME
cls.unloadable.parent.mkdir(parents=True, exist_ok=True)
@@ -2662,6 +2696,21 @@ class XdnaOptionalBindingTest(unittest.TestCase):
out = self._run("probe", self.bad_abi)
self.assertEqual(self._state(out), "ABI_INCOMPATIBLE")
+ def test_previous_abi_generation_is_rejected(self):
+ # The I1 -> I5 generation bump is a compatibility break by design. A
+ # stale generation-1 helper must never become callable under the
+ # generation-2 contract, however many symbol names it happens to share.
+ out = self._run("probe", self.abi1)
+ self.assertEqual(self._state(out), "ABI_INCOMPATIBLE")
+ self.assertIn("bound=0", out)
+
+ def test_missing_execute_entry_point_is_rejected(self):
+ # All-or-nothing applies to the entry points ABI 2 added, not only to
+ # the ones it inherited from ABI 1.
+ out = self._run("probe", self.no_execute)
+ self.assertEqual(self._state(out), "SYMBOL_INCOMPLETE")
+ self.assertIn("bound=0", out)
+
def test_missing_required_entry_point_is_rejected(self):
out = self._run("probe", self.incomplete)
self.assertEqual(self._state(out), "SYMBOL_INCOMPLETE")
@@ -2692,7 +2741,7 @@ class XdnaOptionalBindingTest(unittest.TestCase):
out = self._run("repeat", path, 8)
# attempts is reported on the same line as state=
line = [l for l in out.splitlines() if l.startswith("state=")][0]
- attempts = int(line.split("attempts=")[1])
+ attempts = int(line.split("attempts=")[1].split(" ")[0])
self.assertLessEqual(attempts, 1,
"loader re-probed %d times: %s" % (attempts, out))
diff --git a/c/tests/test_xdna_execution.c b/c/tests/test_xdna_execution.c
new file mode 100644
index 00000000..7c1585f6
--- /dev/null
+++ b/c/tests/test_xdna_execution.c
@@ -0,0 +1,507 @@
+/* W2-N7-I5 -- XDNA execution seam, qualified WITHOUT hardware.
+ *
+ * This is the owner of the execution control flow: gate order, artifact open,
+ * userptr wrap, activation staging and padding, logical-row extraction, failure
+ * classification, and the guarantee that every non-success path falls back to
+ * the caller's current operation with its output buffer untouched.
+ *
+ * It runs against a synthetic ABI-2 helper (tests/xdna_fake_helper.c) that
+ * contains no XRT and computes the GEMM on the CPU, so a machine with no NPU
+ * qualifies all of it. The NUMERICS of real device execution are qualified
+ * separately and explicitly on hardware; nothing here stands in for that.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <math.h>
+
+#include "../backend_xdna.c"
+
+static int g_fail = 0;
+static void ck(int cond, const char *what){
+ if(!cond){ printf(" FAIL %s\n", what); g_fail = 1; }
+ else printf(" ok %s\n", what);
+}
+
+/* ---- fixtures ----------------------------------------------------------- */
+
+#define TK 256 /* K -- small so the CPU GEMM in the fake stays quick */
+#define TN 128 /* N */
+#define TM 64 /* artifact M, the bucket this slice implements */
+
+static char g_root[1024];
+static char g_helper[1024], g_helper_abi1[1024], g_helper_partial[1024];
+static char g_xclbin[2048], g_insts[2048];
+static char g_xhex[65], g_ihex[65];
+static ColiXdnaArtifact g_test_rows[1];
+
+static void hexify(const unsigned char h[32], char out[65]){
+ static const char *H = "0123456789abcdef";
+ for(int i = 0; i < 32; i++){ out[i*2] = H[h[i]>>4]; out[i*2+1] = H[h[i]&15]; }
+ out[64] = '\0';
+}
+
+static int write_blob(const char *path, unsigned seed, size_t n){
+ FILE *f = fopen(path, "wb");
+ if(!f) return 0;
+ unsigned st = seed;
+ for(size_t i = 0; i < n; i++){ st = st*1664525u+1013904223u; fputc((int)(st>>24) & 0xFF, f); }
+ fclose(f);
+ return 1;
+}
+
+/* Build a registry whose hashes are the REAL hashes of the fixture files, so the
+ * integrity gate is exercised for what it is rather than stubbed out. */
+static void build_registry(void){
+ unsigned char h[32];
+ if(!coli_xdna_sha256_file(g_xclbin, h)){ printf(" FAIL fixture xclbin unreadable\n"); g_fail=1; return; }
+ hexify(h, g_xhex);
+ if(!coli_xdna_sha256_file(g_insts, h)){ printf(" FAIL fixture insts unreadable\n"); g_fail=1; return; }
+ hexify(h, g_ihex);
+
+ g_test_rows[0].family = COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP;
+ g_test_rows[0].research_family = "F3";
+ g_test_rows[0].artifact_m = TM;
+ g_test_rows[0].k = TK; g_test_rows[0].n = TN;
+ g_test_rows[0].in_dtype = COLI_XDNA_DT_BF16;
+ g_test_rows[0].weight_dtype = COLI_XDNA_DT_BF16;
+ g_test_rows[0].out_dtype = COLI_XDNA_DT_F32;
+ g_test_rows[0].target = COLI_XDNA_TARGET_XDNA2;
+ g_test_rows[0].xclbin_name = "fake.xclbin";
+ g_test_rows[0].xclbin_sha256 = g_xhex;
+ g_test_rows[0].insts_name = "fake_insts.bin";
+ g_test_rows[0].insts_sha256 = g_ihex;
+ g_test_rows[0].runtime_weight_qualified = 1;
+ g_test_rows[0].correctness_qualified = 1;
+ g_test_rows[0].userptr_qualified = 1;
+ g_test_rows[0].structural_qualified = 1;
+ coli_xdna_test_set_registry(g_test_rows, 1);
+}
+
+/* A deterministic fmt4 tensor with I=K, O=N. */
+typedef struct { unsigned char *q4; float *s; int I, O, gs; } Fmt4;
+
+static void fmt4_make(Fmt4 *t, int I, int O, int gs, unsigned seed){
+ int rb = (I+1)/2, ng = (I+gs-1)/gs;
+ t->I = I; t->O = O; t->gs = gs;
+ t->q4 = (unsigned char *)malloc((size_t)O*rb);
+ t->s = (float *)malloc((size_t)O*ng*sizeof(float));
+ unsigned st = seed;
+ for(size_t i = 0; i < (size_t)O*rb; i++){ st = st*1664525u+1013904223u; t->q4[i] = (unsigned char)(st>>24); }
+ for(size_t i = 0; i < (size_t)O*ng; i++){ st = st*1664525u+1013904223u;
+ t->s[i] = 0.25f * (float)(1 + ((st>>26)&3)); }
+}
+static void fmt4_free(Fmt4 *t){ free(t->q4); free(t->s); t->q4=NULL; t->s=NULL; }
+
+static float *mk_x(int S, int K, unsigned seed){
+ float *x = (float *)malloc((size_t)S*K*sizeof(float));
+ unsigned st = seed;
+ for(size_t i = 0; i < (size_t)S*K; i++){ st = st*1664525u+1013904223u;
+ x[i] = (float)((int)((st>>20)&255) - 128) * 0.0078125f; }
+ return x;
+}
+
+static float b2f_t(unsigned short b){
+ unsigned int u = (unsigned int)b << 16; float f; memcpy(&f,&u,4); return f;
+}
+
+/* Reference for the LAYOUT contract: BF16 activation x BF16 prepared weight,
+ * float accumulation in k order -- exactly what the fake computes. This checks
+ * that the right rows went in, the right rows came out, and nothing from the
+ * padded region leaked; it is not a claim about device arithmetic. */
+static void oracle(float *out, const float *x, const unsigned short *W,
+ int S, int K, int N){
+ for(int i = 0; i < S; i++)
+ for(int j = 0; j < N; j++){
+ float acc = 0.0f;
+ for(int k = 0; k < K; k++)
+ acc += b2f_t(coli_xdna_f2b(x[(size_t)i*K+k])) * b2f_t(W[(size_t)k*N+j]);
+ out[(size_t)i*N+j] = acc;
+ }
+}
+
+#define POISON (-123456.0f)
+static void poison(float *y, size_t n){ for(size_t i=0;i<n;i++) y[i]=POISON; }
+static int all_poison(const float *y, size_t n){
+ for(size_t i=0;i<n;i++) if(y[i]!=POISON) return 0; return 1;
+}
+
+/* Reset every lane-visible counter and state between groups. */
+static void lane_reset(void){
+ coli_xdna_execution_shutdown();
+ coli_xdna_test_set_force_execution(0);
+ g_xdna_dispatches = g_xdna_completions = g_xdna_artifact_opens = 0;
+ g_xdna_act_preps = g_xdna_padded_ops = g_xdna_fallbacks = 0;
+ g_xdna_device_opens = g_xdna_helper_calls = g_xdna_userptr_wraps = 0;
+}
+
+/* Rebind the loader to a helper path and clear its sticky verdict. */
+static void use_helper(const char *path){
+ lane_reset();
+ coli_xdna_test_set_helper_path(path);
+}
+
+int main(int argc, char **argv){
+ /* Locate fixtures beside this executable's directory. */
+ char dir[512]; snprintf(dir, sizeof dir, "%s", argc>0?argv[0]:"tests/x");
+ char *sl = strrchr(dir,'\\'); char *fw = strrchr(dir,'/');
+ if(fw && (!sl || fw>sl)) sl = fw;
+ if(sl) *sl = '\0'; else snprintf(dir,sizeof dir,".");
+
+ snprintf(g_root, sizeof g_root, "%s/xdna_exec_fixtures", dir);
+ snprintf(g_helper, sizeof g_helper, "%s/xdna_fake_helper.dll", dir);
+ snprintf(g_helper_abi1, sizeof g_helper_abi1, "%s/xdna_fake_helper_abi1.dll", dir);
+ snprintf(g_helper_partial, sizeof g_helper_partial, "%s/xdna_fake_helper_partial.dll", dir);
+#ifdef _WIN32
+ { char cmd[1200]; snprintf(cmd,sizeof cmd,"mkdir \"%s\" 2>nul", g_root);
+ for(char *p=cmd;*p;p++) if(*p=='/') *p='\\';
+ if(system(cmd)){} }
+#else
+ { char cmd[1200]; snprintf(cmd,sizeof cmd,"mkdir -p \"%s\"", g_root); if(system(cmd)){} }
+#endif
+ snprintf(g_xclbin, sizeof g_xclbin, "%s/fake.xclbin", g_root);
+ snprintf(g_insts, sizeof g_insts, "%s/fake_insts.bin", g_root);
+ if(!write_blob(g_xclbin, 11u, 4096) || !write_blob(g_insts, 22u, 1024)){
+ printf("FAIL could not create fixtures under %s\n", g_root); return 1; }
+ build_registry();
+ if(g_fail) return 1;
+
+ Fmt4 W; fmt4_make(&W, TK, TN, 64, 7u);
+ Fmt4 W2; fmt4_make(&W2, TK, TN, 64, 999u); /* a genuinely different weight */
+
+ /* ================================================================== */
+ printf("default behaviour: the seam is inert without the internal control\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ ColiXdnaPrepared *slot = NULL;
+ float *x = mk_x(TM, TK, 1u);
+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &slot,
+ 4, W.q4, W.s, TK, TN, 64, y, x, TM);
+ ck(handled == 0, "helper, artifact and device all available -- still not handled");
+ ck(coli_xdna_test_dispatches() == 0, "and zero dispatches");
+ ck(all_poison(y,(size_t)TM*TN), "caller output untouched");
+ coli_xdna_prepared_release(&slot); free(x); free(y);
+ }
+
+ /* ================================================================== */
+ printf("forced execution: the full path end to end\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ ColiXdnaPrepared *slot = NULL;
+ float *x = mk_x(TM, TK, 2u);
+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &slot,
+ 4, W.q4, W.s, TK, TN, 64, y, x, TM);
+ ck(handled == 1, "handled");
+ ck(coli_xdna_test_last_hard() == COLI_XDNA_HARD_ELIGIBLE, "FULL_XDNA_HARD_ELIGIBLE");
+ ck(coli_xdna_test_last_exec() == COLI_XDNA_EXEC_OK, "exec OK");
+ ck(coli_xdna_test_dispatches() == 1, "one dispatch");
+ ck(coli_xdna_test_completions() == 1, "one completion");
+ ck(coli_xdna_test_userptr_wraps() == 1, "one userptr wrap");
+ ck(coli_xdna_test_artifact_opens() == 1, "one artifact open");
+ ck(coli_xdna_prepared_state(slot) == COLI_XDNA_PREP_VALID, "weight left VALID");
+ ck(coli_xdna_pointer_alignment_ok(coli_xdna_prepared_image(slot)),
+ "prepared image 4096-aligned");
+
+ const unsigned short *Wi = (const unsigned short *)coli_xdna_prepared_image(slot);
+ float *ref = (float *)malloc((size_t)TM*TN*4);
+ oracle(ref, x, Wi, TM, TK, TN);
+ size_t mism = 0;
+ for(size_t i = 0; i < (size_t)TM*TN; i++) if(y[i] != ref[i]) mism++;
+ ck(mism == 0, "every output element matches the layout reference");
+ free(ref); coli_xdna_prepared_release(&slot); free(x); free(y);
+ }
+
+ /* ================================================================== */
+ printf("logical-M padding inside the qualified range\n");
+ {
+ int cases[] = { 1, 2, 7, 31, 32, 33, 63, 64 };
+ for(size_t c = 0; c < sizeof cases/sizeof cases[0]; c++){
+ int S = cases[c];
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ ColiXdnaPrepared *slot = NULL;
+ float *x = mk_x(S, TK, 100u + (unsigned)S);
+ float *y = (float *)malloc((size_t)S*TN*4); poison(y,(size_t)S*TN);
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &slot,
+ 4, W.q4, W.s, TK, TN, 64, y, x, S);
+ char msg[128];
+ snprintf(msg,sizeof msg,"S=%d handled",S); ck(handled==1,msg);
+ snprintf(msg,sizeof msg,"S=%d padded_ops=%d",S,coli_xdna_test_padded_operations());
+ ck(coli_xdna_test_padded_operations() == (S<TM?1:0), msg);
+
+ const unsigned short *Wi = (const unsigned short *)coli_xdna_prepared_image(slot);
+ float *ref = (float *)malloc((size_t)S*TN*4);
+ oracle(ref, x, Wi, S, TK, TN);
+ size_t mism = 0;
+ for(size_t i = 0; i < (size_t)S*TN; i++) if(y[i] != ref[i]) mism++;
+ snprintf(msg,sizeof msg,"S=%d all %d logical rows exact, no padded row exposed",S,S);
+ ck(mism == 0, msg);
+ free(ref); coli_xdna_prepared_release(&slot); free(x); free(y);
+ }
+ }
+
+ /* ================================================================== */
+ printf("negative controls -- none of these may ever dispatch\n");
+ {
+ struct { const char *what; ColiXdnaFamily fam; int fmt, I, O, gs, S; ColiXdnaHard want; } cases[] = {
+ { "wrong semantic family (same shape)", COLI_XDNA_FAMILY_NONE, 4, TK, TN, 64, TM,
+ COLI_XDNA_HARD_FAMILY_UNSUPPORTED },
+ { "sh_down orientation (K/N transposed)", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TN, TK, 64, TM,
+ COLI_XDNA_HARD_SHAPE_UNSUPPORTED },
+ { "gs != 64 (preparable but not qualified)", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TK, TN, 32, TM,
+ COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED },
+ { "fmt != 4", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 2, TK, TN, 64, TM,
+ COLI_XDNA_HARD_FORMAT_UNSUPPORTED },
+ { "logical M above the qualified range", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TK, TN, 64, TM+1,
+ COLI_XDNA_HARD_M_OUT_OF_RANGE },
+ { "logical M below the qualified range", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TK, TN, 64, 0,
+ COLI_XDNA_HARD_M_OUT_OF_RANGE },
+ { "unqualified shape", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4, TK+8, TN, 64, TM,
+ COLI_XDNA_HARD_SHAPE_UNSUPPORTED }
+ };
+ for(size_t c = 0; c < sizeof cases/sizeof cases[0]; c++){
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ Fmt4 T; fmt4_make(&T, cases[c].I, cases[c].O, cases[c].gs>0?cases[c].gs:64, 5u);
+ ColiXdnaPrepared *slot = NULL;
+ int S = cases[c].S > 0 ? cases[c].S : 1;
+ float *x = mk_x(S, cases[c].I, 3u);
+ float *y = (float *)malloc((size_t)S*cases[c].O*4); poison(y,(size_t)S*cases[c].O);
+ int handled = coli_xdna_try_matmul(cases[c].fam, &slot, cases[c].fmt,
+ T.q4, T.s, cases[c].I, cases[c].O,
+ cases[c].gs, y, x, cases[c].S);
+ char msg[160];
+ snprintf(msg,sizeof msg,"%s -> not handled", cases[c].what); ck(handled==0,msg);
+ snprintf(msg,sizeof msg,"%s -> dispatches 0", cases[c].what);
+ ck(coli_xdna_test_dispatches()==0,msg);
+ snprintf(msg,sizeof msg,"%s -> %s", cases[c].what,
+ coli_xdna_hard_text(coli_xdna_test_last_hard()));
+ ck(coli_xdna_test_last_hard()==cases[c].want,msg);
+ snprintf(msg,sizeof msg,"%s -> output untouched", cases[c].what);
+ ck(all_poison(y,(size_t)S*cases[c].O),msg);
+ snprintf(msg,sizeof msg,"%s -> helper never called", cases[c].what);
+ ck(coli_xdna_test_helper_calls()==0,msg);
+ coli_xdna_prepared_release(&slot); free(x); free(y); fmt4_free(&T);
+ }
+ }
+
+ /* ================================================================== */
+ printf("gs!=64 is refused even though the converter can prepare it\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ Fmt4 T; fmt4_make(&T, TK, TN, 32, 8u);
+ ColiXdnaPrepared *p = coli_xdna_prepared_create();
+ ck(coli_xdna_prepare_from_fmt4(p,4,T.q4,T.s,TK,TN,32) == COLI_XDNA_PREP_OK,
+ "gs=32 prepares successfully -- a valid derived host image");
+ ck(coli_xdna_prepared_state(p) == COLI_XDNA_PREP_VALID, "and it is PREPARED_VALID");
+ coli_xdna_prepared_release(&p);
+
+ coli_xdna_test_set_force_execution(1);
+ ColiXdnaPrepared *slot = NULL;
+ float *x = mk_x(TM, TK, 4u);
+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);
+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,4,T.q4,T.s,
+ TK,TN,32,y,x,TM) == 0,
+ "GENERIC_PREPARABLE is not INITIAL_XDNA_QUALIFIED");
+ ck(coli_xdna_test_dispatches()==0, "dispatches 0");
+ coli_xdna_prepared_release(&slot); free(x); free(y); fmt4_free(&T);
+ }
+
+ /* ================================================================== */
+ printf("helper absent / ABI 1 / incomplete ABI 2\n");
+ {
+ struct { const char *path, *what; } hs[] = {
+ { "tests/definitely_no_such_helper.dll", "helper absent" },
+ { g_helper_abi1, "ABI generation 1 helper" },
+ { g_helper_partial, "incomplete ABI 2 helper" }
+ };
+ for(size_t c = 0; c < 3; c++){
+ use_helper(hs[c].path);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ ColiXdnaPrepared *slot = NULL;
+ float *x = mk_x(TM, TK, 5u);
+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,
+ 4,W.q4,W.s,TK,TN,64,y,x,TM);
+ char msg[160];
+ snprintf(msg,sizeof msg,"%s -> not handled", hs[c].what); ck(handled==0,msg);
+ snprintf(msg,sizeof msg,"%s -> dispatches 0", hs[c].what);
+ ck(coli_xdna_test_dispatches()==0,msg);
+ snprintf(msg,sizeof msg,"%s -> HELPER_UNAVAILABLE", hs[c].what);
+ ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_HELPER_UNAVAILABLE,msg);
+ snprintf(msg,sizeof msg,"%s -> no entry point bound", hs[c].what);
+ ck(coli_xdna_test_entry_points_bound()==0,msg);
+ snprintf(msg,sizeof msg,"%s -> output untouched", hs[c].what);
+ ck(all_poison(y,(size_t)TM*TN),msg);
+ snprintf(msg,sizeof msg,"%s -> no weight prepared (gate precedes preparation)", hs[c].what);
+ ck(slot==NULL,msg);
+ coli_xdna_prepared_release(&slot); free(x); free(y);
+ }
+ }
+
+ /* ================================================================== */
+ printf("artifact absent / corrupt -- refused before any helper call\n");
+ {
+ char bad[2048]; snprintf(bad,sizeof bad,"%s/nonexistent_root", g_root);
+ const char *roots[2]; roots[0] = bad; roots[1] = g_root;
+ for(int c = 0; c < 2; c++){
+ if(c == 1){
+ /* corrupt a STAGING copy; canonical fixtures are never damaged */
+ FILE *f = fopen(g_xclbin, "r+b");
+ if(f){ fseek(f, 100, SEEK_SET); fputc(0xFF, f); fclose(f); }
+ }
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(roots[c]);
+ coli_xdna_test_set_force_execution(1);
+ ColiXdnaPrepared *slot = NULL;
+ float *x = mk_x(TM, TK, 6u);
+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,
+ 4,W.q4,W.s,TK,TN,64,y,x,TM);
+ const char *what = c==0 ? "artifact absent" : "artifact hash mismatch";
+ char msg[160];
+ snprintf(msg,sizeof msg,"%s -> not handled", what); ck(handled==0,msg);
+ snprintf(msg,sizeof msg,"%s -> ARTIFACT_NOT_QUALIFIED", what);
+ ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED,msg);
+ snprintf(msg,sizeof msg,"%s -> helper never called", what);
+ ck(coli_xdna_test_helper_calls()==0,msg);
+ snprintf(msg,sizeof msg,"%s -> unverified bytes never reached the helper", what);
+ ck(coli_xdna_test_device_opens()==0,msg);
+ snprintf(msg,sizeof msg,"%s -> output untouched", what);
+ ck(all_poison(y,(size_t)TM*TN),msg);
+ coli_xdna_prepared_release(&slot); free(x); free(y);
+ }
+ /* restore the fixture and the registry hash agreement */
+ write_blob(g_xclbin, 11u, 4096);
+ build_registry();
+ }
+
+ /* ================================================================== */
+ printf("helper failure at every stage -- fallback intact, output untouched\n");
+ {
+ struct { int stage; const char *what; ColiXdnaExec want; } fs[] = {
+ { 1, "device init", COLI_XDNA_EXEC_DEVICE_INIT_FAILED },
+ { 2, "artifact open",COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED },
+ { 3, "weight wrap", COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED },
+ { 4, "dispatch", COLI_XDNA_EXEC_EXECUTE_FAILED },
+ { 5, "completion (output already written by the helper)",
+ COLI_XDNA_EXEC_COMPLETION_FAILED }
+ };
+ for(size_t c = 0; c < sizeof fs/sizeof fs[0]; c++){
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ /* reach into the loaded fake to arm the injection */
+ ck(coli_xdna_binding()==COLI_XDNA_AVAILABLE, "fake helper bound");
+ {
+ void (*setf)(int) = (void(*)(int))(void*)GetProcAddress(g_xdna.dll,"fake_set_fail");
+ void (*rst)(void) = (void(*)(void))(void*)GetProcAddress(g_xdna.dll,"fake_reset");
+ if(rst) rst();
+ if(setf) setf(fs[c].stage);
+ }
+ ColiXdnaPrepared *slot = NULL;
+ float *x = mk_x(TM, TK, 7u);
+ float *y = (float *)malloc((size_t)TM*TN*4); poison(y,(size_t)TM*TN);
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,
+ 4,W.q4,W.s,TK,TN,64,y,x,TM);
+ char msg[200];
+ snprintf(msg,sizeof msg,"%s failure -> not handled", fs[c].what); ck(handled==0,msg);
+ snprintf(msg,sizeof msg,"%s failure -> %s", fs[c].what,
+ coli_xdna_exec_text(coli_xdna_test_last_exec()));
+ ck(coli_xdna_test_last_exec()==fs[c].want,msg);
+ snprintf(msg,sizeof msg,"%s failure -> caller output STILL untouched", fs[c].what);
+ ck(all_poison(y,(size_t)TM*TN),msg);
+ snprintf(msg,sizeof msg,"%s failure -> counted as a fallback", fs[c].what);
+ ck(coli_xdna_test_fallbacks()>=1,msg);
+ snprintf(msg,sizeof msg,"%s failure -> zero successful dispatches", fs[c].what);
+ ck(coli_xdna_test_dispatches()==0,msg);
+ snprintf(msg,sizeof msg,"%s failure -> fmt4 source unchanged", fs[c].what);
+ ck(W.q4 != NULL && W.gs == 64, msg);
+ coli_xdna_prepared_release(&slot); free(x); free(y);
+ { void (*rst)(void) = (void(*)(void))(void*)GetProcAddress(g_xdna.dll,"fake_reset");
+ if(rst) rst(); }
+ }
+ }
+
+ /* ================================================================== */
+ printf("weight reuse and genuinely distinct runtime weights\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ int conv0 = coli_xdna_test_conversions();
+
+ ColiXdnaPrepared *sa = NULL, *sb = NULL;
+ float *x = mk_x(TM, TK, 9u);
+ float *ya = (float *)malloc((size_t)TM*TN*4);
+ float *yb = (float *)malloc((size_t)TM*TN*4);
+ float *ya2= (float *)malloc((size_t)TM*TN*4);
+
+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&sa,4,W.q4,W.s,
+ TK,TN,64,ya,x,TM)==1, "gate-like weight executes");
+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&sa,4,W.q4,W.s,
+ TK,TN,64,ya2,x,TM)==1, "same weight again executes");
+ ck(coli_xdna_test_conversions()-conv0 == 1, "prepared once, reused on the second call");
+ ck(coli_xdna_test_userptr_wraps() == 1, "wrapped once for the same pointer");
+ ck(coli_xdna_test_artifact_opens() == 1, "artifact runtime opened once, not per operation");
+ ck(memcmp(ya,ya2,(size_t)TM*TN*4)==0, "identical inputs give identical outputs");
+
+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&sb,4,W2.q4,W2.s,
+ TK,TN,64,yb,x,TM)==1, "up-like weight executes");
+ ck(coli_xdna_test_userptr_wraps() == 2, "a different prepared image is wrapped again");
+ ck(coli_xdna_test_artifact_opens() == 1, "and reuses the same artifact runtime");
+ ck(memcmp(ya,yb,(size_t)TM*TN*4)!=0, "different runtime weights give different outputs");
+
+ const unsigned short *Wb = (const unsigned short *)coli_xdna_prepared_image(sb);
+ float *ref = (float *)malloc((size_t)TM*TN*4);
+ oracle(ref, x, Wb, TM, TK, TN);
+ size_t mism=0; for(size_t i=0;i<(size_t)TM*TN;i++) if(yb[i]!=ref[i]) mism++;
+ ck(mism==0, "the second weight's output is exact too");
+ free(ref);
+ coli_xdna_prepared_release(&sa); coli_xdna_prepared_release(&sb);
+ free(x); free(ya); free(yb); free(ya2);
+ }
+
+ /* ================================================================== */
+ printf("shutdown lifetime\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_execution_shutdown();
+ ck(1, "shutdown when never initialised is safe");
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ ColiXdnaPrepared *slot = NULL;
+ float *x = mk_x(TM,TK,10u);
+ float *y = (float*)malloc((size_t)TM*TN*4);
+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,4,W.q4,W.s,
+ TK,TN,64,y,x,TM)==1, "operation succeeds");
+ coli_xdna_execution_shutdown();
+ ck(1, "shutdown after a successful operation is safe");
+ coli_xdna_execution_shutdown();
+ ck(1, "repeated shutdown is safe");
+ coli_xdna_shutdown();
+ ck(coli_xdna_test_entry_points_bound()==0, "module released, no pointer survives it");
+ coli_xdna_prepared_release(&slot); free(x); free(y);
+ }
+
+ coli_xdna_test_set_helper_path(NULL);
+ coli_xdna_shutdown();
+ coli_xdna_test_set_registry(NULL,0);
+ fmt4_free(&W); fmt4_free(&W2);
+
+ ck(coli_xdna_prepared_live_objects()==0, "no prepared object leaked");
+ ck(coli_xdna_prepared_total_bytes()==0, "no prepared bytes leaked");
+
+ printf("test_xdna_execution: %s\n", g_fail ? "FAIL" : "ok");
+ return g_fail;
+}
diff --git a/c/tests/xdna_fake_helper.c b/c/tests/xdna_fake_helper.c
new file mode 100644
index 00000000..a5def5c5
--- /dev/null
+++ b/c/tests/xdna_fake_helper.c
@@ -0,0 +1,128 @@
+/* Synthetic XDNA helper for hardware-free tests. Contains NO XRT and touches no
+ * device: it implements the ABI generation 2 contract in plain C and computes
+ * the GEMM on the CPU.
+ *
+ * This exists so the whole execution control flow -- gate order, artifact open,
+ * userptr wrap, activation staging, padding, logical-row extraction, failure
+ * classification and fallback -- is qualified on a machine with no NPU. The
+ * physical qualification of the NUMERICS is a separate, explicitly hardware
+ * run; nothing here substitutes for it.
+ *
+ * Build variants:
+ * (default) complete, ABI 2
+ * -DFAKE_ABI1 reports ABI generation 1 -- must be refused outright
+ * -DFAKE_PARTIAL ABI 2 but missing one required export -- must be refused
+ */
+
+#include <stdint.h>
+#include <string.h>
+#include <stdlib.h>
+#include <stdio.h>
+
+#define API __declspec(dllexport)
+
+#ifdef FAKE_ABI1
+#define FAKE_ABI 1u
+#else
+#define FAKE_ABI 2u
+#endif
+
+enum {
+ H_OK = 0,
+ H_E_DEVICE = -1,
+ H_E_ARTIFACT = -2,
+ H_E_NOT_OPEN = -3,
+ H_E_WRAP = -4,
+ H_E_SIZE = -5,
+ H_E_DISPATCH = -6,
+ H_E_COMPLETION = -7,
+ H_E_EXCEPTION = -8
+};
+
+/* Which stage the next call should fail at. 0 = none. */
+enum { F_NONE = 0, F_DEVICE, F_ARTIFACT, F_WRAP, F_EXECUTE, F_COMPLETION };
+
+static int g_fail;
+static int g_open, g_wrap, g_exec, g_relw, g_shut;
+static int g_is_open;
+static const unsigned short *g_w; /* borrowed, never owned */
+static size_t g_m, g_k, g_n;
+static char g_err[256] = {0};
+
+static float b2f(unsigned short b){
+ unsigned int u = (unsigned int)b << 16;
+ float f; memcpy(&f, &u, sizeof f); return f;
+}
+
+API unsigned int coli_xdna_helper_abi_version(void){ return FAKE_ABI; }
+API const char *coli_xdna_helper_last_error(void){ return g_err; }
+
+API int coli_xdna_helper_open(const char *xclbin, const char *insts,
+ uint32_t m, uint32_t k, uint32_t n){
+ g_open++;
+ if(g_fail == F_DEVICE){ snprintf(g_err,sizeof g_err,"injected device failure"); return H_E_DEVICE; }
+ if(g_fail == F_ARTIFACT){ snprintf(g_err,sizeof g_err,"injected artifact failure"); return H_E_ARTIFACT; }
+ /* The host is supposed to hand us real, already-verified paths. Checking
+ * they are at least openable keeps the fake honest about that contract. */
+ if(!xclbin || !insts) return H_E_ARTIFACT;
+ { FILE *f = fopen(xclbin, "rb"); if(!f) return H_E_ARTIFACT; fclose(f); }
+ { FILE *f = fopen(insts, "rb"); if(!f) return H_E_ARTIFACT; fclose(f); }
+ g_m = m; g_k = k; g_n = n; g_is_open = 1; g_w = NULL;
+ return H_OK;
+}
+
+API int coli_xdna_helper_wrap_weight(void *bf16, uint64_t bytes){
+ g_wrap++;
+ if(!g_is_open) return H_E_NOT_OPEN;
+ if(g_fail == F_WRAP){ snprintf(g_err,sizeof g_err,"injected wrap failure"); return H_E_WRAP; }
+ if(!bf16 || bytes != (uint64_t)g_k * g_n * 2) return H_E_SIZE;
+ /* The alignment the qualified userptr path requires. The fake enforces it
+ * so a host regression that dropped the check would still be caught here. */
+ if(((uintptr_t)bf16 % 4096u) != 0){ snprintf(g_err,sizeof g_err,"unaligned userptr"); return H_E_WRAP; }
+ g_w = (const unsigned short *)bf16; /* borrowed; no copy is made */
+ return H_OK;
+}
+
+#ifndef FAKE_PARTIAL
+API int coli_xdna_helper_execute(const void *a_bf16, uint64_t a_bytes,
+ void *c_f32, uint64_t c_bytes){
+ g_exec++;
+ if(!g_is_open || !g_w) return H_E_NOT_OPEN;
+ if(g_fail == F_EXECUTE){ snprintf(g_err,sizeof g_err,"injected dispatch failure"); return H_E_DISPATCH; }
+ if(a_bytes != (uint64_t)g_m * g_k * 2) return H_E_SIZE;
+ if(c_bytes != (uint64_t)g_m * g_n * 4) return H_E_SIZE;
+ {
+ const unsigned short *A = (const unsigned short *)a_bf16;
+ float *C = (float *)c_f32;
+ size_t i, j, kk;
+ for(i = 0; i < g_m; i++)
+ for(j = 0; j < g_n; j++){
+ float acc = 0.0f;
+ for(kk = 0; kk < g_k; kk++)
+ acc += b2f(A[i*g_k + kk]) * b2f(g_w[kk*g_n + j]);
+ C[i*g_n + j] = acc;
+ }
+ }
+ if(g_fail == F_COMPLETION){
+ /* The hostile shape: the output buffer has already been written and we
+ * then declare failure. The host must still not treat it as handled. */
+ snprintf(g_err,sizeof g_err,"injected completion failure");
+ return H_E_COMPLETION;
+ }
+ return H_OK;
+}
+#endif
+
+API int coli_xdna_helper_release_weight(void){ g_relw++; g_w = NULL; return H_OK; }
+API void coli_xdna_helper_shutdown(void){ g_shut++; g_is_open = 0; g_w = NULL; }
+
+/* -- test controls, not part of the ABI ---------------------------------- */
+API void fake_set_fail(int stage){ g_fail = stage; }
+API void fake_reset(void){
+ g_fail = 0; g_open = g_wrap = g_exec = g_relw = g_shut = 0;
+ g_is_open = 0; g_w = NULL; g_err[0] = '\0';
+}
+API void fake_counts(int *open, int *wrap, int *exec, int *relw, int *shut){
+ if(open) *open = g_open; if(wrap) *wrap = g_wrap; if(exec) *exec = g_exec;
+ if(relw) *relw = g_relw; if(shut) *shut = g_shut;
+}
diff --git a/c/tests/xdna_physical_probe.c b/c/tests/xdna_physical_probe.c
new file mode 100644
index 00000000..e3618d44
--- /dev/null
+++ b/c/tests/xdna_physical_probe.c
@@ -0,0 +1,329 @@
+/* W2-N7-I5 -- PHYSICAL qualification of the native XDNA lane.
+ *
+ * NOT a test gate: it requires an XDNA2 NPU, a working XRT installation, the
+ * built coli_xdna.dll and the qualified F3 artifact bytes, none of which a
+ * build machine is entitled to assume. It is built and run explicitly.
+ *
+ * Every step below is PRODUCTION code -- the production registry, the
+ * production integrity check, the production loader, the production weight
+ * preparation, the production hard-eligibility gates and the production
+ * candidate function that the GLM shared gate/up call sites invoke. This file
+ * supplies only a synthetic tensor and synthetic activations, and the oracles.
+ *
+ * usage: xdna_physical_probe <artifact-root> <helper-dll>
+ */
+
+#define main coli_glm_main_unused
+#include "../colibri.c"
+#undef main
+
+#include "../backend_xdna.h"
+
+#define PK 6144 /* the frozen F3 K */
+#define PN 2048 /* the frozen F3 N */
+#define PGS 64
+
+static int g_bad = 0;
+
+static float b2f(unsigned short b){
+ unsigned int u = (unsigned int)b << 16; float f; memcpy(&f,&u,4); return f;
+}
+static unsigned short f2b_local(float f){
+ unsigned int u; memcpy(&u,&f,4);
+ u += 0x7FFFu + ((u>>16)&1u);
+ return (unsigned short)(u>>16);
+}
+
+/* ---- the two BF16 oracles, and why there are two -------------------------
+ *
+ * ORACLE A -- ACCEPTANCE. The criterion frozen by N6-A1C-1 and reused unchanged
+ * by A1A: take the exact BF16 inputs the lane submits, accumulate in DOUBLE,
+ * round once to f32, and accept on absolute 0.5 OR relative 0.05. A1C-1's own
+ * note on this oracle is that "the only error present is the device's
+ * accumulation order", which is precisely what it is built to tolerate. This is
+ * a pre-declared research criterion, not a threshold chosen after seeing
+ * output.
+ *
+ * ORACLE B -- CONTINUITY. The same BF16 inputs accumulated sequentially in f32,
+ * which is the comparator A5-R1 reported bit-exactness against.
+ *
+ * They are kept separate because bit-exactness against Oracle B is NOT a
+ * universal property, and this slice is where that became visible. A5-R1's
+ * fixture used activations that are exact multiples of 2^-7 and weights that
+ * are small multiples of 0.25 -- every value exactly representable in BF16, and
+ * every partial sum exact -- so accumulation order could not matter and any
+ * order gave the identical result. Once activations carry a full f32 mantissa,
+ * partial sums round, order matters, and the device legitimately differs from
+ * one particular host summation order. Oracle B is therefore REPORTED at both
+ * fixtures and GATES at neither; Oracle A is the acceptance criterion. */
+
+static const float ABS_TOL = 0.5f, REL_TOL = 0.05f;
+static int accept_a1c1(float expected, float actual){
+ float d = fabsf(expected - actual);
+ if(d <= ABS_TOL) return 1;
+ float norm = fabsf(expected) + fabsf(actual);
+ return d <= norm * REL_TOL;
+}
+
+static void oracle_double(float *out, const float *x, const unsigned short *W,
+ int S, int K, int N){
+ #pragma omp parallel for schedule(static)
+ for(int i = 0; i < S; i++)
+ for(int j = 0; j < N; j++){
+ double acc = 0.0;
+ for(int k = 0; k < K; k++)
+ acc += (double)b2f(f2b_local(x[(size_t)i*K+k])) * (double)b2f(W[(size_t)k*N+j]);
+ out[(size_t)i*N+j] = (float)acc;
+ }
+}
+
+static void oracle_bf16(float *out, const float *x, const unsigned short *W,
+ int S, int K, int N){
+ #pragma omp parallel for schedule(static)
+ for(int i = 0; i < S; i++)
+ for(int j = 0; j < N; j++){
+ float acc = 0.0f;
+ for(int k = 0; k < K; k++)
+ acc += b2f(f2b_local(x[(size_t)i*K+k])) * b2f(W[(size_t)k*N+j]);
+ out[(size_t)i*N+j] = acc;
+ }
+}
+
+/* Report both oracles for one result. Returns 1 when the ACCEPTANCE criterion
+ * holds. */
+static int report_oracles(const float *y, const float *x, const unsigned short *W,
+ int S, int K, int N){
+ float *refA = (float*)malloc((size_t)S*N*4);
+ float *refB = (float*)malloc((size_t)S*N*4);
+ oracle_double(refA, x, W, S, K, N);
+ oracle_bf16(refB, x, W, S, K, N);
+ size_t mism = 0, bitmis = 0, nan_n = 0, inf_n = 0;
+ double maxabsA = 0, maxrelA = 0, maxabsB = 0;
+ for(size_t i = 0; i < (size_t)S*N; i++){
+ if(!accept_a1c1(refA[i], y[i])) mism++;
+ double ae = fabs((double)y[i]-(double)refA[i]);
+ if(ae > maxabsA) maxabsA = ae;
+ double den = fabs((double)refA[i]);
+ if(den > 1e-6 && ae/den > maxrelA) maxrelA = ae/den;
+ if(y[i] != refB[i]) bitmis++;
+ double be = fabs((double)y[i]-(double)refB[i]);
+ if(be > maxabsB) maxabsB = be;
+ if(y[i] != y[i]) nan_n++;
+ else if(y[i] > 3.0e38f || y[i] < -3.0e38f) inf_n++;
+ }
+ printf("ORACLE_A_ACCEPT mismatches=%llu (criterion abs<=0.5 OR rel<=0.05) max_abs=%.9f max_rel=%.9f\n",
+ (unsigned long long)mism, maxabsA, maxrelA);
+ printf("ORACLE_B_EXACT bit_mismatches=%llu max_abs=%.9f\n",
+ (unsigned long long)bitmis, maxabsB);
+ printf("NAN %llu\nINF %llu\n",
+ (unsigned long long)nan_n, (unsigned long long)inf_n);
+ free(refA); free(refB);
+ return mism == 0;
+}
+
+int main(int argc, char **argv){
+ if(argc < 3){ fprintf(stderr,"usage: xdna_physical_probe <artifact-root> <helper-dll>\n"); return 2; }
+ const char *root = argv[1], *helper = argv[2];
+
+ printf("W2-N7-I5 PHYSICAL XDNA QUALIFICATION\n");
+ printf("PID %lu\n", (unsigned long)GetCurrentProcessId());
+ printf("ARTIFACT_ROOT %s\n", root);
+ printf("HELPER %s\n", helper);
+ printf("REGISTRY production (not a test registry)\n");
+
+ coli_xdna_test_set_helper_path(helper);
+ coli_xdna_test_set_artifact_root(root);
+
+ printf("BINDING %s\n", coli_xdna_binding_text(coli_xdna_binding()));
+ if(coli_xdna_binding() != COLI_XDNA_AVAILABLE){
+ printf("RESULT FAIL -- helper did not bind\n"); return 1; }
+ printf("ABI host %u\n", (unsigned)COLI_XDNA_ABI_VERSION);
+
+ /* Production static verdict, on the production registry and the real bytes. */
+ ColiXdnaRequest q;
+ q.family = COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP;
+ q.m = 64; q.k = PK; q.n = PN;
+ q.in_dtype = COLI_XDNA_DT_BF16; q.weight_dtype = COLI_XDNA_DT_BF16;
+ q.out_dtype = COLI_XDNA_DT_F32;
+ ColiXdnaStatic st = coli_xdna_static_eligibility(&q, root);
+ printf("STATIC %s\n", coli_xdna_static_text(st));
+ if(st != COLI_XDNA_STATIC_QUALIFIED){
+ printf("RESULT FAIL -- artifact integrity gate refused\n"); return 1; }
+
+ /* Two genuinely different runtime weights: gate-like and up-like. */
+ const int rb = (PK+1)/2, ng = (PK+PGS-1)/PGS;
+ QT G, U;
+ memset(&G,0,sizeof G); memset(&U,0,sizeof U);
+ QT *ts[2] = { &G, &U };
+ unsigned seeds[2] = { 20260824u, 77771111u };
+ for(int t = 0; t < 2; t++){
+ QT *w = ts[t];
+ w->fmt = 4; w->I = PK; w->O = PN; w->gs = PGS;
+ w->q4 = (uint8_t*)malloc((size_t)PN*rb);
+ w->s = (float*)malloc((size_t)PN*ng*sizeof(float));
+ unsigned s = seeds[t];
+ for(size_t i=0;i<(size_t)PN*rb;i++){ s=s*1664525u+1013904223u; w->q4[i]=(uint8_t)(s>>24); }
+ for(size_t i=0;i<(size_t)PN*ng;i++){ s=s*1664525u+1013904223u;
+ w->s[i] = 0.25f*(float)(1+((s>>26)&3)); }
+ }
+
+ coli_xdna_test_set_force_execution(1);
+
+ int Ms[] = { 1, 32, 64 };
+ const char *names[2] = { "sh_gate-like", "sh_up-like" };
+ unsigned long long hashes[2][3];
+
+ for(int t = 0; t < 2; t++){
+ for(size_t c = 0; c < sizeof Ms/sizeof Ms[0]; c++){
+ int S = Ms[c];
+ QT *w = ts[t];
+ float *x = (float*)malloc((size_t)S*PK*4);
+ unsigned s = 31337u + (unsigned)S + 1000u*(unsigned)t;
+ for(size_t i=0;i<(size_t)S*PK;i++){ s=s*1664525u+1013904223u;
+ x[i] = (float)((int)((s>>20)&255)-128)*0.0078125f; }
+ float *y = (float*)malloc((size_t)S*PN*4);
+ for(size_t i=0;i<(size_t)S*PN;i++) y[i] = -123456.0f;
+
+ int d0 = coli_xdna_test_dispatches();
+ double t0 = now_s();
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,
+ &w->xdna, w->fmt, w->q4, w->s,
+ w->I, w->O, w->gs, y, x, S);
+ double ms = (now_s()-t0)*1000.0;
+
+ printf("\n-- %s M=%d --\n", names[t], S);
+ printf("HANDLED %d\n", handled);
+ printf("HARD %s\n", coli_xdna_hard_text(coli_xdna_test_last_hard()));
+ printf("EXEC %s\n", coli_xdna_exec_text(coli_xdna_test_last_exec()));
+ if(!handled){
+ printf("HELPER_ERR (see loader)\n");
+ printf("RESULT FAIL -- native execution declined or failed\n");
+ g_bad = 1; free(x); free(y); continue;
+ }
+ printf("DISPATCHES %d (+%d)\n", coli_xdna_test_dispatches(),
+ coli_xdna_test_dispatches()-d0);
+ printf("ARTIFACT_OPENS %d\n", coli_xdna_test_artifact_opens());
+ printf("USERPTR_WRAPS %d\n", coli_xdna_test_userptr_wraps());
+ printf("PADDED_OPS %d\n", coli_xdna_test_padded_operations());
+ printf("ELAPSED_MS %.3f\n", ms);
+
+ const unsigned short *W = (const unsigned short*)coli_xdna_prepared_image(w->xdna);
+ printf("PREPARED_PTR %p mod4096=%llu\n", (const void*)W,
+ (unsigned long long)((uintptr_t)W % 4096u));
+ printf("PREPARED_BYTES %llu\n",
+ (unsigned long long)coli_xdna_prepared_bytes(w->xdna));
+
+ if(!report_oracles(y, x, W, S, PK, PN)) g_bad = 1;
+
+ /* ---- ORACLE B: the exact current path, for characterisation ---- */
+ float *refB = (float*)malloc((size_t)S*PN*4);
+ matmul_qt(refB, x, w, S);
+ double maxabsB = 0, maxrelB = 0, sse = 0;
+ for(size_t i=0;i<(size_t)S*PN;i++){
+ double a = (double)y[i], b = (double)refB[i], d = fabs(a-b);
+ if(d>maxabsB) maxabsB=d;
+ double den = fabs(b);
+ if(den > 1e-12 && d/den > maxrelB) maxrelB = d/den;
+ sse += d*d;
+ }
+ printf("VS_MATMUL_QT max_abs=%.6f max_rel=%.3f rms=%.6f\n",
+ maxabsB, maxrelB, sqrt(sse/((double)S*PN)));
+
+ unsigned long long h = 1469598103934665603ULL;
+ for(size_t i=0;i<(size_t)S*PN*4;i++){ h ^= ((const unsigned char*)y)[i];
+ h *= 1099511628211ULL; }
+ hashes[t][c] = h;
+ printf("OUTPUT_HASH %016llx\n", h);
+
+ free(refB); free(x); free(y);
+ }
+ }
+
+ printf("\n-- dynamic runtime weights --\n");
+ for(size_t c = 0; c < 3; c++)
+ printf("M=%-3d gate=%016llx up=%016llx distinct=%s\n", Ms[c],
+ hashes[0][c], hashes[1][c],
+ hashes[0][c]!=hashes[1][c] ? "YES" : "NO");
+ for(size_t c = 0; c < 3; c++) if(hashes[0][c]==hashes[1][c]) g_bad = 1;
+
+ printf("\nARTIFACT_OPENS_TOTAL %d (one artifact serving every M and both weights)\n",
+ coli_xdna_test_artifact_opens());
+ printf("DISPATCHES_TOTAL %d\n", coli_xdna_test_dispatches());
+ printf("COMPLETIONS_TOTAL %d\n", coli_xdna_test_completions());
+ printf("DEVICE_OPENS %d\n", coli_xdna_test_device_opens());
+ printf("CONVERSIONS %d\n", coli_xdna_test_conversions());
+
+ /* ---- characterising the difference against the CURRENT path -------------
+ *
+ * The activations above are all exact multiples of 2^-7 and the weights all
+ * small multiples of 0.25, so every value is exactly representable in BF16
+ * and the f32->BF16 conversion is lossless. That makes the current path and
+ * the lane agree exactly -- a true result, but one that characterises
+ * nothing, because it never exercises the conversion the two paths differ
+ * over.
+ *
+ * This case uses activations with a full f32 mantissa, which BF16 genuinely
+ * rounds. The lane must STILL be bit-exact against its own BF16 oracle; the
+ * difference against matmul_qt is the real, expected representational gap,
+ * reported and not tested against any invented threshold. */
+ {
+ int S = 64;
+ float *x = (float*)malloc((size_t)S*PK*4);
+ unsigned s = 0xC0FFEEu;
+ for(size_t i=0;i<(size_t)S*PK;i++){
+ s = s*1664525u+1013904223u;
+ /* a generic f32 in [-1,1): 24 significant bits, not BF16-exact */
+ x[i] = ((float)(s >> 8) / (float)(1u<<24)) * 2.0f - 1.0f;
+ }
+ float *y = (float*)malloc((size_t)S*PN*4);
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,
+ &G.xdna, G.fmt, G.q4, G.s,
+ G.I, G.O, G.gs, y, x, S);
+ printf("\n-- generic f32 activations (BF16 rounding is LOSSY here), M=%d --\n", S);
+ printf("HANDLED %d\n", handled);
+ if(!handled) g_bad = 1;
+ else {
+ const unsigned short *W = (const unsigned short*)coli_xdna_prepared_image(G.xdna);
+ if(!report_oracles(y, x, W, S, PK, PN)) g_bad = 1;
+
+ float *refB = (float*)malloc((size_t)S*PN*4);
+ matmul_qt(refB, x, &G, S);
+ double maxabsB=0, maxrelB=0, sse=0, refmax=0;
+ for(size_t i=0;i<(size_t)S*PN;i++){
+ double a=(double)y[i], b=(double)refB[i], d=fabs(a-b);
+ if(d>maxabsB) maxabsB=d;
+ if(fabs(b)>refmax) refmax=fabs(b);
+ if(fabs(b)>1e-12 && d/fabs(b) > maxrelB) maxrelB = d/fabs(b);
+ sse += d*d;
+ }
+ printf("VS_MATMUL_QT max_abs=%.6f max_rel=%.3f rms=%.6f ref_max_abs=%.6f\n",
+ maxabsB, maxrelB, sqrt(sse/((double)S*PN)), refmax);
+ printf("INTERPRETATION expected representational gap: matmul_qt accumulates f32\n");
+ printf(" activations against dequantised int4; the lane is BF16\n");
+ printf(" throughout. max_rel is inflated by near-zero reference\n");
+ printf(" dot products and is NOT an error measure.\n");
+ free(refB);
+ }
+ free(x); free(y);
+ }
+
+ /* Negative control on live hardware: same shape, wrong family. */
+ {
+ float *x = (float*)malloc((size_t)8*PK*4); memset(x,0,(size_t)8*PK*4);
+ float *y = (float*)malloc((size_t)8*PN*4);
+ int before = coli_xdna_test_dispatches();
+ int h2 = coli_xdna_try_matmul(COLI_XDNA_FAMILY_NONE, &G.xdna, G.fmt, G.q4, G.s,
+ G.I, G.O, G.gs, y, x, 8);
+ printf("\nWRONG_FAMILY_HANDLED %d dispatch_delta %d (%s)\n", h2,
+ coli_xdna_test_dispatches()-before,
+ coli_xdna_hard_text(coli_xdna_test_last_hard()));
+ if(h2 || coli_xdna_test_dispatches()!=before) g_bad = 1;
+ free(x); free(y);
+ }
+
+ coli_xdna_execution_shutdown();
+ coli_xdna_shutdown();
+ printf("\nSHUTDOWN clean\n");
+ printf("RESULT %s\n", g_bad ? "FAIL" : "PASS");
+ return g_bad;
+}