Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

feat(xdna): make the native lane fail closed

Kenneth-Javier Ortigoza2e7578ba7dc9
diff --git a/GPU_BACKENDS.md b/GPU_BACKENDS.md
index c71423e1..cd572dde 100644
--- a/GPU_BACKENDS.md
+++ b/GPU_BACKENDS.md
@@ -528,3 +528,141 @@ value β€” every request declines.
No full-model acceleration, no token-throughput figure, no general XDNA backend,
no routed-expert support, no concurrency with the GPU, and no scheduler. One
family, one shape, one bucket, one dispatch at a time.
+
+### Failure and fallback
+
+The invariant the lane is built around:
+
+> **Optional XDNA work may fail. Current Colibri operation semantics may not.**
+
+Every decline and every runtime failure ends the same way: the candidate returns
+"not handled" and the caller runs the exact `matmul_qt` call that stood at that
+site before the lane existed. There is no special CPU fallback, no alternative
+dequantisation path, no partial-result salvage, and no helper-side recovery. The
+candidate never calls `matmul_qt` itself, so the call graph stays acyclic and no
+operation can be computed twice.
+
+Failure stages are classified rather than collapsed, because they call for
+different actions:
+
+```
+DECLINED not eligible -- never an error
+HELPER_UNAVAILABLE absent, or would not load
+HELPER_ABI_INCOMPATIBLE loaded, wrong generation or incomplete
+ARTIFACT_UNAVAILABLE this build does not ship those bytes
+ARTIFACT_INTEGRITY_FAILED the bytes are not the bytes that were qualified
+ARTIFACT_UNQUALIFIED a known artifact that was never correctness-qualified
+REGISTRY_INVALID the registry itself is malformed
+WEIGHT_PREPARE_FAILED fmt4 to BF16 conversion failed
+PREPARED_INVALID no usable prepared image
+ALIGNMENT_INVALID prepared pointer not 4096-aligned
+DEVICE_INIT_FAILED the device would not initialise
+ARTIFACT_OPEN_FAILED verified bytes, but the runtime would not open them
+WEIGHT_WRAP_FAILED the userptr wrap was refused
+EXECUTE_FAILED dispatch refused or threw
+COMPLETION_FAILED dispatched, did not complete cleanly
+```
+
+An absent artifact and a *tampered* one are deliberately different verdicts. So
+are a missing helper and an incompatible one.
+
+### Output validity is a state, not a measurement
+
+XDNA output is valid **only** after the helper reports successful completion.
+A failure at any stage β€” including one that occurs after the helper has already
+written a full, finite, entirely plausible result β€” leaves the output invalid,
+and the caller's buffer untouched. Nothing about the bytes themselves can raise
+that verdict: there is no NaN scan, no finiteness check and no plausibility
+heuristic anywhere in the lane, because none of them would be evidence.
+
+Structurally, the caller's output buffer is never passed to the helper at all.
+The helper writes into a lane-owned staging buffer, and only a successful
+completion causes the logical rows to be copied out. A late failure therefore
+cannot leave a half-written result behind even in principle.
+
+### Lane health
+
+The narrowest model the observed failures justify:
+
+| failure | scope |
+|---|---|
+| helper absent / unloadable / ABI mismatch | sticky loader verdict, never retried |
+| **device init** | **process-scoped** β€” the lane is marked unavailable |
+| artifact runtime open | one shape; the lane stays healthy |
+| weight wrap, dispatch, completion | one operation; the lane stays healthy |
+
+There is no retry, backoff or quarantine policy beyond this. A full
+`coli_xdna_execution_shutdown()` is the only thing that clears an unavailable
+lane, because a fresh attempt after a complete teardown is meaningful and a
+fresh attempt on every operation is not.
+
+### Prepared state survives runtime failure
+
+Whether the prepared BF16 image is **correct** and whether the runtime
+**succeeded** are separate facts, and a runtime failure does not invalidate a
+correct image. A wrap, dispatch or completion failure leaves the prepared weight
+`PREPARED_VALID` and reusable; only a conversion failure produces
+`PREPARED_INVALID`. This matters for any future cache policy, which would
+otherwise throw away good work on unrelated news.
+
+### Userptr wrapper lifetime
+
+The helper-owned wrapper is **persistent runtime state**: it is created on
+demand, reused across operations, and released when it is replaced, when the
+engine frees or invalidates the memory it borrows, or at shutdown.
+
+It is keyed on **(pointer, publication generation)**, not on the pointer alone.
+Retained prepared capacity is reused in place, so a different weight can occupy
+the same address β€” and because the helper snapshots at wrap time via
+`sync(BO_TO_DEVICE)`, keying on the address alone left the device computing
+against a view that no longer matched the engine's image. That was measured on
+real XDNA2 hardware. Every publication now bumps a generation, and the engine
+tells the lane to release a wrapper before freeing or invalidating the memory it
+borrows, so a wrapper can never outlive or alias released engine memory.
+
+### Two internal modes, neither of them public
+
+```
+AUTO-LIKE coli_xdna_try_matmul() decline or failure -> current path
+EXPLICIT coli_xdna_test_attempt() decline or failure -> classified failure,
+ no fallback, no output claimed
+```
+
+Both share one implementation so their gate order and classification cannot
+drift apart. Explicit mode is a separate entry point rather than a mode flag, so
+no global state can leave the production seam in a no-fallback configuration.
+
+**Default behaviour is unchanged and remains so.** With a helper present, a
+device available and valid artifacts staged, an ordinary build still runs the
+current path and dispatches zero XDNA operations. Automatic selection needs an
+economic policy that does not exist, and a semantic qualification that has not
+been done β€” see below.
+
+### What successful XDNA execution does and does not mean
+
+```
+SUCCESSFUL_XDNA_CORRECTNESS_OWNER = the qualified BF16 oracle
+FAILED_XDNA_FALLBACK_CORRECTNESS_OWNER = the current matmul_qt path
+```
+
+These are different contracts and must not be conflated. Device execution is
+qualified against a BF16 oracle under a criterion frozen by the research
+programme. It is **not** a claim that the lane is numerically interchangeable
+with `matmul_qt`: the current path accumulates f32 activations against
+dequantised int4, the lane is BF16 throughout, and the two legitimately differ.
+
+```
+MODEL_LEVEL_BF16_REPLACEMENT_ACCEPTABILITY = NOT YET QUALIFIED
+```
+
+Whether substituting BF16 activation semantics is acceptable *for a model* is an
+open question, and answering it is a prerequisite for any automatic selection.
+It is not answered by inventing an elementwise tolerance.
+
+### A note on the registry
+
+The compiled-in production registry is now covered by a regression that reads it
+with no test rows installed. An earlier revision initialised its row count to
+zero, leaving the table present but empty; every caller at the time was a test
+that installed its own registry, so nothing noticed until the first production
+consumer arrived.
diff --git a/c/.gitignore b/c/.gitignore
index 5b1b58af..5cfdc74f 100644
--- a/c/.gitignore
+++ b/c/.gitignore
@@ -48,4 +48,5 @@ cublasLt64*.dll
cudart64*.dll
# Optional XDNA test scaffolding that the *.o / *.dll globs above do not cover.
tests/xdna_exec_fixtures/
+tests/xdna_fail_fixtures/
tests/xdna_physical_probe
diff --git a/c/Makefile b/c/Makefile
index 9cb1c58d..f14f4feb 100644
--- a/c/Makefile
+++ b/c/Makefile
@@ -1107,6 +1107,12 @@ tests/xdna_fake_helper_partial.dll: tests/xdna_fake_helper.c
tests/test_xdna_execution$(EXE): tests/test_xdna_execution.c backend_xdna.c backend_xdna.h compat.h $(XDNA_FAKE_HELPERS)
$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)
+# Failure/fallback owner. Needs BOTH the engine (matmul_qt is the fallback
+# target it verifies against) and the synthetic ABI-2 helpers, which is why it
+# is a separate owner from test_xdna_execution.
+tests/test_xdna_failure$(EXE): tests/test_xdna_failure.c colibri.c backend_xdna.c backend_xdna.h st.h uring.h json.h tok.h tok_unicode.h compat.h grammar.h tier.h quant.h sample.h kv_persist.h telemetry.h route_trace.h $(XDNA_FAKE_HELPERS)
+ $(CC) $(CFLAGS) -DCOLI_XDNA $< backend_xdna.c -o $@ $(LDFLAGS)
+
# NOT a test gate -- intentionally absent from TEST_BINS (its name does not match
# the tests/test_*$(EXE) pattern the gate list is derived from). It needs a real
# XDNA2 NPU, a working XRT, a built coli_xdna.dll and the qualified artifact
diff --git a/c/backend_xdna.c b/c/backend_xdna.c
index 659e20b6..5c052615 100644
--- a/c/backend_xdna.c
+++ b/c/backend_xdna.c
@@ -611,6 +611,7 @@ struct ColiXdnaPrepared {
size_t bytes; /* logical payload: k*n*sizeof(bf16) */
size_t capacity; /* what base can actually hold */
unsigned k, n;
+ unsigned long long generation; /* bumped on every successful publication */
};
/* Engine-side host accounting. Deliberately counts LOGICAL payload bytes: the
@@ -620,6 +621,11 @@ struct ColiXdnaPrepared {
static size_t g_xdna_prepared_bytes;
static int g_xdna_prepared_objects;
+/* Defined with the execution lane below, declared here because the prepared
+ * buffer owner must be able to tell the lane to let go of memory it is about
+ * to free or replace. */
+static void coli_xdna_lane_forget_pointer(const void *p);
+
const char *coli_xdna_prep_text(ColiXdnaPrepState s){
switch(s){
case COLI_XDNA_PREP_UNPREPARED: return "UNPREPARED";
@@ -647,6 +653,9 @@ ColiXdnaPrepared *coli_xdna_prepared_create(void){
* so they get separate operations. */
void coli_xdna_prepared_free_buffer(ColiXdnaPrepared *p){
if(!p) return;
+ /* Tell the lane before the memory goes away: a helper-owned userptr wrapper
+ * borrowing this allocation must never outlive it. */
+ coli_xdna_lane_forget_pointer(p->base);
if(p->base){
compat_aligned_free(p->base); /* MUST pair with posix_memalign */
p->base = NULL;
@@ -735,6 +744,10 @@ int coli_xdna_prepare_publish_success(ColiXdnaPrepared *p){
if(!p->base || !coli_xdna_pointer_alignment_ok(p->base)) return 0;
if(p->bytes == 0 || p->capacity < p->bytes) return 0;
p->state = COLI_XDNA_PREP_VALID;
+ /* A new publication is a new image even when it lands on the same address:
+ * retained capacity is reused in place. Anything caching a view of these
+ * bytes keys on (pointer, generation), never on the pointer alone. */
+ p->generation++;
return 1;
}
@@ -749,6 +762,9 @@ int coli_xdna_prepared_invalidate(ColiXdnaPrepared *p){
if(!p) return 0;
if(p->state != COLI_XDNA_PREP_VALID) return 0;
p->state = COLI_XDNA_PREP_INVALID;
+ /* The image is no longer authoritative, so no device-side view of it may
+ * remain current either. */
+ coli_xdna_lane_forget_pointer(p->base);
return 1;
}
@@ -758,6 +774,9 @@ ColiXdnaPrepState coli_xdna_prepared_state(const ColiXdnaPrepared *p){
size_t coli_xdna_prepared_bytes(const ColiXdnaPrepared *p){ return p ? p->bytes : 0; }
unsigned coli_xdna_prepared_k(const ColiXdnaPrepared *p){ return p ? p->k : 0u; }
unsigned coli_xdna_prepared_n(const ColiXdnaPrepared *p){ return p ? p->n : 0u; }
+unsigned long long coli_xdna_prepared_generation(const ColiXdnaPrepared *p){
+ return p ? p->generation : 0ull;
+}
size_t coli_xdna_prepared_total_bytes(void){ return g_xdna_prepared_bytes; }
int coli_xdna_prepared_live_objects(void){ return g_xdna_prepared_objects; }
@@ -899,10 +918,14 @@ const char *coli_xdna_hard_text(ColiXdnaHard h){
case COLI_XDNA_HARD_SHAPE_UNSUPPORTED: return "SHAPE_UNSUPPORTED";
case COLI_XDNA_HARD_FORMAT_UNSUPPORTED: return "FORMAT_UNSUPPORTED";
case COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED: return "GROUP_SIZE_UNSUPPORTED";
- case COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED: return "ARTIFACT_NOT_QUALIFIED";
+ case COLI_XDNA_HARD_ARTIFACT_UNAVAILABLE: return "ARTIFACT_UNAVAILABLE";
+ case COLI_XDNA_HARD_ARTIFACT_INTEGRITY_FAILED: return "ARTIFACT_INTEGRITY_FAILED";
+ case COLI_XDNA_HARD_ARTIFACT_UNQUALIFIED: return "ARTIFACT_UNQUALIFIED";
+ case COLI_XDNA_HARD_REGISTRY_INVALID: return "REGISTRY_INVALID";
case COLI_XDNA_HARD_PREPARED_INVALID: return "PREPARED_INVALID";
case COLI_XDNA_HARD_ALIGNMENT_INVALID: return "ALIGNMENT_INVALID";
case COLI_XDNA_HARD_HELPER_UNAVAILABLE: return "HELPER_UNAVAILABLE";
+ case COLI_XDNA_HARD_HELPER_ABI_INCOMPATIBLE: return "HELPER_ABI_INCOMPATIBLE";
case COLI_XDNA_HARD_DEVICE_UNAVAILABLE: return "DEVICE_UNAVAILABLE";
case COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE:return "ARTIFACT_RUNTIME_UNAVAILABLE";
case COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE: return "WEIGHT_WRAP_UNAVAILABLE";
@@ -937,6 +960,9 @@ static ColiXdnaExec g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
static int g_xdna_dispatches, g_xdna_completions, g_xdna_artifact_opens;
static int g_xdna_act_preps, g_xdna_padded_ops, g_xdna_fallbacks;
static int g_xdna_device_opens, g_xdna_helper_calls, g_xdna_userptr_wraps;
+static int g_xdna_wrapper_reuses, g_xdna_wrapper_releases, g_xdna_stale_rejects;
+static int g_xdna_output_valid;
+static ColiXdnaLaneHealth g_xdna_lane_health = COLI_XDNA_LANE_UNINITIALIZED;
/* Which artifact the helper currently holds open, so a run of operations on the
* same shape does not reopen the device, the context or the program. This is a
@@ -945,11 +971,36 @@ static int g_xdna_device_opens, g_xdna_helper_calls, g_xdna_userptr_wraps;
static struct {
int open;
unsigned m, k, n;
- /* the wrapped weight, identified by the exact pointer the engine handed us */
- const void *wrapped;
+ /* The wrapped weight, identified by (pointer, generation). The pointer
+ * alone is NOT an identity: retained capacity is reused in place, so a
+ * different image can occupy the same address. */
+ const void *wrapped;
+ unsigned long long wrapped_gen;
unsigned short *act; size_t act_cap; /* [artifact_m, K] BF16 staging */
float *out; size_t out_cap; /* [artifact_m, N] F32 staging */
} g_lane;
+/* Release the helper-owned userptr wrapper and forget its identity.
+ *
+ * Called from three places, and each one matters: before wrapping a different
+ * image, before the engine frees or replaces the memory the wrapper borrows,
+ * and at shutdown. The wrapper borrows CALLER memory, so it must never outlive
+ * the allocation it points into. */
+static void coli_xdna_lane_drop_wrapper(void){
+ if(!g_lane.wrapped) return;
+ coli_xdna_helper_release_weight_call();
+ g_lane.wrapped = NULL;
+ g_lane.wrapped_gen = 0;
+ g_xdna_wrapper_releases++;
+}
+
+/* The engine is about to free or replace prepared memory. If the helper holds a
+ * wrapper over exactly that allocation, drop it FIRST. Without this the wrapper
+ * would reference a freed region until the next wrap replaced it -- and a
+ * subsequent allocation landing on the same address would make the stale
+ * wrapper look current. */
+static void coli_xdna_lane_forget_pointer(const void *p){
+ if(p && g_lane.wrapped == p) coli_xdna_lane_drop_wrapper();
+}
/* ---- helper entry points (ABI generation 2) ------------------------------
*
@@ -975,6 +1026,9 @@ int coli_xdna_test_artifact_opens(void){ return g_xdna_artifact_opens; }
int coli_xdna_test_activation_preparations(void){ return g_xdna_act_preps; }
int coli_xdna_test_padded_operations(void){ return g_xdna_padded_ops; }
int coli_xdna_test_fallbacks(void){ return g_xdna_fallbacks; }
+int coli_xdna_test_wrapper_reuses(void){ return g_xdna_wrapper_reuses; }
+int coli_xdna_test_wrapper_releases(void){ return g_xdna_wrapper_releases; }
+int coli_xdna_test_stale_wrapper_rejects(void){ return g_xdna_stale_rejects; }
ColiXdnaHard coli_xdna_test_last_hard(void){ return g_xdna_last_hard; }
ColiXdnaExec coli_xdna_test_last_exec(void){ return g_xdna_last_exec; }
@@ -1034,10 +1088,29 @@ static ColiXdnaHard coli_xdna_engine_gates(ColiXdnaFamily family,
if(!row) return COLI_XDNA_HARD_SHAPE_UNSUPPORTED;
/* 7-9 qualification flags, artifact presence, artifact integrity. All
- * three live behind coli_xdna_artifact_status, which fails closed. */
- if(coli_xdna_artifact_status(&q, g_xdna_root[0] ? g_xdna_root : NULL)
- != COLI_XDNA_STATIC_QUALIFIED)
- return COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED;
+ * three live behind coli_xdna_artifact_status, which fails closed.
+ * The sub-reason is carried through rather than collapsed: "this
+ * build does not ship that artifact", "the bytes are not the bytes
+ * that were qualified", and "we never qualified this one" call for
+ * different actions, and an operator who is told only
+ * ARTIFACT_NOT_QUALIFIED cannot tell a missing file from a tampered
+ * one. */
+ switch(coli_xdna_artifact_status(&q, g_xdna_root[0] ? g_xdna_root : NULL)){
+ case COLI_XDNA_STATIC_QUALIFIED:
+ break;
+ case COLI_XDNA_STATIC_ARTIFACT_UNAVAILABLE:
+ return COLI_XDNA_HARD_ARTIFACT_UNAVAILABLE;
+ case COLI_XDNA_STATIC_ARTIFACT_INTEGRITY_FAILED:
+ return COLI_XDNA_HARD_ARTIFACT_INTEGRITY_FAILED;
+ case COLI_XDNA_STATIC_ARTIFACT_UNQUALIFIED:
+ return COLI_XDNA_HARD_ARTIFACT_UNQUALIFIED;
+ case COLI_XDNA_STATIC_REGISTRY_INVALID:
+ return COLI_XDNA_HARD_REGISTRY_INVALID;
+ default:
+ /* Family/shape/format verdicts cannot occur here: the lookup above
+ * already matched a row on the full key. */
+ return COLI_XDNA_HARD_SHAPE_UNSUPPORTED;
+ }
*row_out = row;
return COLI_XDNA_HARD_ELIGIBLE;
@@ -1055,46 +1128,62 @@ static int coli_xdna_stage(void **base, size_t *cap, size_t need){
return 1;
}
-int coli_xdna_try_matmul(ColiXdnaFamily family,
- ColiXdnaPrepared **slot,
- int fmt, const unsigned char *q4, const float *scale,
- int I, int O, int gs,
- float *y, const float *x, int S)
+/* The shared execution core. Both modes call this, so their gate order and
+ * their failure classification cannot drift apart.
+ *
+ * It returns a CLASS, never a handled flag: deciding what a failure means for
+ * the caller belongs to the mode, not to this function. It writes y only after
+ * the helper reports successful completion, and never calls matmul_qt. */
+static ColiXdnaExec coli_xdna_attempt(ColiXdnaFamily family,
+ ColiXdnaPrepared **slot,
+ int fmt, const unsigned char *q4, const float *scale,
+ int I, int O, int gs,
+ float *y, const float *x, int S)
{
- /* Gate 0: the internal control. Without it this function is inert, which is
- * what keeps I5 from being an automatic scheduler. */
- if(!g_xdna_force){ g_xdna_fallbacks++; g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED; return 0; }
+ g_xdna_output_valid = 0; /* nothing is valid until completion says so */
if(!slot || !q4 || !scale || !y || !x){
g_xdna_last_hard = COLI_XDNA_HARD_SHAPE_UNSUPPORTED;
- g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
- g_xdna_fallbacks++; return 0;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED);
+ }
+
+ /* A device that would not initialise stays refused for the process. This is
+ * the whole of the retry policy, and deliberately not more: the sealed A5
+ * taxonomy classifies device-init failure PROCESS-scoped, and re-attempting
+ * it on every operation would cost without prospect. */
+ if(g_xdna_lane_health == COLI_XDNA_LANE_UNAVAILABLE){
+ g_xdna_last_hard = COLI_XDNA_HARD_DEVICE_UNAVAILABLE;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED);
}
const ColiXdnaArtifact *row = NULL;
ColiXdnaHard h = coli_xdna_engine_gates(family, fmt, I, O, gs, S, &row);
if(h != COLI_XDNA_HARD_ELIGIBLE){
- g_xdna_last_hard = h; g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
- g_xdna_fallbacks++; return 0;
+ g_xdna_last_hard = h;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED);
}
- /* 10 helper ABI. Checked before the weight is prepared: preparing 24 MiB
- * for a lane that cannot execute would be pure waste. */
- if(coli_xdna_binding() != COLI_XDNA_AVAILABLE){
- g_xdna_last_hard = COLI_XDNA_HARD_HELPER_UNAVAILABLE;
- g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
- g_xdna_fallbacks++; return 0;
+ /* 10 helper binding. Checked before the weight is prepared: preparing
+ * 24 MiB for a lane that cannot execute would be pure waste. The
+ * loader verdict is sticky, so this is a branch, not a module load. */
+ {
+ ColiXdnaBinding b = coli_xdna_binding();
+ if(b != COLI_XDNA_AVAILABLE){
+ g_xdna_last_hard = (b == COLI_XDNA_ABI_INCOMPATIBLE
+ || b == COLI_XDNA_SYMBOL_INCOMPLETE)
+ ? COLI_XDNA_HARD_HELPER_ABI_INCOMPATIBLE
+ : COLI_XDNA_HARD_HELPER_UNAVAILABLE;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED);
+ }
}
- /* 11 prepared weight. Lazy: prepared on first use of this tensor and
- * reused while it stays VALID for the same dimensions. There is no
- * eviction or cache policy here -- that is a later slice's decision. */
+ /* 11 prepared weight. Lazy, and reused while it stays VALID for these
+ * dimensions. No eviction or cache policy here. */
if(!*slot){
*slot = coli_xdna_prepared_create();
if(!*slot){
g_xdna_last_hard = COLI_XDNA_HARD_PREPARED_INVALID;
- g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED;
- g_xdna_fallbacks++; return 0;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED);
}
}
ColiXdnaPrepared *prep = *slot;
@@ -1103,84 +1192,88 @@ int coli_xdna_try_matmul(ColiXdnaFamily family,
|| coli_xdna_prepared_n(prep) != (unsigned)O){
if(coli_xdna_prepare_from_fmt4(prep, fmt, q4, scale, I, O, gs) != COLI_XDNA_PREP_OK){
g_xdna_last_hard = COLI_XDNA_HARD_PREPARED_INVALID;
- g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED;
- g_xdna_fallbacks++; return 0;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED);
}
}
const void *wimg = coli_xdna_prepared_image(prep); /* NULL unless VALID */
if(!wimg){
g_xdna_last_hard = COLI_XDNA_HARD_PREPARED_INVALID;
- g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED;
- g_xdna_fallbacks++; return 0;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED);
}
- /* 12 alignment. The allocator guarantees it; checking anyway is cheap and
- * turns a confusing XRT "insufficient free video memory" into a local,
- * accurate refusal. */
+ /* 12 alignment */
if(!coli_xdna_pointer_alignment_ok(wimg)){
g_xdna_last_hard = COLI_XDNA_HARD_ALIGNMENT_INVALID;
- g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
- g_xdna_fallbacks++; return 0;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED);
}
const unsigned am = row->artifact_m;
- /* 13 device / runtime / artifact runtime object. Opened lazily, only now
- * that every engine-side gate has passed. */
+ /* 13 device / runtime / artifact runtime object, opened lazily. */
if(!g_lane.open || g_lane.m != am || g_lane.k != (unsigned)I || g_lane.n != (unsigned)O){
char xb[2048], ib[2048];
if(!coli_xdna_join(xb, sizeof xb, g_xdna_root, row->xclbin_name)
|| !coli_xdna_join(ib, sizeof ib, g_xdna_root, row->insts_name)){
g_xdna_last_hard = COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE;
- g_xdna_last_exec = COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED;
- g_xdna_fallbacks++; return 0;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED);
}
- if(g_lane.open){ coli_xdna_helper_release_weight_call(); coli_xdna_helper_shutdown_call(); }
+ if(g_lane.open){ coli_xdna_lane_drop_wrapper(); coli_xdna_helper_shutdown_call(); }
+ g_lane.open = 0;
g_xdna_helper_calls++; g_xdna_device_opens++;
int rc = coli_xdna_helper_open_call(xb, ib, am, (unsigned)I, (unsigned)O);
if(rc != COLI_XDNA_H_OK){
- g_xdna_last_hard = (rc == COLI_XDNA_H_E_DEVICE)
- ? COLI_XDNA_HARD_DEVICE_UNAVAILABLE
- : COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE;
- g_xdna_last_exec = (rc == COLI_XDNA_H_E_DEVICE)
- ? COLI_XDNA_EXEC_DEVICE_INIT_FAILED
- : COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED;
- g_xdna_fallbacks++; return 0;
+ if(rc == COLI_XDNA_H_E_DEVICE){
+ g_xdna_lane_health = COLI_XDNA_LANE_UNAVAILABLE; /* process-scoped */
+ g_xdna_last_hard = COLI_XDNA_HARD_DEVICE_UNAVAILABLE;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_DEVICE_INIT_FAILED);
+ }
+ /* Artifact-runtime scope. The engine already verified these bytes,
+ * so this is NOT the same thing as a missing or tampered artifact
+ * and must stay distinguishable from both. The lane stays healthy:
+ * another shape may well open. */
+ g_xdna_last_hard = COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED);
}
g_lane.open = 1; g_lane.m = am; g_lane.k = (unsigned)I; g_lane.n = (unsigned)O;
- g_lane.wrapped = NULL;
+ g_lane.wrapped = NULL; g_lane.wrapped_gen = 0;
+ g_xdna_lane_health = COLI_XDNA_LANE_HEALTHY;
g_xdna_artifact_opens++;
}
- /* 14 userptr wrap of the engine-owned prepared image. The helper wraps
- * THIS memory; it never allocates a second copy of the weight. */
- if(g_lane.wrapped != wimg){
- g_xdna_helper_calls++; g_xdna_userptr_wraps++;
- if(coli_xdna_helper_wrap_call((void *)wimg,
- (uint64_t)coli_xdna_prepared_bytes(prep)) != COLI_XDNA_H_OK){
- g_xdna_last_hard = COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE;
- g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED;
- g_xdna_fallbacks++; return 0;
+ /* 14 userptr wrap, keyed on (pointer, generation).
+ *
+ * The pointer alone is not an identity. prepare_begin reuses retained
+ * capacity IN PLACE, so re-preparing a different weight yields the same
+ * address with different contents -- and the helper snapshots at wrap
+ * time via sync(BO_TO_DEVICE). Keying on the pointer alone therefore
+ * skipped the re-wrap and left the device computing against a view that
+ * no longer matched the engine image. Measured on real XDNA2 hardware,
+ * not merely reasoned about. */
+ {
+ unsigned long long gen = coli_xdna_prepared_generation(prep);
+ if(g_lane.wrapped == wimg && g_lane.wrapped_gen == gen){
+ g_xdna_wrapper_reuses++;
+ } else {
+ if(g_lane.wrapped == wimg) g_xdna_stale_rejects++; /* same address, new image */
+ if(g_lane.wrapped) coli_xdna_lane_drop_wrapper();
+ g_xdna_helper_calls++; g_xdna_userptr_wraps++;
+ if(coli_xdna_helper_wrap_call((void *)wimg,
+ (uint64_t)coli_xdna_prepared_bytes(prep)) != COLI_XDNA_H_OK){
+ g_xdna_last_hard = COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED);
+ }
+ g_lane.wrapped = wimg; g_lane.wrapped_gen = gen;
}
- g_lane.wrapped = wimg;
}
g_xdna_last_hard = COLI_XDNA_HARD_ELIGIBLE; /* FULL_XDNA_HARD_ELIGIBLE */
- /* ---- activation staging -------------------------------------------------
- * Caller activation is f32 [S, K] row-major. The artifact consumes BF16
- * [artifact_m, K]. The conversion uses the SAME round-to-nearest-even rule
- * as the weight, because both were qualified on that rule together.
- *
- * Rows S..artifact_m-1 are zeroed. Zero is the mathematically neutral row
- * for a pure GEMM: it contributes a zero output row and cannot perturb any
- * logical row, because C = A x B is row-independent. */
+ /* ---- activation staging ------------------------------------------------- */
size_t abytes = (size_t)am * (size_t)I * 2u;
size_t obytes = (size_t)am * (size_t)O * 4u;
if(!coli_xdna_stage((void **)&g_lane.act, &g_lane.act_cap, abytes)
|| !coli_xdna_stage((void **)&g_lane.out, &g_lane.out_cap, obytes)){
- g_xdna_last_exec = COLI_XDNA_EXEC_ACTIVATION_FAILED;
- g_xdna_fallbacks++; return 0;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_ACTIVATION_FAILED);
}
for(size_t i = 0; i < (size_t)S * (size_t)I; i++)
g_lane.act[i] = coli_xdna_f2b(x[i]);
@@ -1191,34 +1284,94 @@ int coli_xdna_try_matmul(ColiXdnaFamily family,
}
g_xdna_act_preps++;
- /* ---- blocking execution -------------------------------------------------
- * One operation, one dispatch, one wait. No worker thread, no queue, no
- * second outstanding operation. */
+ /* ---- blocking execution ------------------------------------------------- */
g_xdna_helper_calls++;
int rc = coli_xdna_helper_execute_call(g_lane.act, (uint64_t)abytes,
g_lane.out, (uint64_t)obytes);
if(rc != COLI_XDNA_H_OK){
- g_xdna_last_exec = (rc == COLI_XDNA_H_E_COMPLETION)
- ? COLI_XDNA_EXEC_COMPLETION_FAILED
- : COLI_XDNA_EXEC_EXECUTE_FAILED;
- g_xdna_fallbacks++;
- return 0; /* y untouched: the caller's path owns it */
+ /* The helper may already have written real bytes into the staging
+ * buffer before deciding it had failed. They stay there and are simply
+ * never copied: y is untouched, and no property of those bytes can make
+ * them valid, because validity is a state and this state is failure. */
+ return (g_xdna_last_exec = (rc == COLI_XDNA_H_E_COMPLETION)
+ ? COLI_XDNA_EXEC_COMPLETION_FAILED
+ : COLI_XDNA_EXEC_EXECUTE_FAILED);
}
g_xdna_dispatches++; g_xdna_completions++;
- /* Only the logical rows become output. The padded rows are never copied and
- * so can never reach anything downstream, whatever the device wrote there. */
- memcpy(y, g_lane.out, (size_t)S * (size_t)O * 4u);
- g_xdna_last_exec = COLI_XDNA_EXEC_OK;
- return 1;
+ memcpy(y, g_lane.out, (size_t)S * (size_t)O * 4u); /* logical rows only */
+ g_xdna_output_valid = 1;
+ return (g_xdna_last_exec = COLI_XDNA_EXEC_OK);
}
-void coli_xdna_execution_shutdown(void){
- if(g_lane.open){
- coli_xdna_helper_release_weight_call();
- coli_xdna_helper_shutdown_call();
+/* AUTO-LIKE mode: the production seam. Any decline or failure returns 0 and the
+ * caller runs its current path, which is the exact matmul_qt call that stood
+ * there before this lane existed. */
+int coli_xdna_try_matmul(ColiXdnaFamily family,
+ ColiXdnaPrepared **slot,
+ int fmt, const unsigned char *q4, const float *scale,
+ int I, int O, int gs,
+ float *y, const float *x, int S)
+{
+ /* Gate 0: the internal control. Without it this function is inert, which is
+ * what keeps the lane from being an automatic scheduler. */
+ if(!g_xdna_force){
+ /* Clear validity here too. Declining before the core runs still ends an
+ * attempt, and a stale 1 left over from an earlier success would claim
+ * an XDNA result for an operation the lane never touched. */
+ g_xdna_output_valid = 0;
+ g_xdna_fallbacks++;
+ g_xdna_last_exec = COLI_XDNA_EXEC_DECLINED;
+ return 0;
}
- g_lane.open = 0; g_lane.wrapped = NULL;
+ if(coli_xdna_attempt(family, slot, fmt, q4, scale, I, O, gs, y, x, S)
+ == COLI_XDNA_EXEC_OK)
+ return 1;
+ g_xdna_fallbacks++;
+ return 0;
+}
+
+/* EXPLICIT mode, internal and test-facing. Returns the class and implies no
+ * fallback: a caller here asked for XDNA specifically and gets a classified
+ * failure rather than a silent substitution. It does not consult the force
+ * control, because asking explicitly IS the request. Kept as a separate entry
+ * point rather than a mode flag so that no global state can ever leave the
+ * production seam in a no-fallback configuration. */
+ColiXdnaExec coli_xdna_test_attempt(ColiXdnaFamily family,
+ ColiXdnaPrepared **slot,
+ int fmt, const unsigned char *q4, const float *scale,
+ int I, int O, int gs,
+ float *y, const float *x, int S)
+{
+ return coli_xdna_attempt(family, slot, fmt, q4, scale, I, O, gs, y, x, S);
+}
+
+int coli_xdna_test_output_valid(void){ return g_xdna_output_valid; }
+
+const float *coli_xdna_test_output_staging(size_t *floats){
+ if(floats) *floats = g_lane.out_cap / sizeof(float);
+ return g_lane.out;
+}
+
+ColiXdnaLaneHealth coli_xdna_lane_health(void){ return g_xdna_lane_health; }
+
+const char *coli_xdna_lane_health_text(ColiXdnaLaneHealth h){
+ switch(h){
+ case COLI_XDNA_LANE_UNINITIALIZED: return "UNINITIALIZED";
+ case COLI_XDNA_LANE_HEALTHY: return "HEALTHY";
+ case COLI_XDNA_LANE_UNAVAILABLE: return "UNAVAILABLE";
+ }
+ return "UNKNOWN";
+}
+void coli_xdna_execution_shutdown(void){
+ coli_xdna_lane_drop_wrapper();
+ if(g_lane.open) coli_xdna_helper_shutdown_call();
+ g_lane.open = 0; g_lane.wrapped = NULL; g_lane.wrapped_gen = 0;
+ /* Lane health is lane runtime state, so it resets with the rest of it. A
+ * fresh attempt after a full teardown is a meaningful thing to allow; a
+ * fresh attempt on every operation is not, which is why nothing else
+ * clears it. */
+ g_xdna_lane_health = COLI_XDNA_LANE_UNINITIALIZED;
free(g_lane.act); g_lane.act = NULL; g_lane.act_cap = 0;
free(g_lane.out); g_lane.out = NULL; g_lane.out_cap = 0;
}
diff --git a/c/backend_xdna.h b/c/backend_xdna.h
index 10b320e7..efc6f127 100644
--- a/c/backend_xdna.h
+++ b/c/backend_xdna.h
@@ -316,6 +316,18 @@ size_t coli_xdna_prepared_bytes(const ColiXdnaPrepared *p);
unsigned coli_xdna_prepared_k(const ColiXdnaPrepared *p);
unsigned coli_xdna_prepared_n(const ColiXdnaPrepared *p);
+/* Publication generation: incremented every time this object publishes a VALID
+ * image. Two images can occupy the same address -- retained capacity is reused
+ * in place -- so an address alone does not identify contents. Anything that
+ * caches a view of the prepared bytes must key on (pointer, generation).
+ *
+ * W2-N7-I6 added this because the I5 lane keyed its userptr wrapper on the
+ * pointer alone. Re-preparing a different weight into the same buffer therefore
+ * skipped the re-wrap, and the device -- which snapshots at wrap time -- kept
+ * computing against a view that no longer matched the engine's image. Confirmed
+ * on real XDNA2 hardware, not merely reasoned about. */
+unsigned long long coli_xdna_prepared_generation(const ColiXdnaPrepared *p);
+
/* Defensive validator. The allocator guarantees alignment, but a buffer that
* arrives from elsewhere, from a pool, or at an offset does not -- and a
* misaligned pointer fails at the XRT boundary with a message about video
@@ -347,13 +359,21 @@ typedef enum {
COLI_XDNA_HARD_SHAPE_UNSUPPORTED,
COLI_XDNA_HARD_FORMAT_UNSUPPORTED, /* stored tensor is not fmt=4 */
COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED, /* fmt=4 but gs is not the qualified 64 */
- /* artifact gates */
- COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED, /* any non-QUALIFIED static verdict */
+ /* artifact gates -- kept distinct because they mean different things
+ * operationally: "this build does not ship that artifact", "the bytes are
+ * not the bytes that were qualified", and "we never qualified this one"
+ * lead to different actions, and collapsing them loses the difference
+ * exactly where an operator needs it most. */
+ COLI_XDNA_HARD_ARTIFACT_UNAVAILABLE,
+ COLI_XDNA_HARD_ARTIFACT_INTEGRITY_FAILED,
+ COLI_XDNA_HARD_ARTIFACT_UNQUALIFIED,
+ COLI_XDNA_HARD_REGISTRY_INVALID,
/* representation gates */
COLI_XDNA_HARD_PREPARED_INVALID,
COLI_XDNA_HARD_ALIGNMENT_INVALID,
/* runtime gates */
- COLI_XDNA_HARD_HELPER_UNAVAILABLE,
+ COLI_XDNA_HARD_HELPER_UNAVAILABLE, /* absent, or would not load */
+ COLI_XDNA_HARD_HELPER_ABI_INCOMPATIBLE, /* loaded, wrong generation or incomplete */
COLI_XDNA_HARD_DEVICE_UNAVAILABLE,
COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE,
COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE
@@ -395,6 +415,24 @@ typedef enum {
COLI_XDNA_EXEC_COMPLETION_FAILED /* dispatched, did not complete cleanly */
} ColiXdnaExec;
+/* Lane health. The narrowest model the observed failures justify, and no retry,
+ * backoff or quarantine policy beyond it.
+ *
+ * Most failures are SINGLE-OPERATION: a wrap, a dispatch or a completion can
+ * fail without saying anything about the next operation. A device that will not
+ * initialise is different -- the sealed A5 taxonomy classifies it PROCESS-scoped,
+ * and re-attempting it per operation is pure cost -- so it marks the lane
+ * unavailable for the remainder of the process. Loader verdicts are already
+ * sticky and are reported through ColiXdnaBinding, not here. */
+typedef enum {
+ COLI_XDNA_LANE_UNINITIALIZED = 0, /* nothing has been attempted yet */
+ COLI_XDNA_LANE_HEALTHY, /* usable, or at worst failed per-operation */
+ COLI_XDNA_LANE_UNAVAILABLE /* device unusable for this process */
+} ColiXdnaLaneHealth;
+
+ColiXdnaLaneHealth coli_xdna_lane_health(void);
+const char *coli_xdna_lane_health_text(ColiXdnaLaneHealth h);
+
const char *coli_xdna_exec_text(ColiXdnaExec e);
/* The production candidate, called from the GLM shared gate/up sites and
@@ -427,6 +465,41 @@ int coli_xdna_try_matmul(ColiXdnaFamily family,
* when nothing was ever initialised, and safe to repeat. */
void coli_xdna_execution_shutdown(void);
+/* -- the two internal execution modes -------------------------------------
+ *
+ * AF0 froze two conceptual modes and I6 defines both mechanically. Neither is
+ * public: there is no --xdna flag and no COLI_XDNA variable, and the AUTO-like
+ * mode is additionally inert unless the internal force control is set.
+ *
+ * AUTO-LIKE coli_xdna_try_matmul(). Any decline or failure returns 0 and
+ * the caller runs its current path. This is the production seam.
+ *
+ * EXPLICIT coli_xdna_test_attempt(). Returns the failure class instead of
+ * a handled flag, and implies NO fallback: a caller in this mode
+ * asked for XDNA specifically and gets a classified failure
+ * rather than a silent substitution. Deliberately a separate
+ * entry point rather than a mode flag, so no global state can
+ * leave the production seam in a no-fallback configuration.
+ *
+ * Both share one implementation, so their gate order and classification cannot
+ * drift apart. */
+ColiXdnaExec coli_xdna_test_attempt(ColiXdnaFamily family,
+ ColiXdnaPrepared **slot,
+ int fmt, const unsigned char *q4, const float *scale,
+ int I, int O, int gs,
+ float *y, const float *x, int S);
+
+/* 1 only after an attempt reported successful completion. Any failure -- at any
+ * stage, including one that occurs after the helper has already written output
+ * bytes -- leaves this 0. No property of the bytes themselves (finite, non-NaN,
+ * non-poison, plausible) can raise it: validity is a state, not a measurement. */
+int coli_xdna_test_output_valid(void);
+
+/* The lane's output staging buffer, so a test can observe that a failing helper
+ * really did write real bytes into it before the failure was reported. Never
+ * read by production code, which copies from it only after success. */
+const float *coli_xdna_test_output_staging(size_t *floats);
+
/* Install a registry for tests; NULL restores the production table. */
void coli_xdna_test_set_registry(const ColiXdnaArtifact *rows, int count);
@@ -458,6 +531,12 @@ int coli_xdna_test_artifact_opens(void);
int coli_xdna_test_activation_preparations(void);
int coli_xdna_test_padded_operations(void);
int coli_xdna_test_fallbacks(void); /* candidate returned 0 */
+/* Wrapper accounting. stale_rejects counts the case that made I6 necessary:
+ * the same address carrying a NEW image, which must force a re-wrap rather
+ * than be reused. */
+int coli_xdna_test_wrapper_reuses(void);
+int coli_xdna_test_wrapper_releases(void);
+int coli_xdna_test_stale_wrapper_rejects(void);
/* Deterministic mid-conversion fault injection, for tests only. 0 disables it;
* 1..99 fails after approximately that percentage of the conversion's rows.
diff --git a/c/tests/test_xdna_execution.c b/c/tests/test_xdna_execution.c
index 7c1585f6..f4b569a0 100644
--- a/c/tests/test_xdna_execution.c
+++ b/c/tests/test_xdna_execution.c
@@ -319,10 +319,16 @@ int main(int argc, char **argv){
/* ================================================================== */
printf("helper absent / ABI 1 / incomplete ABI 2\n");
{
- struct { const char *path, *what; } hs[] = {
- { "tests/definitely_no_such_helper.dll", "helper absent" },
- { g_helper_abi1, "ABI generation 1 helper" },
- { g_helper_partial, "incomplete ABI 2 helper" }
+ /* Since W2-N7-I6 absence and ABI failure are different verdicts: one
+ * says this machine simply has no optional lane, the other says a
+ * helper is installed and something about it is wrong. */
+ struct { const char *path, *what; ColiXdnaHard want; } hs[] = {
+ { "tests/definitely_no_such_helper.dll", "helper absent",
+ COLI_XDNA_HARD_HELPER_UNAVAILABLE },
+ { g_helper_abi1, "ABI generation 1 helper",
+ COLI_XDNA_HARD_HELPER_ABI_INCOMPATIBLE },
+ { g_helper_partial, "incomplete ABI 2 helper",
+ COLI_XDNA_HARD_HELPER_ABI_INCOMPATIBLE }
};
for(size_t c = 0; c < 3; c++){
use_helper(hs[c].path);
@@ -337,8 +343,8 @@ int main(int argc, char **argv){
snprintf(msg,sizeof msg,"%s -> not handled", hs[c].what); ck(handled==0,msg);
snprintf(msg,sizeof msg,"%s -> dispatches 0", hs[c].what);
ck(coli_xdna_test_dispatches()==0,msg);
- snprintf(msg,sizeof msg,"%s -> HELPER_UNAVAILABLE", hs[c].what);
- ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_HELPER_UNAVAILABLE,msg);
+ snprintf(msg,sizeof msg,"%s -> %s", hs[c].what, coli_xdna_hard_text(hs[c].want));
+ ck(coli_xdna_test_last_hard()==hs[c].want,msg);
snprintf(msg,sizeof msg,"%s -> no entry point bound", hs[c].what);
ck(coli_xdna_test_entry_points_bound()==0,msg);
snprintf(msg,sizeof msg,"%s -> output untouched", hs[c].what);
@@ -369,10 +375,15 @@ int main(int argc, char **argv){
int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&slot,
4,W.q4,W.s,TK,TN,64,y,x,TM);
const char *what = c==0 ? "artifact absent" : "artifact hash mismatch";
+ /* Since W2-N7-I6 these are DIFFERENT verdicts: a build that does not
+ * ship the bytes and a build whose bytes were tampered with call for
+ * different responses. */
+ ColiXdnaHard want = c==0 ? COLI_XDNA_HARD_ARTIFACT_UNAVAILABLE
+ : COLI_XDNA_HARD_ARTIFACT_INTEGRITY_FAILED;
char msg[160];
snprintf(msg,sizeof msg,"%s -> not handled", what); ck(handled==0,msg);
- snprintf(msg,sizeof msg,"%s -> ARTIFACT_NOT_QUALIFIED", what);
- ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_ARTIFACT_NOT_QUALIFIED,msg);
+ snprintf(msg,sizeof msg,"%s -> %s", what, coli_xdna_hard_text(want));
+ ck(coli_xdna_test_last_hard()==want,msg);
snprintf(msg,sizeof msg,"%s -> helper never called", what);
ck(coli_xdna_test_helper_calls()==0,msg);
snprintf(msg,sizeof msg,"%s -> unverified bytes never reached the helper", what);
diff --git a/c/tests/test_xdna_failure.c b/c/tests/test_xdna_failure.c
new file mode 100644
index 00000000..55e1343f
--- /dev/null
+++ b/c/tests/test_xdna_failure.c
@@ -0,0 +1,686 @@
+/* W2-N7-I6 -- XDNA failure classification, fallback exactness and late-output
+ * invalidation.
+ *
+ * The invariant this file exists to defend:
+ *
+ * Optional XDNA work may fail. Current Colibri operation semantics may not.
+ *
+ * So every case here poisons the caller output, drives the production candidate
+ * into a failure, runs the EXACT current path the production seam runs, and
+ * requires the result to equal a matmul_qt reference with zero residual poison.
+ *
+ * A new owner rather than an extension of an existing one: this is the only
+ * place that needs BOTH the engine (for matmul_qt, the fallback target) and the
+ * synthetic helper fixtures (for the failure stages). test_xdna_execution.c has
+ * the helper but no engine; test_xdna_qt_state.c has the engine but no helper.
+ *
+ * Hardware-free. The synthetic ABI-2 helper contains no XRT and touches no
+ * device, and it models the real helper faithfully enough to matter -- it
+ * snapshots the weight at wrap time, exactly as sync(BO_TO_DEVICE) does.
+ */
+
+#define main coli_glm_main_unused
+#include "../colibri.c"
+#undef main
+
+#include "../backend_xdna.h"
+
+static int g_fail = 0;
+static int g_checks = 0;
+static void ck(int cond, const char *what){
+ g_checks++;
+ if(!cond){ printf(" FAIL %s\n", what); g_fail = 1; }
+ else printf(" ok %s\n", what);
+}
+
+/* ---- fixtures ----------------------------------------------------------- */
+
+#define TK 256
+#define TN 128
+#define TM 64
+
+/* Failure stages the synthetic helper can be told to produce. Mirrors the enum
+ * in tests/xdna_fake_helper.c. */
+enum { F_NONE = 0, F_DEVICE, F_ARTIFACT, F_WRAP, F_EXECUTE, F_COMPLETION };
+
+static char g_root[1024], g_helper[1024], g_helper_abi1[1024], g_helper_partial[1024];
+static char g_xclbin[2048], g_insts[2048];
+static char g_xhex[65], g_ihex[65];
+static ColiXdnaArtifact g_test_rows[1];
+static HMODULE g_fake;
+static void (*p_set_fail)(int);
+static void (*p_reset)(void);
+
+#define POISON (-987654.0f)
+
+static void hexify(const unsigned char h[32], char out[65]){
+ static const char *H = "0123456789abcdef";
+ for(int i = 0; i < 32; i++){ out[i*2] = H[h[i]>>4]; out[i*2+1] = H[h[i]&15]; }
+ out[64] = '\0';
+}
+static int write_blob(const char *path, unsigned seed, size_t n){
+ FILE *f = fopen(path, "wb");
+ if(!f) return 0;
+ unsigned st = seed;
+ for(size_t i = 0; i < n; i++){ st = st*1664525u+1013904223u; fputc((int)(st>>24) & 0xFF, f); }
+ fclose(f); return 1;
+}
+static void build_registry(void){
+ unsigned char h[32];
+ if(!coli_xdna_sha256_file(g_xclbin, h)){ printf(" FAIL fixture unreadable\n"); g_fail=1; return; }
+ hexify(h, g_xhex);
+ if(!coli_xdna_sha256_file(g_insts, h)){ printf(" FAIL fixture unreadable\n"); g_fail=1; return; }
+ hexify(h, g_ihex);
+ g_test_rows[0].family = COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP;
+ g_test_rows[0].research_family = "F3";
+ g_test_rows[0].artifact_m = TM;
+ g_test_rows[0].k = TK; g_test_rows[0].n = TN;
+ g_test_rows[0].in_dtype = COLI_XDNA_DT_BF16;
+ g_test_rows[0].weight_dtype = COLI_XDNA_DT_BF16;
+ g_test_rows[0].out_dtype = COLI_XDNA_DT_F32;
+ g_test_rows[0].target = COLI_XDNA_TARGET_XDNA2;
+ g_test_rows[0].xclbin_name = "fake.xclbin";
+ g_test_rows[0].xclbin_sha256 = g_xhex;
+ g_test_rows[0].insts_name = "fake_insts.bin";
+ g_test_rows[0].insts_sha256 = g_ihex;
+ g_test_rows[0].runtime_weight_qualified = 1;
+ g_test_rows[0].correctness_qualified = 1;
+ g_test_rows[0].userptr_qualified = 1;
+ g_test_rows[0].structural_qualified = 1;
+ coli_xdna_test_set_registry(g_test_rows, 1);
+}
+
+static void qt_make(QT *t, int I, int O, int gs, unsigned seed){
+ memset(t, 0, sizeof *t);
+ int rb = (I+1)/2, ng = (I+gs-1)/gs;
+ t->fmt = 4; t->I = I; t->O = O; t->gs = gs;
+ t->q4 = (uint8_t*)malloc((size_t)O*rb);
+ t->s = (float*)malloc((size_t)O*ng*sizeof(float));
+ unsigned st = seed;
+ for(size_t i = 0; i < (size_t)O*rb; i++){ st = st*1664525u+1013904223u; t->q4[i] = (uint8_t)(st>>24); }
+ for(size_t i = 0; i < (size_t)O*ng; i++){ st = st*1664525u+1013904223u;
+ t->s[i] = 0.25f*(float)(1 + ((st>>26)&3)); }
+}
+static float *mk_x(int S, int K, unsigned seed){
+ float *x = (float*)malloc((size_t)S*K*sizeof(float));
+ unsigned st = seed;
+ for(size_t i = 0; i < (size_t)S*K; i++){ st = st*1664525u+1013904223u;
+ x[i] = (float)((int)((st>>20)&255)-128)*0.0078125f; }
+ return x;
+}
+static unsigned short f2b_t(float f){
+ unsigned int u; memcpy(&u,&f,4);
+ u += 0x7FFFu + ((u>>16)&1u);
+ return (unsigned short)(u>>16);
+}
+static void poison_buf(float *y, size_t n){ for(size_t i=0;i<n;i++) y[i]=POISON; }
+static size_t count_poison(const float *y, size_t n){
+ size_t c=0; for(size_t i=0;i<n;i++) if(y[i]==POISON) c++; return c;
+}
+static unsigned long long fnv(const void *p, size_t n){
+ unsigned long long h = 1469598103934665603ULL;
+ for(size_t i=0;i<n;i++){ h ^= ((const unsigned char*)p)[i]; h *= 1099511628211ULL; }
+ return h;
+}
+
+/* Arm a failure stage. Deliberately does NOT reset the fake: reset clears its
+ * open state, which would desynchronise it from a host lane that still believes
+ * the artifact is open. Reset happens once, in use_helper, where the host lane
+ * has just been torn down too. */
+static void arm(int stage){ if(p_set_fail) p_set_fail(stage); }
+
+static void use_helper(const char *path){
+ coli_xdna_execution_shutdown();
+ coli_xdna_test_set_force_execution(0);
+ coli_xdna_test_set_helper_path(path);
+ g_fake = NULL; p_set_fail = NULL; p_reset = NULL;
+ if(coli_xdna_binding() == COLI_XDNA_AVAILABLE){
+ g_fake = GetModuleHandleA(path);
+ if(!g_fake){
+ const char *base = strrchr(path, '\\');
+ const char *fw = strrchr(path, '/');
+ if(fw && (!base || fw > base)) base = fw;
+ g_fake = GetModuleHandleA(base ? base+1 : path);
+ }
+ if(g_fake){
+ p_set_fail = (void(*)(int))(void*)GetProcAddress(g_fake, "fake_set_fail");
+ p_reset = (void(*)(void))(void*)GetProcAddress(g_fake, "fake_reset");
+ if(p_reset) p_reset(); /* host lane and fake start together */
+ }
+ }
+}
+
+/* ---- the production seam, reproduced exactly ----------------------------
+ *
+ * This is the shape of the two GLM shared gate/up sites, character for
+ * character in structure: try the lane, and if it does not handle the
+ * operation run the SAME matmul_qt call that stood there before the lane
+ * existed. Nothing else is a fallback. */
+static void seam(float *y, const float *x, QT *w, int S){
+ if(!coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &w->xdna,
+ w->fmt, w->q4, w->s, w->I, w->O, w->gs, y, x, S))
+ matmul_qt(y, x, w, S);
+}
+
+/* Drive one failure and require the exact current-path result. */
+static void expect_exact_fallback(const char *what, QT *w, const float *x, int S,
+ const float *ref){
+ float *y = (float*)malloc((size_t)S*w->O*4);
+ poison_buf(y, (size_t)S*w->O);
+ int d0 = coli_xdna_test_dispatches();
+ seam(y, x, w, S);
+ char m[220];
+ snprintf(m,sizeof m,"%s -> no successful dispatch", what);
+ ck(coli_xdna_test_dispatches()==d0, m);
+ snprintf(m,sizeof m,"%s -> XDNA output not valid", what);
+ ck(coli_xdna_test_output_valid()==0, m);
+ snprintf(m,sizeof m,"%s -> residual poison 0", what);
+ ck(count_poison(y,(size_t)S*w->O)==0, m);
+ snprintf(m,sizeof m,"%s -> result is bit-identical to the current path", what);
+ ck(memcmp(y, ref, (size_t)S*w->O*4)==0, m);
+ free(y);
+}
+
+int main(int argc, char **argv){
+ char dir[512]; snprintf(dir, sizeof dir, "%s", argc>0?argv[0]:"tests/x");
+ { char *sl = strrchr(dir,'\\'); char *fw = strrchr(dir,'/');
+ if(fw && (!sl || fw>sl)) sl = fw;
+ if(sl) *sl = '\0'; else snprintf(dir,sizeof dir,"."); }
+
+ snprintf(g_root, sizeof g_root, "%s/xdna_fail_fixtures", dir);
+ snprintf(g_helper, sizeof g_helper, "%s/xdna_fake_helper.dll", dir);
+ snprintf(g_helper_abi1, sizeof g_helper_abi1, "%s/xdna_fake_helper_abi1.dll", dir);
+ snprintf(g_helper_partial, sizeof g_helper_partial, "%s/xdna_fake_helper_partial.dll", dir);
+ { char cmd[1200]; snprintf(cmd,sizeof cmd,"mkdir \"%s\" 2>nul", g_root);
+ for(char *p=cmd;*p;p++) if(*p=='/') *p='\\';
+ if(system(cmd)){} }
+ snprintf(g_xclbin, sizeof g_xclbin, "%s/fake.xclbin", g_root);
+ snprintf(g_insts, sizeof g_insts, "%s/fake_insts.bin", g_root);
+ if(!write_blob(g_xclbin, 31u, 4096) || !write_blob(g_insts, 41u, 1024)){
+ printf("FAIL fixtures\n"); return 1; }
+ build_registry();
+ if(g_fail) return 1;
+
+ QT W; qt_make(&W, TK, TN, 64, 4242u);
+ QT W2; qt_make(&W2, TK, TN, 64, 8888u);
+ float *x = mk_x(TM, TK, 77u);
+
+ /* The current-path reference, computed once, by the same call the seam
+ * falls back to. Every failure case must reproduce this exactly. */
+ float *REF = (float*)malloc((size_t)TM*TN*4);
+ matmul_qt(REF, x, &W, TM);
+ unsigned long long ref_hash = fnv(REF, (size_t)TM*TN*4);
+ printf("current-path reference hash %016llx\n", ref_hash);
+
+ /* fmt4 source fingerprints, so "the authoritative weight is untouched" is
+ * a measurement rather than an assumption. */
+ const size_t q4b = (size_t)TN*((TK+1)/2), scb = (size_t)TN*((TK+63)/64)*4;
+
+ /* ================================================================== */
+ printf("\npre-device declines -- nothing may reach the device\n");
+ {
+ struct { const char *what; ColiXdnaFamily fam; int fmt,I,O,gs,S; ColiXdnaHard want; } c[] = {
+ { "wrong semantic family", COLI_XDNA_FAMILY_NONE, 4,TK,TN,64,TM, COLI_XDNA_HARD_FAMILY_UNSUPPORTED },
+ { "sh_down orientation", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4,TN,TK,64,TM, COLI_XDNA_HARD_SHAPE_UNSUPPORTED },
+ { "gs != 64", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4,TK,TN,32,TM, COLI_XDNA_HARD_GROUP_SIZE_UNSUPPORTED },
+ { "fmt != 4", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 2,TK,TN,64,TM, COLI_XDNA_HARD_FORMAT_UNSUPPORTED },
+ { "M above range", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4,TK,TN,64,TM+1, COLI_XDNA_HARD_M_OUT_OF_RANGE },
+ { "K/N mismatch", COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, 4,TK+8,TN,64,TM, COLI_XDNA_HARD_SHAPE_UNSUPPORTED }
+ };
+ for(size_t i = 0; i < sizeof c/sizeof c[0]; i++){
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ QT T; qt_make(&T, c[i].I, c[i].O, c[i].gs, 11u);
+ T.fmt = c[i].fmt;
+ float *xx = mk_x(c[i].S, c[i].I, 12u);
+ float *yy = (float*)malloc((size_t)c[i].S*c[i].O*4);
+ float *rr = (float*)malloc((size_t)c[i].S*c[i].O*4);
+ T.fmt = 4; matmul_qt(rr, xx, &T, c[i].S); T.fmt = c[i].fmt;
+ poison_buf(yy,(size_t)c[i].S*c[i].O);
+ int handled = coli_xdna_try_matmul(c[i].fam, &T.xdna, c[i].fmt, T.q4, T.s,
+ c[i].I, c[i].O, c[i].gs, yy, xx, c[i].S);
+ char m[200];
+ snprintf(m,sizeof m,"%s -> %s", c[i].what, coli_xdna_hard_text(coli_xdna_test_last_hard()));
+ ck(coli_xdna_test_last_hard()==c[i].want, m);
+ snprintf(m,sizeof m,"%s -> not handled", c[i].what); ck(handled==0,m);
+ snprintf(m,sizeof m,"%s -> device opens 0", c[i].what); ck(coli_xdna_test_device_opens()==0,m);
+ snprintf(m,sizeof m,"%s -> artifact runtime opens 0", c[i].what); ck(coli_xdna_test_artifact_opens()==0,m);
+ snprintf(m,sizeof m,"%s -> userptr wraps 0", c[i].what); ck(coli_xdna_test_userptr_wraps()==0,m);
+ snprintf(m,sizeof m,"%s -> dispatches 0", c[i].what); ck(coli_xdna_test_dispatches()==0,m);
+ /* the fallback still produces the current-path answer */
+ T.fmt = 4;
+ matmul_qt(yy, xx, &T, c[i].S);
+ snprintf(m,sizeof m,"%s -> current path recomputes exactly", c[i].what);
+ ck(memcmp(yy,rr,(size_t)c[i].S*c[i].O*4)==0, m);
+ free(xx); free(yy); free(rr);
+ coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+ }
+
+ /* ================================================================== */
+ printf("\nartifact declines -- distinguishable, and refused before the helper\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_force_execution(1);
+ char none[1200]; snprintf(none,sizeof none,"%s/absent", g_root);
+ coli_xdna_test_set_artifact_root(none);
+ float *y = (float*)malloc((size_t)TM*TN*4); poison_buf(y,(size_t)TM*TN);
+ coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&W.xdna,4,W.q4,W.s,TK,TN,64,y,x,TM);
+ ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_ARTIFACT_UNAVAILABLE,
+ "artifact absent -> ARTIFACT_UNAVAILABLE");
+ ck(coli_xdna_test_helper_calls()==0, "artifact absent -> helper never called");
+ expect_exact_fallback("artifact absent", &W, x, TM, REF);
+
+ coli_xdna_test_set_artifact_root(g_root);
+ { FILE *f = fopen(g_xclbin,"r+b"); if(f){ fseek(f,64,SEEK_SET); fputc(0x5A,f); fclose(f); } }
+ coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&W.xdna,4,W.q4,W.s,TK,TN,64,y,x,TM);
+ ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_ARTIFACT_INTEGRITY_FAILED,
+ "artifact tampered -> ARTIFACT_INTEGRITY_FAILED (not the same as absent)");
+ ck(coli_xdna_test_helper_calls()==0, "artifact tampered -> unverified bytes never reach the helper");
+ expect_exact_fallback("artifact tampered", &W, x, TM, REF);
+ write_blob(g_xclbin, 31u, 4096); build_registry();
+
+ g_test_rows[0].correctness_qualified = 0;
+ coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&W.xdna,4,W.q4,W.s,TK,TN,64,y,x,TM);
+ ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_ARTIFACT_UNQUALIFIED,
+ "artifact never correctness-qualified -> ARTIFACT_UNQUALIFIED");
+ g_test_rows[0].correctness_qualified = 1;
+ free(y);
+ }
+
+ /* ================================================================== */
+ printf("\nhelper declines -- absence and ABI failure are different\n");
+ {
+ struct { const char *path, *what; ColiXdnaHard want; } h[] = {
+ { "tests/no_such_helper_at_all.dll", "helper absent", COLI_XDNA_HARD_HELPER_UNAVAILABLE },
+ { g_helper_abi1, "previous ABI generation", COLI_XDNA_HARD_HELPER_ABI_INCOMPATIBLE },
+ { g_helper_partial, "incomplete ABI 2", COLI_XDNA_HARD_HELPER_ABI_INCOMPATIBLE }
+ };
+ for(size_t i = 0; i < 3; i++){
+ use_helper(h[i].path);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *y = (float*)malloc((size_t)TM*TN*4); poison_buf(y,(size_t)TM*TN);
+ coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&T.xdna,4,T.q4,T.s,TK,TN,64,y,x,TM);
+ char m[200];
+ snprintf(m,sizeof m,"%s -> %s", h[i].what, coli_xdna_hard_text(coli_xdna_test_last_hard()));
+ ck(coli_xdna_test_last_hard()==h[i].want, m);
+ snprintf(m,sizeof m,"%s -> no weight prepared (the gate precedes preparation)", h[i].what);
+ ck(T.xdna==NULL, m);
+ snprintf(m,sizeof m,"%s -> load attempted at most once", h[i].what);
+ ck(coli_xdna_test_load_attempts()<=1, m);
+ free(y); coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+ /* stickiness: a second query must not re-attempt the load */
+ use_helper(g_helper_abi1);
+ coli_xdna_binding(); coli_xdna_binding(); coli_xdna_binding();
+ ck(coli_xdna_test_load_attempts()<=1, "loader verdict stays sticky across repeated queries");
+ ck(coli_xdna_test_entry_points_bound()==0, "and nothing is bound");
+ }
+
+ /* ================================================================== */
+ printf("\npreparation failure -- no XRT is reached, fmt4 survives\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ arm(F_NONE);
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *ref = (float*)malloc((size_t)TM*TN*4); matmul_qt(ref, x, &T, TM);
+ coli_xdna_test_set_convert_fail_pct(50);
+ float *y = (float*)malloc((size_t)TM*TN*4); poison_buf(y,(size_t)TM*TN);
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&T.xdna,
+ 4,T.q4,T.s,TK,TN,64,y,x,TM);
+ ck(handled==0, "mid-conversion failure -> not handled");
+ ck(coli_xdna_test_last_exec()==COLI_XDNA_EXEC_WEIGHT_PREPARE_FAILED,
+ "classified WEIGHT_PREPARE_FAILED");
+ ck(coli_xdna_prepared_state(T.xdna)==COLI_XDNA_PREP_INVALID, "state PREPARED_INVALID");
+ ck(coli_xdna_prepared_bytes(T.xdna)>0, "partial BF16 bytes are still allocated");
+ ck(coli_xdna_prepared_image(T.xdna)==NULL, "but the image is not readable");
+ ck(coli_xdna_test_userptr_wraps()==0, "userptr wraps 0 -- no XRT was reached");
+ ck(coli_xdna_test_dispatches()==0, "dispatches 0");
+ ck(coli_xdna_test_device_opens()==0, "device opens 0");
+ ck(fnv(T.q4,q4b)==fnv(W.q4,q4b) && fnv(T.s,scb)==fnv(W.s,scb),
+ "authoritative fmt4 weight and scales unchanged");
+ matmul_qt(y, x, &T, TM);
+ ck(count_poison(y,(size_t)TM*TN)==0, "current path leaves no residual poison");
+ ck(memcmp(y,ref,(size_t)TM*TN*4)==0, "current path result exact");
+ /* a fresh complete preparation recovers */
+ coli_xdna_test_set_convert_fail_pct(0);
+ poison_buf(y,(size_t)TM*TN);
+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&T.xdna,
+ 4,T.q4,T.s,TK,TN,64,y,x,TM)==1,
+ "a complete re-preparation recovers and executes");
+ ck(coli_xdna_prepared_state(T.xdna)==COLI_XDNA_PREP_VALID, "state VALID again");
+ ck(coli_xdna_test_output_valid()==1, "and the output is valid");
+ free(y); free(ref); coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+
+ /* ================================================================== */
+ printf("\nruntime failure stages -- each classified, each falls back exactly\n");
+ {
+ struct { int stage; const char *what; ColiXdnaExec want; ColiXdnaHard hard; } st[] = {
+ { F_DEVICE, "device init", COLI_XDNA_EXEC_DEVICE_INIT_FAILED, COLI_XDNA_HARD_DEVICE_UNAVAILABLE },
+ { F_ARTIFACT, "artifact runtime open", COLI_XDNA_EXEC_ARTIFACT_OPEN_FAILED, COLI_XDNA_HARD_ARTIFACT_RUNTIME_UNAVAILABLE },
+ { F_WRAP, "userptr wrap", COLI_XDNA_EXEC_WEIGHT_WRAP_FAILED, COLI_XDNA_HARD_WEIGHT_WRAP_UNAVAILABLE },
+ { F_EXECUTE, "dispatch", COLI_XDNA_EXEC_EXECUTE_FAILED, COLI_XDNA_HARD_ELIGIBLE },
+ { F_COMPLETION, "completion", COLI_XDNA_EXEC_COMPLETION_FAILED, COLI_XDNA_HARD_ELIGIBLE }
+ };
+ for(size_t i = 0; i < sizeof st/sizeof st[0]; i++){
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ arm(st[i].stage);
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *ref = (float*)malloc((size_t)TM*TN*4); matmul_qt(ref, x, &T, TM);
+ float *y = (float*)malloc((size_t)TM*TN*4); poison_buf(y,(size_t)TM*TN);
+ int d0 = coli_xdna_test_dispatches();
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&T.xdna,
+ 4,T.q4,T.s,TK,TN,64,y,x,TM);
+ char m[220];
+ snprintf(m,sizeof m,"%s failure -> not handled", st[i].what); ck(handled==0,m);
+ snprintf(m,sizeof m,"%s failure -> %s", st[i].what,
+ coli_xdna_exec_text(coli_xdna_test_last_exec()));
+ ck(coli_xdna_test_last_exec()==st[i].want, m);
+ snprintf(m,sizeof m,"%s failure -> hard verdict %s", st[i].what,
+ coli_xdna_hard_text(coli_xdna_test_last_hard()));
+ ck(coli_xdna_test_last_hard()==st[i].hard, m);
+ snprintf(m,sizeof m,"%s failure -> output not valid", st[i].what);
+ ck(coli_xdna_test_output_valid()==0, m);
+ snprintf(m,sizeof m,"%s failure -> caller output untouched by the lane", st[i].what);
+ ck(count_poison(y,(size_t)TM*TN)==(size_t)TM*TN, m);
+ snprintf(m,sizeof m,"%s failure -> no successful dispatch", st[i].what);
+ ck(coli_xdna_test_dispatches()==d0, m);
+ /* the prepared image is a separate concern from runtime success */
+ if(st[i].stage != F_DEVICE && st[i].stage != F_ARTIFACT){
+ snprintf(m,sizeof m,"%s failure -> a correct prepared image stays VALID", st[i].what);
+ ck(coli_xdna_prepared_state(T.xdna)==COLI_XDNA_PREP_VALID, m);
+ }
+ snprintf(m,sizeof m,"%s failure -> authoritative fmt4 unchanged", st[i].what);
+ ck(fnv(T.q4,q4b)==fnv(W.q4,q4b) && fnv(T.s,scb)==fnv(W.s,scb), m);
+ /* and the seam recomputes exactly */
+ matmul_qt(y, x, &T, TM);
+ snprintf(m,sizeof m,"%s failure -> residual poison 0", st[i].what);
+ ck(count_poison(y,(size_t)TM*TN)==0, m);
+ snprintf(m,sizeof m,"%s failure -> current path result exact", st[i].what);
+ ck(memcmp(y,ref,(size_t)TM*TN*4)==0, m);
+ free(y); free(ref); coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+ }
+
+ /* ================================================================== */
+ printf("\nlate output failure -- the helper wrote real bytes, then failed\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ arm(F_COMPLETION);
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *ref = (float*)malloc((size_t)TM*TN*4); matmul_qt(ref, x, &T, TM);
+ float *y = (float*)malloc((size_t)TM*TN*4); poison_buf(y,(size_t)TM*TN);
+
+ int d_late = coli_xdna_test_dispatches();
+ int handled = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&T.xdna,
+ 4,T.q4,T.s,TK,TN,64,y,x,TM);
+ ck(handled==0, "not handled");
+ ck(coli_xdna_test_last_exec()==COLI_XDNA_EXEC_COMPLETION_FAILED, "COMPLETION_FAILED");
+
+ /* Prove the helper really did write output before failing: the staging
+ * buffer holds a real result, not zeros and not poison. */
+ size_t nfl = 0;
+ const float *stg = coli_xdna_test_output_staging(&nfl);
+ size_t nonzero = 0, finite = 0;
+ if(stg) for(size_t i = 0; i < (size_t)TM*TN; i++){
+ if(stg[i] != 0.0f) nonzero++;
+ if(stg[i] == stg[i]) finite++;
+ }
+ ck(stg != NULL, "the lane staging buffer exists");
+ ck(nonzero > (size_t)TM*TN/2, "the helper genuinely wrote real output bytes before failing");
+ ck(finite == (size_t)TM*TN, "and those bytes are finite -- plausibility is not validity");
+
+ ck(coli_xdna_test_output_valid()==0, "the written output is still INVALID");
+ ck(count_poison(y,(size_t)TM*TN)==(size_t)TM*TN,
+ "not one of those bytes reached the caller");
+ ck(coli_xdna_test_dispatches()==d_late, "not counted as a dispatch");
+
+ matmul_qt(y, x, &T, TM);
+ ck(count_poison(y,(size_t)TM*TN)==0, "current path overwrote every logical element");
+ ck(memcmp(y,ref,(size_t)TM*TN*4)==0, "and produced the exact current-path result");
+ ck(fnv(ref,(size_t)TM*TN*4)==fnv(REF,(size_t)TM*TN*4) || T.q4!=W.q4,
+ "reference is the matmul_qt answer for this tensor");
+ free(y); free(ref); coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+
+ /* ================================================================== */
+ printf("\ninternal explicit mode -- classified failure, no silent substitution\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(0); /* explicit does not need the force control */
+ arm(F_EXECUTE);
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *y = (float*)malloc((size_t)TM*TN*4); poison_buf(y,(size_t)TM*TN);
+ ColiXdnaExec e = coli_xdna_test_attempt(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&T.xdna,
+ 4,T.q4,T.s,TK,TN,64,y,x,TM);
+ ck(e==COLI_XDNA_EXEC_EXECUTE_FAILED, "explicit mode returns the failure class");
+ ck(coli_xdna_test_output_valid()==0, "output not valid");
+ ck(count_poison(y,(size_t)TM*TN)==(size_t)TM*TN,
+ "and no fallback happened -- the caller output is untouched");
+ arm(F_NONE);
+ poison_buf(y,(size_t)TM*TN);
+ e = coli_xdna_test_attempt(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&T.xdna,
+ 4,T.q4,T.s,TK,TN,64,y,x,TM);
+ ck(e==COLI_XDNA_EXEC_OK, "explicit mode succeeds when the lane works");
+ ck(coli_xdna_test_output_valid()==1, "and the output is valid");
+ ck(count_poison(y,(size_t)TM*TN)==0, "and was written");
+ /* the AUTO-like seam remains inert without the force control */
+ arm(F_NONE);
+ poison_buf(y,(size_t)TM*TN);
+ int d0 = coli_xdna_test_dispatches();
+ seam(y, x, &T, TM);
+ ck(coli_xdna_test_dispatches()==d0,
+ "the production seam is still inert without the internal control");
+ free(y); coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+
+ /* ================================================================== */
+ printf("\nlane health -- device failure is process-scoped, the rest is not\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ ck(coli_xdna_lane_health()==COLI_XDNA_LANE_UNINITIALIZED, "lane starts UNINITIALIZED");
+ arm(F_WRAP);
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *y = (float*)malloc((size_t)TM*TN*4);
+ coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&T.xdna,4,T.q4,T.s,TK,TN,64,y,x,TM);
+ ck(coli_xdna_lane_health()==COLI_XDNA_LANE_HEALTHY,
+ "a wrap failure leaves the lane HEALTHY -- it is one operation");
+ arm(F_NONE);
+ ck(coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&T.xdna,4,T.q4,T.s,TK,TN,64,y,x,TM)==1,
+ "and the very next operation succeeds");
+
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ arm(F_DEVICE);
+ QT D; qt_make(&D, TK, TN, 64, 4242u);
+ coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&D.xdna,4,D.q4,D.s,TK,TN,64,y,x,TM);
+ ck(coli_xdna_lane_health()==COLI_XDNA_LANE_UNAVAILABLE,
+ "a device-init failure marks the lane UNAVAILABLE for the process");
+ arm(F_NONE);
+ int o0 = coli_xdna_test_device_opens();
+ coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP,&D.xdna,4,D.q4,D.s,TK,TN,64,y,x,TM);
+ ck(coli_xdna_test_device_opens()==o0,
+ "and the device is not re-attempted per operation");
+ ck(coli_xdna_test_last_hard()==COLI_XDNA_HARD_DEVICE_UNAVAILABLE, "the decline says why");
+ coli_xdna_execution_shutdown();
+ ck(coli_xdna_lane_health()==COLI_XDNA_LANE_UNINITIALIZED,
+ "a full lane teardown is the one thing that clears it");
+ free(y);
+ coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ coli_xdna_prepared_release(&D.xdna); free(D.q4); free(D.s);
+ }
+
+ /* ================================================================== */
+ printf("\nwrapper lifetime\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ arm(F_NONE);
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *y = (float*)malloc((size_t)TM*TN*4);
+
+ int wbase = coli_xdna_test_userptr_wraps();
+ int rubase = coli_xdna_test_wrapper_reuses();
+ seam(y, x, &T, TM);
+ int w1 = coli_xdna_test_userptr_wraps();
+ ck(w1==wbase+1, "one wrapper for the first operation");
+ seam(y, x, &T, TM);
+ seam(y, x, &T, TM);
+ ck(coli_xdna_test_userptr_wraps()==w1, "reused across operations, not recreated");
+ ck(coli_xdna_test_wrapper_reuses()==rubase+2, "and the reuse is counted");
+
+ /* same address, NEW image: the defect this slice fixed */
+ const void *p1 = coli_xdna_prepared_image(T.xdna);
+ unsigned long long g1 = coli_xdna_prepared_generation(T.xdna);
+ int wr0 = coli_xdna_test_wrapper_releases();
+ coli_xdna_prepared_invalidate(T.xdna);
+ ck(coli_xdna_prepare_from_fmt4(T.xdna,4,W2.q4,W2.s,TK,TN,64)==COLI_XDNA_PREP_OK,
+ "a different weight re-prepares into the retained buffer");
+ const void *p2 = coli_xdna_prepared_image(T.xdna);
+ ck(p1==p2, "and lands on the same address");
+ ck(coli_xdna_prepared_generation(T.xdna)!=g1, "but a new generation");
+ seam(y, x, &T, TM);
+ ck(coli_xdna_test_userptr_wraps()==w1+1,
+ "the wrapper is REBUILT, because the address is not an identity");
+ /* Invalidating the image already dropped the wrapper: the bytes stopped
+ * being authoritative at that moment, so no device-side view of them may
+ * stay current either. */
+ ck(coli_xdna_test_wrapper_releases()>wr0,
+ "and the previous wrapper was released when the image was invalidated");
+
+ /* and the result is the NEW weight's answer, not the old one */
+ float *want = (float*)malloc((size_t)TM*TN*4);
+ {
+ const unsigned short *Wi = (const unsigned short*)coli_xdna_prepared_image(T.xdna);
+ for(int i=0;i<TM;i++) for(int j=0;j<TN;j++){
+ float a=0;
+ for(int k=0;k<TK;k++){
+ unsigned int u=(unsigned int)f2b_t(x[(size_t)i*TK+k])<<16; float xv; memcpy(&xv,&u,4);
+ unsigned int v=(unsigned int)Wi[(size_t)k*TN+j]<<16; float wv; memcpy(&wv,&v,4);
+ a+=xv*wv; }
+ want[(size_t)i*TN+j]=a; }
+ }
+ ck(memcmp(y,want,(size_t)TM*TN*4)==0,
+ "the device computed the NEW image, not a stale view of the old one");
+ free(want);
+
+ /* freeing engine memory must drop the wrapper first */
+ int wr1 = coli_xdna_test_wrapper_releases();
+ coli_xdna_prepared_free_buffer(T.xdna);
+ ck(coli_xdna_test_wrapper_releases()>wr1,
+ "freeing the prepared buffer releases the wrapper that borrowed it");
+ free(y); coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+
+ /* ================================================================== */
+ printf("\nsuccess -> failure -> success in one process\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ arm(F_NONE);
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *ref = (float*)malloc((size_t)TM*TN*4); matmul_qt(ref, x, &T, TM);
+ float *y1 = (float*)malloc((size_t)TM*TN*4);
+ float *y2 = (float*)malloc((size_t)TM*TN*4);
+ float *y3 = (float*)malloc((size_t)TM*TN*4);
+ int opens0 = coli_xdna_test_artifact_opens();
+
+ poison_buf(y1,(size_t)TM*TN); seam(y1, x, &T, TM);
+ ck(coli_xdna_test_output_valid()==1, "first operation succeeds on the lane");
+ unsigned long long h1 = fnv(y1,(size_t)TM*TN*4);
+
+ arm(F_COMPLETION);
+ poison_buf(y2,(size_t)TM*TN); seam(y2, x, &T, TM);
+ ck(coli_xdna_test_output_valid()==0, "middle operation fails");
+ ck(count_poison(y2,(size_t)TM*TN)==0, "and the current path filled it");
+ ck(memcmp(y2,ref,(size_t)TM*TN*4)==0, "exactly");
+
+ arm(F_NONE);
+ poison_buf(y3,(size_t)TM*TN); seam(y3, x, &T, TM);
+ ck(coli_xdna_test_output_valid()==1, "third operation succeeds again");
+ ck(fnv(y3,(size_t)TM*TN*4)==h1, "with the same result as the first -- no stale state");
+ ck(coli_xdna_test_artifact_opens()==opens0+1,
+ "no helper reload was required by these failure classes");
+ ck(coli_xdna_prepared_live_objects()>0, "prepared state survived the cycle");
+ free(y1); free(y2); free(y3); free(ref);
+ coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+
+ /* ================================================================== */
+ printf("\nrepeated failure stability\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ coli_xdna_test_set_force_execution(1);
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *ref = (float*)malloc((size_t)TM*TN*4); matmul_qt(ref, x, &T, TM);
+ float *y = (float*)malloc((size_t)TM*TN*4);
+ int stages[] = { F_WRAP, F_EXECUTE, F_COMPLETION, F_NONE };
+ size_t bytes_mid = 0; int objs_mid = 0, wraps_mid = 0;
+ int bad = 0;
+ for(int cycle = 0; cycle < 200; cycle++){
+ arm(stages[cycle & 3]);
+ poison_buf(y,(size_t)TM*TN);
+ seam(y, x, &T, TM);
+ if(stages[cycle & 3] != F_NONE && memcmp(y,ref,(size_t)TM*TN*4)!=0) bad = 1;
+ if(count_poison(y,(size_t)TM*TN)!=0) bad = 1;
+ if(cycle == 99){ bytes_mid = coli_xdna_prepared_total_bytes();
+ objs_mid = coli_xdna_prepared_live_objects();
+ wraps_mid = coli_xdna_test_userptr_wraps(); }
+ }
+ ck(bad==0, "200 mixed failure/success cycles, every fallback exact and fully overwritten");
+ ck(coli_xdna_prepared_total_bytes()==bytes_mid, "prepared host bytes do not grow");
+ ck(coli_xdna_prepared_live_objects()==objs_mid, "prepared objects do not grow");
+ ck(coli_xdna_test_userptr_wraps()-wraps_mid <= 100,
+ "wrapper creations stay bounded by the failures that invalidate them");
+ ck(coli_xdna_test_wrapper_releases() <= coli_xdna_test_userptr_wraps(),
+ "never more releases than wraps");
+ free(y); free(ref); coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+
+ /* ================================================================== */
+ printf("\ndefault behaviour\n");
+ {
+ use_helper(g_helper);
+ coli_xdna_test_set_artifact_root(g_root);
+ arm(F_NONE);
+ /* force deliberately NOT set: helper present, artifact valid, device fine */
+ QT T; qt_make(&T, TK, TN, 64, 4242u);
+ float *ref = (float*)malloc((size_t)TM*TN*4); matmul_qt(ref, x, &T, TM);
+ float *y = (float*)malloc((size_t)TM*TN*4); poison_buf(y,(size_t)TM*TN);
+ int d0 = coli_xdna_test_dispatches();
+ seam(y, x, &T, TM);
+ ck(coli_xdna_test_dispatches()==d0, "everything available, still zero dispatches");
+ ck(memcmp(y,ref,(size_t)TM*TN*4)==0, "and the current path produced the result");
+ ck(coli_xdna_test_output_valid()==0, "no XDNA output was claimed");
+ free(y); free(ref); coli_xdna_prepared_release(&T.xdna); free(T.q4); free(T.s);
+ }
+
+ coli_xdna_execution_shutdown();
+ coli_xdna_shutdown();
+ coli_xdna_test_set_registry(NULL,0);
+ coli_xdna_prepared_release(&W.xdna); coli_xdna_prepared_release(&W2.xdna);
+ free(W.q4); free(W.s); free(W2.q4); free(W2.s); free(x); free(REF);
+
+ ck(coli_xdna_prepared_live_objects()==0, "no prepared object leaked");
+ ck(coli_xdna_prepared_total_bytes()==0, "no prepared bytes leaked");
+
+ printf("\ntest_xdna_failure: %s (%d checks)\n", g_fail ? "FAIL" : "ok", g_checks);
+ return g_fail;
+}
diff --git a/c/tests/test_xdna_registry.c b/c/tests/test_xdna_registry.c
index 14ed7297..967dd12b 100644
--- a/c/tests/test_xdna_registry.c
+++ b/c/tests/test_xdna_registry.c
@@ -400,6 +400,83 @@ static void test_production_rows(void){
"unshipped production artifact -> ARTIFACT_UNAVAILABLE");
}
+/* ---- the compiled-in production registry -------------------------------
+ *
+ * REGRESSION for a defect W2-N7-I5 found in this file: g_nrows was initialised
+ * to 0 with no lazy initialisation, so the production table was installed but
+ * EMPTY and every production lookup answered UNKNOWN_FAMILY. It stayed
+ * invisible for two slices because every caller was a test that installed its
+ * own registry first, and it surfaced only when I5 became the first production
+ * consumer.
+ *
+ * So this runs FIRST, before anything calls coli_xdna_test_set_registry, and it
+ * asks the production table what it actually contains. */
+static void test_production_registry_default(void){
+ printf("compiled-in production registry, with no test rows installed\n");
+
+ ck(coli_xdna_registry_validate(), "the production registry validates as compiled");
+
+ struct { unsigned m, k, n; const char *label; } want[] = {
+ { 64, 6144, 2048, "F3 M64 K6144 N2048" },
+ { 256, 6144, 2048, "F3 M256 K6144 N2048" }
+ };
+ for(size_t i = 0; i < sizeof want/sizeof want[0]; i++){
+ ColiXdnaRequest q;
+ q.family = COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP;
+ q.m = want[i].m; q.k = want[i].k; q.n = want[i].n;
+ q.in_dtype = COLI_XDNA_DT_BF16;
+ q.weight_dtype = COLI_XDNA_DT_BF16;
+ q.out_dtype = COLI_XDNA_DT_F32;
+ const ColiXdnaArtifact *a = coli_xdna_registry_lookup(&q);
+ char m[96];
+ snprintf(m, sizeof m, "%s is visible without any test setup", want[i].label);
+ ck(a != NULL, m);
+ if(a){
+ snprintf(m, sizeof m, "%s carries its research family", want[i].label);
+ ck(a->research_family && !strcmp(a->research_family, "F3"), m);
+ snprintf(m, sizeof m, "%s carries a 64-hex xclbin hash", want[i].label);
+ ck(a->xclbin_sha256 && strlen(a->xclbin_sha256) == 64, m);
+ snprintf(m, sizeof m, "%s is fully research-qualified", want[i].label);
+ ck(a->runtime_weight_qualified && a->correctness_qualified
+ && a->userptr_qualified && a->structural_qualified, m);
+ }
+ }
+
+ /* Exactly those rows, and nothing that was never qualified. */
+ ColiXdnaRequest q;
+ q.family = COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP;
+ q.m = 128; q.k = 6144; q.n = 2048;
+ q.in_dtype = COLI_XDNA_DT_BF16; q.weight_dtype = COLI_XDNA_DT_BF16;
+ q.out_dtype = COLI_XDNA_DT_F32;
+ ck(coli_xdna_registry_lookup(&q) == NULL,
+ "an M bucket that was never qualified is absent");
+
+ /* And a test registry stays a TEST registry: installing one must not be how
+ * the production table becomes visible, and restoring must bring it back. */
+ ColiXdnaArtifact none[1];
+ memset(none, 0, sizeof none);
+ none[0].family = COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP;
+ none[0].research_family = "TEST";
+ none[0].artifact_m = 8; none[0].k = 8; none[0].n = 8;
+ none[0].in_dtype = COLI_XDNA_DT_BF16;
+ none[0].weight_dtype = COLI_XDNA_DT_BF16;
+ none[0].out_dtype = COLI_XDNA_DT_F32;
+ none[0].target = COLI_XDNA_TARGET_XDNA2;
+ none[0].xclbin_name = "t.xclbin";
+ none[0].xclbin_sha256 = "0000000000000000000000000000000000000000000000000000000000000000";
+ none[0].insts_name = "t.bin";
+ none[0].insts_sha256 = "0000000000000000000000000000000000000000000000000000000000000000";
+ none[0].runtime_weight_qualified = none[0].correctness_qualified = 1;
+ none[0].userptr_qualified = none[0].structural_qualified = 1;
+ coli_xdna_test_set_registry(none, 1);
+ q.m = 64;
+ ck(coli_xdna_registry_lookup(&q) == NULL,
+ "a test registry replaces the production one rather than adding to it");
+ coli_xdna_test_set_registry(NULL, 0);
+ ck(coli_xdna_registry_lookup(&q) != NULL,
+ "and restoring brings the production rows back");
+}
+
int main(void){
/* Every fixture lives in a private directory beside the test binary; the
* registry never reads the current directory of its own accord. */
@@ -410,6 +487,9 @@ int main(void){
mkdir(g_root, 0777);
#endif
+ /* FIRST, before any test registry is installed. */
+ test_production_registry_default();
+
build_registry();
test_sha256_vectors();
test_lookup();
diff --git a/c/tests/xdna_fake_helper.c b/c/tests/xdna_fake_helper.c
index a5def5c5..3b133a4b 100644
--- a/c/tests/xdna_fake_helper.c
+++ b/c/tests/xdna_fake_helper.c
@@ -45,7 +45,9 @@ enum { F_NONE = 0, F_DEVICE, F_ARTIFACT, F_WRAP, F_EXECUTE, F_COMPLETION };
static int g_fail;
static int g_open, g_wrap, g_exec, g_relw, g_shut;
static int g_is_open;
-static const unsigned short *g_w; /* borrowed, never owned */
+static unsigned short *g_w; /* SNAPSHOT of the caller image, see wrap */
+static const void *g_w_src; /* the caller pointer it was taken from */
+static size_t g_w_elems;
static size_t g_m, g_k, g_n;
static char g_err[256] = {0};
@@ -67,7 +69,7 @@ API int coli_xdna_helper_open(const char *xclbin, const char *insts,
if(!xclbin || !insts) return H_E_ARTIFACT;
{ FILE *f = fopen(xclbin, "rb"); if(!f) return H_E_ARTIFACT; fclose(f); }
{ FILE *f = fopen(insts, "rb"); if(!f) return H_E_ARTIFACT; fclose(f); }
- g_m = m; g_k = k; g_n = n; g_is_open = 1; g_w = NULL;
+ g_m = m; g_k = k; g_n = n; g_is_open = 1; free(g_w); g_w = NULL; g_w_src = NULL;
return H_OK;
}
@@ -79,7 +81,16 @@ API int coli_xdna_helper_wrap_weight(void *bf16, uint64_t bytes){
/* The alignment the qualified userptr path requires. The fake enforces it
* so a host regression that dropped the check would still be caught here. */
if(((uintptr_t)bf16 % 4096u) != 0){ snprintf(g_err,sizeof g_err,"unaligned userptr"); return H_E_WRAP; }
- g_w = (const unsigned short *)bf16; /* borrowed; no copy is made */
+ /* SNAPSHOT, deliberately. The real helper wraps this memory in an
+ * xrt::ext::bo and immediately sync()s it BO_TO_DEVICE, so the device sees
+ * the bytes as they were AT WRAP TIME. Reading the caller pointer live at
+ * execute time would model a shortcut the real path does not take, and
+ * would hide any host bug that fails to re-wrap after the image changes. */
+ { size_t elems = (size_t)(bytes / 2);
+ unsigned short *snap = (unsigned short *)malloc(bytes ? (size_t)bytes : 2);
+ if(!snap) return H_E_WRAP;
+ memcpy(snap, bf16, (size_t)bytes);
+ free(g_w); g_w = snap; g_w_src = bf16; g_w_elems = elems; }
return H_OK;
}
@@ -113,8 +124,8 @@ API int coli_xdna_helper_execute(const void *a_bf16, uint64_t a_bytes,
}
#endif
-API int coli_xdna_helper_release_weight(void){ g_relw++; g_w = NULL; return H_OK; }
-API void coli_xdna_helper_shutdown(void){ g_shut++; g_is_open = 0; g_w = NULL; }
+API int coli_xdna_helper_release_weight(void){ g_relw++; free(g_w); g_w = NULL; g_w_src = NULL; return H_OK; }
+API void coli_xdna_helper_shutdown(void){ g_shut++; g_is_open = 0; free(g_w); g_w = NULL; g_w_src = NULL; }
/* -- test controls, not part of the ABI ---------------------------------- */
API void fake_set_fail(int stage){ g_fail = stage; }
diff --git a/c/tests/xdna_physical_probe.c b/c/tests/xdna_physical_probe.c
index e3618d44..62a68318 100644
--- a/c/tests/xdna_physical_probe.c
+++ b/c/tests/xdna_physical_probe.c
@@ -307,6 +307,55 @@ int main(int argc, char **argv){
free(x); free(y);
}
+ /* ---- stale-wrapper control -------------------------------------------
+ * Re-prepare a DIFFERENT weight into the SAME prepared object. Capacity is
+ * retained, so the buffer pointer is unchanged. If the lane keys its
+ * userptr wrapper on the pointer alone it skips the re-wrap, and the device
+ * -- which snapshotted at wrap time via sync(BO_TO_DEVICE) -- computes with
+ * the previous weight. */
+ {
+ int S = 64;
+ float *x = (float*)malloc((size_t)S*PK*4);
+ unsigned s = 0xBEEF01u;
+ for(size_t i=0;i<(size_t)S*PK;i++){ s=s*1664525u+1013904223u;
+ x[i] = (float)((int)((s>>20)&255)-128)*0.0078125f; }
+ float *ya = (float*)malloc((size_t)S*PN*4);
+ float *yb = (float*)malloc((size_t)S*PN*4);
+ printf("\n-- stale-wrapper control: reprepare a different weight in place --\n");
+ int ha = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &G.xdna,
+ G.fmt, G.q4, G.s, G.I, G.O, G.gs, ya, x, S);
+ const void *p1 = coli_xdna_prepared_image(G.xdna);
+ int w0 = coli_xdna_test_userptr_wraps();
+ coli_xdna_prepared_invalidate(G.xdna);
+ ColiXdnaPrepResult pr = coli_xdna_prepare_from_fmt4(G.xdna, 4, U.q4, U.s, PK, PN, PGS);
+ const void *p2 = coli_xdna_prepared_image(G.xdna);
+ int hb = coli_xdna_try_matmul(COLI_XDNA_FAMILY_MOE_SHARED_GATE_UP, &G.xdna,
+ G.fmt, G.q4, G.s, G.I, G.O, G.gs, yb, x, S);
+ printf("PREPARE_RC %d ptr_same %s\n", (int)pr, p1==p2?"YES":"no");
+ printf("HANDLED %d %d\n", ha, hb);
+ printf("WRAPS before=%d after=%d rewrap=%s\n", w0,
+ coli_xdna_test_userptr_wraps(),
+ coli_xdna_test_userptr_wraps()>w0 ? "HAPPENED" : "SKIPPED");
+ if(ha && hb){
+ const unsigned short *W = (const unsigned short*)p2;
+ float *ref = (float*)malloc((size_t)S*PN*4);
+ oracle_double(ref, x, W, S, PK, PN);
+ size_t mism=0, sameA=0;
+ for(size_t i=0;i<(size_t)S*PN;i++){
+ if(!accept_a1c1(ref[i], yb[i])) mism++;
+ if(yb[i]==ya[i]) sameA++;
+ }
+ printf("SECOND_VS_ITS_OWN_WEIGHT mismatches %llu / %d\n",
+ (unsigned long long)mism, S*PN);
+ printf("SECOND_VS_FIRST_RESULT identical %llu / %d\n",
+ (unsigned long long)sameA, S*PN);
+ if(mism){ printf("WEIGHT_VIEW DEFECT -- device view not resynchronised\n"); g_bad = 1; }
+ else printf("WEIGHT_VIEW coherent -- the re-wrap happened\n");
+ free(ref);
+ } else { printf("WEIGHT_VIEW inconclusive (an operation declined)\n"); g_bad = 1; }
+ free(x); free(ya); free(yb);
+ }
+
/* Negative control on live hardware: same shape, wrong family. */
{
float *x = (float*)malloc((size_t)8*PK*4); memset(x,0,(size_t)8*PK*4);