A computer, compiled into the weights of a language model. An exact CPU inside Qwen3.5-4B, with Gated DeltaNet state as RAM.
README: objective framing for the Gated DeltaNet section
Divyansh Agrawal302cd8477b04
diff --git a/README.md b/README.mdindex bf76427..711b989 100644--- a/README.md+++ b/README.md@@ -71,11 +71,11 @@ signed and decimal numbers, mod, gcd, and whole expressions with brackets. Stockevery layer the same width, so the merged model is 14 GB vs 9.3 GB for base Qwen. The extra isalmost all zeros, so the download is about the same size.-### We use Gated DeltaNet as RAM+### Gated DeltaNet as RAM<p align="center"><img src="docs/gdn-ram.png" alt="Gated DeltaNet state used as RAM: a one-hot key overwrites one row, a one-hot query reads it back, and the CPU's registers, memory and program counter live in DeltaNet heads." width="100%"></p>-We use Qwen3.5's own Gated DeltaNet layers as the computer's RAM. Qwen3.5 mixes full attention with these layers, and each DeltaNet head keeps a state matrix that it updates at every token with the gated delta rule (panel **a**). We open the gate (α = 1, so nothing fades) and set the write strength to β = 1. Each key is a one-hot vector picking a single row, so every update replaces exactly that row with the value and leaves the rest untouched. That's a RAM write. A RAM read is the same head's query: a one-hot query returns exactly that row (panel **b**).+The computer's RAM is the state of Qwen3.5's own Gated DeltaNet layers. Qwen3.5 mixes full attention with these layers, and each DeltaNet head keeps a state matrix that it updates at every token with the gated delta rule (panel **a**). In the computer's heads the gate is held open (α = 1, so nothing fades) and the write strength is β = 1. Each key is a one-hot vector picking a single row, so every update replaces exactly that row with the value and leaves the rest untouched. That's a RAM write. A RAM read is the same head's query: a one-hot query returns exactly that row (panel **b**).The rest is layout (panel **c**). The low bits of an address choose the head and the remainingbits choose the row, and each row stores one byte as bits. The machine's memory is spread over a