Skip to content

A computer, compiled into the weights of a language model. An exact CPU inside Qwen3.5-4B, with Gated DeltaNet state as RAM.

3.0 KB
Raw Blame

Trace format

The transformer never sees "a machine state". It sees a token stream of write events, and its only job is to predict the next token. Source of truth: chipformer/spec.py, chipformer/ref/emu.py, chipformer/trace.py.

Cells

All state lives in one flat array of 4609 cells, each holding an integer 0..0xFFF.

Address Cell Range
0x0000–0x0FFF RAM byte
0x1000–0x100F V0–VF byte
0x1010 I 12-bit
0x1011 PC 12-bit
0x1012 SP 0–15 (wraps)
0x1013 / 0x1014 DT / ST byte
0x1015 RNG (12-bit Galois LFSR, mask 0x829) 1–4095
0x1016 CLS: writing it zeroes the display –
0x1017–0x101C quirk flags (see quirks.md) 0/1
0x1020–0x102F stack (return addresses) 12-bit
0x1030–0x103F keys 0–F 0/1
0x1100–0x11FF display, 32 rows × 8 bytes, MSB is the leftmost pixel byte
0x1200 NULL sink (writes with no effect) –

Tokens

Token Id
VAL v v (0–4095)
ADDR a 4096 + a
STEP 8705
TICK 8706
BOOT 8707

A write is the pair ADDR a, VAL v.

Records

BOOT <full state dump> STEP           prompt (also used for context rollover)
<instruction writes> STEP             generated by the model
TICK <DT write> <ST write> STEP       TICK injected by the host, rest generated
<KEY cell write> STEP                 injected whole by the host
  • Host injection only ever happens right after a STEP.
  • Every value in a record is computed from the state before the record. The PC write is always last. Writes inside a record never read cells written earlier in the same record. The model reads register operands once, at the record's first position, and carries them forward.
  • The state dump writes CLS first, then every other nonzero cell (except NULL) in address order. A read of a cell with no write in the context returns 0.

Instruction records

Each record is a list of writes, then the PC write, then STEP.

Opcode Writes before PC
00E0 CLS
00EE SP←SP−1, PC←stack[SP−1]
2NNN stack[SP]←PC+2, SP←SP+1
1NNN, BNNN, skips, EX9E/EXA1, 0NNN none
6XNN, 7XNN, 8XY0–3, FX07, FX0A VX (8XY1–3 add VF←0 with logic)
8XY4–7, 8XYE VX, VF (VF, VX with vforder)
ANNN, FX1E, FX29 I
CXNN VX←rnd & NN, RNG←lfsr(RNG) where rnd = low byte of the new state
DXYN 2N display writes (left byte, right byte per row; off-screen bytes wrap, or go to NULL with clip), then VF
FX15 / FX18 DT / ST
FX33 M[I], M[I+1], M[I+2]
FX55 M[I..I+X], then I←I+X+1 unless loadstore
FX65 V0..VX, then I←I+X+1 unless loadstore

Record lengths depend only on the opcode (and N/X) and the quirk flags, never on the data. The one exception is FX0A's PC value: if no key is held it rewrites VX unchanged and keeps PC.