A computer, compiled into the weights of a language model. An exact CPU inside Qwen3.5-4B, with Gated DeltaNet state as RAM.
RISC-V: calc v2 firmware (exact fractions throughout); README table adds mul_div_multiple_longer and the v2 div_big score
Divyansh Agrawal9b05e65087ef
diff --git a/README.md b/README.mdindex 577d0bd..7f1397f 100644--- a/README.md+++ b/README.md@@ -179,7 +179,8 @@ itself, digit by digit. You can run it with no runtime or tool executor at all.|---|---|---|---|| Synthetic integer arithmetic | 70.0 | **97.5** | 97.5 || DeepMind `mul_big` | 65.0 | **97.5** | 100 |-| DeepMind `div_big` | 80.0 | **95.0** | 97.5 |+| DeepMind `div_big` | 80.0 | **97.5** | 95.0 |+| DeepMind `mul_div_multiple_longer` | 77.5 | **100.0** | 97.5 || DeepMind `add_or_sub_big` | 95.0 | **97.5** | 97.5 |The computer has never printed a wrong result. We replayed about 230,000 of its output positionsdiff --git a/experimental/riscv/chipformer/rv/calc.py b/experimental/riscv/chipformer/rv/calc.pyindex 89b80e9..790b780 100644--- a/experimental/riscv/chipformer/rv/calc.py+++ b/experimental/riscv/chipformer/rv/calc.py@@ -71,9 +71,36 @@ class _Err(Exception):pass+def _nd(x: int) -> int:+ return len(str(abs(x)))+++def _frac(op, a, b):+ """The firmware's rational step (calc.c frac_op): its products must fit MAXD digits."""+ from fractions import Fraction++ an, ad, bn, bd = a.numerator, a.denominator, b.numerator, b.denominator+ fa, fb = ad != 1, bd != 1+ if op == "*":+ if _nd(an) + _nd(bn) > MAXD or (fa and fb and _nd(ad) + _nd(bd) > MAXD):+ raise _Err+ return a * b+ if op == "/":+ if bn == 0 or (fb and _nd(an) + _nd(bd) > MAXD) or (fa and _nd(ad) + _nd(bn) > MAXD):+ raise _Err+ return Fraction(a) / b+ if (fb and _nd(an) + _nd(bd) > MAXD) or (fa and _nd(bn) + _nd(ad) > MAXD) or \+ (fa and fb and _nd(ad) + _nd(bd) > MAXD):+ raise _Err+ t1, t2 = an * bd, (bn * ad if op == "+" else -bn * ad)+ if (t1 < 0) == (t2 < 0) and max(_nd(t1), _nd(t2)) + 1 > MAXD:+ raise _Err+ return a + b if op == "+" else a - b++def _value(node):- """An integer expression's value as a Fraction: / is exact, but a non-integer value may only- be the final result (or negated); // and % floor, ** needs an exponent 0..999."""+ """An integer expression's value as a Fraction: + - * / are exact on rationals; // and %+ floor, ** (an exponent 0..999) and gcd need integers."""import astfrom fractions import Fraction@@ -92,15 +119,18 @@ def _value(node):return Fraction(gcd(*ints(*map(_value, node.args))))if not isinstance(node, ast.BinOp):raise _Err- a, b = ints(_value(node.left), _value(node.right))+ fa, fb = _value(node.left), _value(node.right)op = type(node.op)- if op in (ast.Div, ast.FloorDiv, ast.Mod) and b == 0:+ sym = {ast.Add: "+", ast.Sub: "-", ast.Mult: "*", ast.Div: "/"}.get(op)+ if sym == "/" or (sym and (fa.denominator != 1 or fb.denominator != 1)):+ return _frac(sym, fa, fb)+ a, b = ints(fa, fb)+ if op in (ast.FloorDiv, ast.Mod) and b == 0:raise _Errif op is ast.Pow and not 0 <= b <= 999:raise _Errf = {ast.Add: lambda: a + b, ast.Sub: lambda: a - b, ast.Mult: lambda: a * b,- ast.Div: lambda: Fraction(a, b), ast.FloorDiv: lambda: a // b, ast.Mod: lambda: a % b,- ast.Pow: lambda: a**b}.get(op)+ ast.FloorDiv: lambda: a // b, ast.Mod: lambda: a % b, ast.Pow: lambda: a**b}.get(op)if f is None:raise _Errreturn Fraction(f())@@ -125,7 +155,7 @@ def _n(rng, d):return str(rng.randint(10 ** (d - 1) if d > 1 else 0, 10**d - 1))-OPS = ["add", "sub", "mul", "div", "mod", "gcd", "pow", "expr", "frac"]+OPS = ["add", "sub", "mul", "div", "mod", "gcd", "pow", "expr", "frac", "rat"]def case(rng: random.Random, op: str) -> str:@@ -157,6 +187,16 @@ def case(rng: random.Random, op: str) -> str:if rng.random() < 0.3:x = y * int(_n(rng, rng.randint(1, 8)))return rng.choice([f"{sa}{x}/{y}", f"({x}+{y})/{sb}{y}", f"{x}*{g}/({y}-{g})"])+ if op == "rat": # rational chains, like the DeepMind mul_div / mixed modules+ k = rng.randint(2, 7)+ s = _n(rng, rng.randint(1, 3))+ for _ in range(k):+ x = _n(rng, rng.randint(1, 3))+ x = f"(-{x})" if rng.random() < 0.3 else x+ if rng.random() < 0.2:+ x = f"({x}/{_n(rng, 1) or 1})"+ s += rng.choice(["*", "/", "+", "-", "*", "/"]) + x+ return s# a random expression of 2-4 operandsk = rng.randint(2, 4)nums = [_n(rng, rng.randint(1, 8)) for _ in range(k)]@@ -205,17 +245,20 @@ def main():returnt = time.time()per = {op: [0, 0, 0] for op in OPS} # cases, instructions, tokens- bad = 0+ bad = limited = 0for op, e in cases(args.cases, args.seed):out, toks = run(e)- if out != expected(e):+ if out.startswith("error (too complex"): # out of registers (7): a limit, not a wrong value+ limited += 1+ elif out != expected(e):bad += 1print("MISMATCH", e, repr(out), repr(expected(e)))st = ref.trace_stats(toks)per[op][0] += 1per[op][1] += st["instructions"]per[op][2] += st["tokens"]- print(f"{args.cases} cases, {bad} mismatches ({time.time() - t:.0f} s)")+ print(f"{args.cases} cases, {bad} mismatches, {limited} too complex for the registers "+ f"({time.time() - t:.0f} s)")for op, (n, i, tk) in per.items():if n:print(f" {op:5s} {n:5d} cases {i / n:8.0f} instructions {tk / n:8.0f} tokens")diff --git a/experimental/riscv/chipformer/rv/chat.py b/experimental/riscv/chipformer/rv/chat.pyindex 05b6717..8913ad7 100644--- a/experimental/riscv/chipformer/rv/chat.py+++ b/experimental/riscv/chipformer/rv/chat.py@@ -29,9 +29,9 @@ from chipformer.rv import spec as SCALC_TOOL = {"type": "function", "function": {"name": "calc","description": "Evaluate an arithmetic expression exactly, with numbers of up to 62 digits: "- "+ - * and parentheses on integers or decimals; on integers also / (exact: "- "an integer or a reduced fraction p/q), // (floor division), % , ** and "- "gcd(a,b). Returns the exact value.",+ "+ - * / and parentheses on integers and fractions (exact: the result is an "+ "integer or a reduced fraction p/q), + - * on decimals, and on integers also "+ "// (floor division), %, ** and gcd(a,b). Returns the exact value.","parameters": {"type": "object", "properties": {"expression": {"type": "string","description": "e.g. 123456789*987654321-42 or 1.25*-3.5 or 84/18"}},@@ -374,7 +374,11 @@ class RvChat:breakelse:k = len(lock.toks)- lock.qwen([nxt])+ try:+ lock.qwen([nxt])+ except RuntimeError as e: # the reference run did not halt+ lock_error = {"at": len(gen) - 1, "got": str(e), "want": "a halt"}+ breakexpect = list(lock.toks[k + 1:])if nxt in self.eos:breakdiff --git a/experimental/riscv/chipformer/rv/env/calc/calc.c b/experimental/riscv/chipformer/rv/env/calc/calc.cindex bdf2d61..79a1a33 100644--- a/experimental/riscv/chipformer/rv/env/calc/calc.c+++ b/experimental/riscv/chipformer/rv/env/calc/calc.c@@ -5,9 +5,11 @@// expr = term {(+|-) term} term = factor {(*|/|//|%) factor}// factor = (-|+) factor | power power = atom [(**|^) factor]// atom = digits | ( expr ) | gcd ( expr , expr )-// / and // are floor division, % the matching modulo (the divisor's sign), ** / ^ a power with-// a small non-negative exponent. Output: the value in decimal and "\n", or "error\n" (division by-// zero, a value over MAXD digits, a syntax error).+// / is exact (rationals: + - * / on reduced fractions p/q), // floor division, % the matching+// modulo (the divisor's sign), ** / ^ a power with a small non-negative exponent; // % ** gcd+// need integers. Decimal operands (+ - * only) print with a point. Output: the value ("p/q" for+// a fraction) and "\n", or "error\n" (division by zero, a value or product over MAXD digits, a+// syntax error).//// A value is one word: register | negative << 8 | length << 16; a register's digits at and above// the value's length are 0. Register 7 holds the constant 1 (quotient digits, +1).@@ -46,7 +48,7 @@ static num alloc(int len) {dirty[r] = 0;return NUM(r, 0, len);}- err = 1;+ err = 2; // out of registers: "too complex"return NUM(0, 0, len);}@@ -140,9 +142,9 @@ __attribute__((noinline)) static qr divmod(num a, num b) { // |a| = q |b| + r:if (qd > 9) qd = 9;if (qd) {uint32_t o = BN_OFF(t);- if (bn_msb(msb | o | BN_M(qd))) // negative: add b back- do qd--;- while (!bn_mac(mac | o));+ if (bn_msb(msb | o | BN_M(qd))) // negative: add b back (at most twice; bounded+ do qd--; // anyway, should a register be corrupt)+ while (!bn_mac(mac | o) && qd > 0);if (qd) bn_mac(setq | o | BN_M(qd));}}@@ -218,8 +220,9 @@ static int prec(int op) { // 'f': // (floor), 'd': / (exact)return op == '^' ? 4 : op == 'n' ? 3 : op == '*' || op == 'f' || op == 'd' || op == '%' ? 2 : 1;}-// a / b exact: an integer, or (only as the expression's final value) a reduced fraction whose-// denominator is vden[i]+// Rationals: / is exact, so a value may be a fraction vals[i] / vden[i] (hasden[i]; the+// denominator positive and > 1, the fraction reduced). + - * / take fractions; // % ** gcd and+// decimal operands need integers.static num vden[16];static int hasden[16];@@ -230,25 +233,89 @@ static num copy_of(num a) {return NUM(REG(c), NEG(a), LEN(a));}-__attribute__((noinline)) static num exact_div(num a, num b, int i) { // consumes both- int neg = NEG(a) ^ NEG(b);- if (is_zero(b)) {- err = 1;- return a;+__attribute__((noinline)) static void reduce(int i) { // vals[i] / vden[i] in lowest terms+ num n = vals[i], d = vden[i];+ hasden[i] = 0;+ if (err) return;+ if (is_zero(n)) {+ release(d);+ return;}- num g = gcd(copy_of(a), copy_of(b));- qr x = divmod(a & ~0x100u, g);+ int neg = NEG(n);+ num g = gcd(copy_of(n), copy_of(d));+ qr x = divmod(n & ~0x100u, g);release(x.r);- qr y = divmod(b & ~0x100u, g);+ qr y = divmod(d, g);release(y.r);release(g);+ vals[i] = x.q | (uint32_t)neg << 8;if (LEN(y.q) == 1 && dig(y.q, 0) == 1) {release(y.q);} else {vden[i] = y.q;hasden[i] = 1;}- return is_zero(x.q) ? x.q : x.q | (uint32_t)neg << 8;+}++// (a / da) op (b / db) for op in + - * / (da, db absent where fa, fb are 0): the result's+// numerator and denominator into slot i, then reduced. Each operand register is released as+// soon as its last product is made (7 registers: a/b + c/d peaks at 5).+__attribute__((noinline)) static void frac_op(int op, int i, num a, num da, int fa, num b, num db,+ int fb) {+ num n, d;+ if (op == '*') {+ n = mul(a, b);+ release(a);+ release(b);+ if (fa && fb) {+ d = mul(da, db);+ release(da);+ release(db);+ } else {+ d = fa ? da : db;+ }+ } else if (op == 'd') { // (a db) / (da b)+ if (is_zero(b)) err = 1;+ if (fb) {+ n = mul(a, db);+ release(a);+ release(db);+ } else {+ n = a;+ }+ if (fa) {+ d = mul(da, b);+ release(da);+ release(b);+ } else {+ d = b;+ }+ if (NEG(d)) {+ d &= ~0x100u;+ if (!is_zero(n)) n ^= 0x100u;+ }+ } else { // + -: (a db +- b da) / (da db)+ num t1 = a, t2 = b;+ if (fb) {+ t1 = mul(a, db);+ release(a);+ }+ if (fa) {+ t2 = mul(b, da);+ release(b);+ }+ if (fa && fb) {+ d = mul(da, db);+ release(da);+ release(db);+ } else {+ d = fa ? da : db;+ }+ n = add(t1, op == '-' && !is_zero(t2) ? t2 ^ 0x100u : t2);+ }+ vals[i] = n;+ vden[i] = d;+ reduce(i);}// decimals: a value is the integer vals[i] scaled by 10^-vsc[i] (+ - * keep it exact; the other@@ -279,10 +346,17 @@ static int apply(int op, int nv) { // returns the new value countreturn nv;}int sb = vsc[nv - 1], sa = vsc[nv - 2];- if (hasden[nv - 1] || hasden[nv - 2]) err = 1; // a fraction is only ever the final value+ int fb = hasden[nv - 1], fa = hasden[nv - 2];num b = vals[--nv], a = vals[nv - 1], r;+ int rational = op == 'd' || ((fa || fb) && (op == '+' || op == '-' || op == '*'));+ if ((fa || fb) && !rational) err = 1; // // % ** gcd need integers+ if (rational && (sa || sb)) err = 1; // fractions and decimals do not mixif (err) {r = a;+ } else if (rational) {+ frac_op(op, nv - 1, a, vden[nv - 1], fa, b, vden[nv], fb);+ vsc[nv - 1] = 0;+ return nv;} else if (op == '+' || op == '-') {if (sa < sb) a = shift(a, sb - sa), sa = sb;if (sb < sa) b = shift(b, sa - sb);@@ -299,13 +373,10 @@ static int apply(int op, int nv) { // returns the new value countr = power(a, b);} else if (op == 'g') {r = gcd(a, b);- } else if (op == 'd') {- hasden[nv - 1] = 0;- r = exact_div(a, b, nv - 1);} else {r = divide(a, b, op == '%');}- if (op != 'd') hasden[nv - 1] = 0;+ hasden[nv - 1] = 0;vals[nv - 1] = r;vsc[nv - 1] = sa;return nv;@@ -433,7 +504,8 @@ static int calc_eval(void) { // prints the value (or "error") and a newline; 0}if (err || nv != 1 || operand) {fail:- for (const char *m = "error\n"; *m; m++) rv_putchar(*m);+ for (const char *m = err == 2 ? "error (too complex: split it)\n" : "error\n"; *m; m++)+ rv_putchar(*m);return 1;}print:;@@ -472,8 +544,8 @@ enum { TOOL_RESPONSE = 256, TOOL_RESPONSE_END, THINK, THINK_END };static void put(int c) { rv_ecall2(1, c, 0, 0); }static int gather(void) {- int n = 0, gt = 0;- for (int i = 1; i <= ROWS; i++) {+ int n = 0, gt = 0, i;+ for (i = 1; i <= ROWS; i++) {const volatile unsigned char *r = TEXT + 16 * i;int c = r[0];if (!c) break;@@ -489,7 +561,8 @@ static int gather(void) {}}calc_in[n] = 0;- return gt < 2 ? -1 : n;+ if (gt < 2) return -1;+ return i > ROWS ? -2 : n; // -2: no closing '<' within the capture window (truncated)}int main(void) {@@ -500,7 +573,8 @@ int main(void) {put(TOOL_RESPONSE);put('\n');if (n < 0) {- for (const char *m = "error\n"; *m; m++) put(*m);+ for (const char *m = n == -2 ? "error (too long: split it)\n" : "error\n"; *m; m++)+ put(*m);} else {calc_eval();}diff --git a/experimental/riscv/chipformer/rv/evalarith.py b/experimental/riscv/chipformer/rv/evalarith.pyindex ca8c417..f6c1caf 100644--- a/experimental/riscv/chipformer/rv/evalarith.py+++ b/experimental/riscv/chipformer/rv/evalarith.py@@ -45,13 +45,41 @@ def calc_response(expr: str) -> str:return f"\n<tool_response>\n{v}</tool_response>\n<think>\n\n</think>\n\n"+class BridgeHost:+ """pytool's host as the machine itself: a reference bridge.Session on the calc firmware,+ fed base Qwen's own tokens (the prompt as history). At </tool_call> it runs the call exactly+ as rv1's machine would (the same capture window, firmware and echo tokens) and returns the+ response's Qwen tokens to force."""++ def __init__(self, T, firmware, prompt: list[int]):+ m = ref.Machine.from_program(elf.program(firmware))+ BR.park(m)+ self.ses = BR.Session(T, m)+ self.ses.history(prompt)+ self.tcc = T.TCC++ def feed(self, t: int) -> list[int]:+ """Qwen's token t; the response tokens to force (empty unless t ran a call)."""+ from chipformer.rv import spec as S++ k = len(self.ses.toks)+ try:+ self.ses.qwen([t])+ except RuntimeError: # the run did not halt: an error response (rv1 would loop on)+ p = self.ses.T.printable+ body = [10, 256, 10, *b"error (did not halt)\n", 257, 10, 258, 10, 10, 259, 10, 10]+ return [p[b] for b in body]+ return [x - S.TEXT_TOKEN for x in self.ses.toks[k + 1:] if S.is_text(x)]++def problems(data: str, n: int, seed: int) -> list[dict]:return E.load_data("arith", n, seed) if data == "arith" else AB.load(data, n, seed)@torch.no_grad()-def run_base(model, tok, ids, tool: bool, budget: int):- """Greedy decoding of base Qwen; with ``tool`` the host answers calc calls."""+def run_base(model, tok, ids, tool: bool, budget: int, host: BridgeHost | None = None):+ """Greedy decoding of base Qwen; with ``tool`` the host answers calc calls (``host``: the+ machine's own responses, else calc.expected's)."""dev = next(model.parameters()).devicetcc = tok.convert_tokens_to_ids("</tool_call>")eos = {tok.convert_tokens_to_ids("<|im_end|>"), tok.convert_tokens_to_ids("<|endoftext|>")}@@ -63,11 +91,17 @@ def run_base(model, tok, ids, tool: bool, budget: int):gen += newseq += newown += len(new)+ resp = []+ if host is not None:+ for t in new:+ resp = host.feed(t)if not new or new[-1] in eos or not tool or new[-1] != tcc:break- txt = tok.decode(gen, skip_special_tokens=False)- m = re.findall(r"<parameter=[^>]*>(.*?)</parameter>", txt.split("<tool_call>")[-1], re.S)- resp = tok(calc_response(m[0] if m else ""), add_special_tokens=False).input_ids+ if host is None:+ txt = tok.decode(gen, skip_special_tokens=False)+ m = re.findall(r"<parameter=[^>]*>(.*?)</parameter>", txt.split("<tool_call>")[-1],+ re.S)+ resp = tok(calc_response(m[0] if m else ""), add_special_tokens=False).input_idsgen += respseq += respcalls += 1@@ -91,6 +125,9 @@ def main() -> None:ap.add_argument("--max-len", type=int, default=6144, help="rv1 --batch: Qwen KV slots")ap.add_argument("--shard", default=None, help="i/k: every k-th problem from the i-th")ap.add_argument("--ids", default=None, help="comma-separated problem ids (a subset)")+ ap.add_argument("--host", default="bridge", choices=("bridge", "expected"),+ help="pytool's responses: the calc firmware on a reference bridge Session "+ "(rv1's machine exactly), or calc.expected (Python, BPE-tokenized)")args = ap.parse_args()n = args.n or (40 if args.data == "arith" else None)probs = problems(args.data, n, args.seed)@@ -107,6 +144,11 @@ def main() -> None:tok = AutoTokenizer.from_pretrained(args.model)model = AutoModelForImageTextToText.from_pretrained(args.model, dtype=torch.bfloat16)model = model.to("cuda").eval()+ if tool and args.host == "bridge":+ T = BR.Text(Path(args.model) / "tokenizer.json")++ def make_host(prompt):+ return BridgeHost(T, args.firmware, prompt)if args.shard:i, k = map(int, args.shard.split("/"))probs = probs[i::k]@@ -179,7 +221,9 @@ def main() -> None:tokenize=True)prompts.append(ids if isinstance(ids, list) else ids["input_ids"])tcc = tok.convert_tokens_to_ids("</tool_call>")- state = [{"forced": [], "own": 0, "gen": [], "calls": 0} for _ in todo]+ state = [{"forced": [], "own": 0, "gen": [], "calls": 0,+ "host": make_host(pr) if tool and args.host == "bridge" else None}+ for pr in prompts]def hook(i, t):"""pytool: after </tool_call> the next tokens are the host's response (forced);@@ -191,7 +235,10 @@ def main() -> None:return t, Falses["own"] += 1s["gen"].append(t)- if tool and t == tcc:+ if s["host"] is not None:+ s["forced"] = s["host"].feed(t)+ s["calls"] += bool(s["forced"])+ elif tool and t == tcc:txt = tok.decode(s["gen"], skip_special_tokens=False)mm = re.findall(r"<parameter=[^>]*>(.*?)</parameter>",txt.split("<tool_call>")[-1], re.S)@@ -208,7 +255,8 @@ def main() -> None:s = state[i]text = tok.decode(toks, skip_special_tokens=False)emit(todo[i], text, dict(calls=s["calls"], n_qwen=s["own"], n_chip=0,- batch=args.batch), time.time() - t0)+ batch=args.batch, host=args.host if tool else None),+ time.time() - t0)if prompts:bd.run(prompts, 2 * args.budget, on_done=fin, hook=hook)@@ -224,9 +272,10 @@ def main() -> None:add_generation_prompt=True, enable_thinking=False,tokenize=True)ids = ids if isinstance(ids, list) else ids["input_ids"]- gen, calls, own = run_base(model, tok, ids, tool, args.budget)+ host = make_host(ids) if tool and args.host == "bridge" else None+ gen, calls, own = run_base(model, tok, ids, tool, args.budget, host)text = tok.decode(gen, skip_special_tokens=False)- fields = dict(calls=calls, n_qwen=own, n_chip=0)+ fields = dict(calls=calls, n_qwen=own, n_chip=0, host=args.host if tool else None)emit(p, text, fields, time.time() - t0)wall = time.time() - t_allif todo:diff --git a/experimental/riscv/chipformer/rv/rvbatch.py b/experimental/riscv/chipformer/rv/rvbatch.pyindex eccb3fb..b523fb6 100644--- a/experimental/riscv/chipformer/rv/rvbatch.py+++ b/experimental/riscv/chipformer/rv/rvbatch.py@@ -113,7 +113,11 @@ class RvBatch:breakelse:k = len(lock.toks)- lock.qwen([nxt])+ try:+ lock.qwen([nxt])+ except RuntimeError as e: # the reference run did not halt+ lock_error = {"at": len(gen) - 1, "got": str(e), "want": "a halt"}+ breakexpect = list(lock.toks[k + 1:])if nxt in chat.eos:breakdiff --git a/experimental/riscv/docs/rv32-p3-status.md b/experimental/riscv/docs/rv32-p3-status.mdindex 97fd546..c2e57c3 100644--- a/experimental/riscv/docs/rv32-p3-status.md+++ b/experimental/riscv/docs/rv32-p3-status.md@@ -1,51 +1,33 @@-# RV32 P3 status (deferred until P2 lands)+# RV32 P3 status-Branch `rv-p3`, from rv-p1 `de634e7`. Code: `chipformer/rv/{bridge,merge,chat,evalarith,replay,textcheck}.py`,-the bridge in `core.build(text=...)`, `env/progs/calc.c`, `tests/test_rv_bridge.py`. Design notes are in-`docs/rv32.md` (P3 section).+Branch `rv-p3b` (on rv-p2). The full write-up is the P3 section of `docs/rv32.md`.-## Built and verified-- **The bridge in the circuit:** tool-call text capture into a text buffer read as memory at 0x8000; GO at- `</tool_call>`; putchar echoed as Qwen tokens (bytes plus `<tool_response>`, `</tool_response>`,- `<think>`, `</think>`); owner, tau and mu; the HALT echo. Still 11 chip layers, 73 heads.-- **IR tests (`tests/test_rv_bridge.py`, 6 tests):** pass. The circuit equals the reference on owner bits and- every owned token, including bignum calls, prefilled history and two calls in one stream. The rv P1 suites- (`test_rv_ref`, `test_rv_core`) still pass.-- **Merged Qwen3.5-4B (layout rv1):** a text-only `Qwen3_5ForCausalLM`, hidden 3328 (no overlay needed), 15 GB.-- **End to end on GPU:** Qwen calls `calc`, the machine inside the weights computes- 123456789 × 987654321 = 121932631112635269, and Qwen answers it. The lockstep check against the reference- machine was clean, and the stock head's argmax agreed with the circuit at every owned position.-- **Arith n=40** (evals' seeded set, exact scoring), on an L4 box AWS reclaimed afterwards, so its result- files are gone:+## Where it is+- rv-q4p2 (Qwen3.5-4B + the P2 core + the bridge, 43 layers, 18.7 GB) works end to end. Qwen+ calls `calc`, the machine inside the weights computes exactly, and Qwen answers.+- Lockstep, the stock head and the IR replay are clean on every record checked.+- The chip layers run on the Triton executor in generation: bit-identical to the HF modules,+ 13x faster (3,200 chip tokens/s on an L4).+- Batched decoding (`evalarith --batch`): results don't depend on the batch's other rows, but+ aren't bit-identical to batch-1 decoding.+- calc v2: exact rationals. v1's "a fraction only as the final value" made 610 of the 1,040+ mul_div calls fail.- | Run | Correct | Notes |- |---|---|---|- | rv1 | **38/40** | 54 calc calls, all exact; 0 lockstep errors; 0 head mismatches; median about 3.5k chip tokens per call, about 70 s per problem on an L4 |- | base Qwen + host-executed calc tool (same response format) | 39/40 | |+## Code+- `chipformer/rv/`:+ - the bridge: `core.py`, `spec.py`, `ref.py`, `bridge.py`;+ - `merge.py`, `chat.py` (RvChat; ExecRow);+ - `rvbatch.py`, `evalarith.py`;+ - the checks: `replay.py`, `textcheck.py`, `execcheck.py`, `batchcheck.py`;+ - `calc.py`, `env/calc/calc.c`.+- Tests: `tests/test_rv_bridge.py` (9 tests, the circuit vs the reference in the IR).- Both rv1 misses are Qwen's own: one re-calls in a loop, the other re-subtracts a result the computer- already returned. The base-without-tool run was not finished.-- **Bug found and fixed:** the select dims outlived the exit and derailed Qwen's norms after a run. The- merged circuit now keeps only code, tau and mu after the exit.--## Not done / next-- **0.8B does not fit this core:** it has 1 value head per key head and 3584 MLP units, while the core- needs 2 value heads per key head and up to 6198 units in one sublayer. It would need a core variant;- I'd defer it to after P2.-- **Not run yet:**- - the text-fidelity check (`rv/textcheck.py`, KL vs base against the sdpa/eager noise floor)- - the offline IR replay of eval records (`rv/replay.py`)- - the base-without-tool control-- Rebase onto rv-p2 (DIV, Zba, 256 KB ROM, custom bignum uops); then run DeepMind sets via the- `evalarith`-style harness.-- **Speed:** about 50-60 chip tokens/s on the eager HF chip-only path. The compact executor inside- generation isn't wired in.-- **Capture window:** CAP_MAX = 80 tokens. The thermometer weights are 8·2^m, which overflows bf16 at- 96 after the emitter's 2^36 fold; a wider window needs a two-level counter.--## Box state-- **q35-batch:** nothing of mine is running; GPU idle. My files:- - `~/wrv3` (code)- - `~/weights/dist/rv-q4` (the merged checkpoint, 15 GB; the disk is at 97%, delete it if space is needed)- - `~/rvres` (`chat.log`, plus a partial `arith.rv1.jsonl`)-- **q35-s1 / q35-s2:** terminated by AWS (spot).+## Boxes+- **q35-eval** (L4, us-east-1): the gate runs (L4 batch-1 records in `~/rvres`).+- **q35-inline** (A10G, us-east-1): needed room for the checkpoint, so `~/weights/dist/q4-v15`+ was deleted (allowed); `q4-v18` is kept.+- **q35-eval2** (A10G, us-west-2).+- Each holds `~/weights/dist/rv-q4p2`, the three bit-identical (per-tensor sha1). The A10G+ builds were rebuilt there, with a temporary swap file; their 8 anti-sink `k_proj` tensors+ were copied from the L4 build.+- Code in `~/wrv3`.diff --git a/experimental/riscv/docs/rv32.md b/experimental/riscv/docs/rv32.mdindex 8be24cb..43d546b 100644--- a/experimental/riscv/docs/rv32.md+++ b/experimental/riscv/docs/rv32.md@@ -2,8 +2,146 @@`chipformer/rv/` compiles a RISC-V core into stock Qwen3.5 Gated DeltaNet + SwiGLU layers withthe same q35 compiler as the CHIP-8 machine (`chipformer/q35/ir.py`, `prims.py`, `emit.py`).-P2 (branch `rv-p2`) is first, then P1 (branch `rv-p1`), then the P0 feasibility study-(branch `rv-p0`).+P3 (branch `rv-p3b`) is first, then P2 (branch `rv-p2`), P1 (branch `rv-p1`) and the P0+feasibility study (branch `rv-p0`).++## P3: the RV32 computer inside Qwen3.5-4B (a `calc` tool)++### What it is+- **rv-q4p2:** a stock text-only `Qwen3_5ForCausalLM`.+ - Layers: the 11 chip layers of the P2 core plus the bridge first (`core.build(text=...)`),+ then Qwen3.5-4B's 32 layers.+ - Hidden 3968; 478,656 tokens (Qwen's plus 230,324 chip tokens appended as special tokens);+ 18.7 GB in bf16.+ - Built by `python -m chipformer.rv.merge BASE --out DIR` (needs about 31 GB of RAM).+- **The bridge** (in the circuit):+ - Qwen's tool-call tokens after `<tool_call>` are captured into a text buffer that the core+ reads as memory at 0x8000: 16 bytes per token, up to 80 tokens per call.+ - `</tool_call>` starts a run (GO). Each byte the program prints is echoed as the matching Qwen+ token, so Qwen reads the response as ordinary text; HALT is echoed as a newline.+ - The circuit owns exactly those positions (owner, tau and mu). Everywhere else Qwen predicts.+- **The firmware:** `env/calc/calc.c`, built with `-DBRIDGE` as `tests/data/rv/calc-bridge.elf`.+ - It parks between calls. It takes the call's first parameter, evaluates it on P2's digit file+ and prints `\n<tool_response>\nVALUE\n</tool_response>\n<think>\n\n</think>\n`.+ - Values have up to 62 digits.+ - Integers: `+ - * // % ** gcd()` and parentheses.+ - Decimals: `+ - *`, printed with a point and no trailing zeros.+ - calc v2 (`2db7827`): `/` is exact rational arithmetic throughout (reduced fractions `p/q`).+ - Error messages:+ - `error (too complex: split it)` when the 7 bignum registers run out;+ - `error (too long: split it)` for a call longer than the 80-token capture window.+- **Decoding** (`rv/chat.py` `RvChat`):+ - Chip tokens run only the chip layers and are invisible to Qwen's layers.+ - The snapshot (firmware, RAM, registers) goes into the chip layers' states after the prompt.+ - Every circuit token is checked in lockstep against the reference machine (`bridge.Session`).++### The chip layers on the Triton executor (`908fdd4`)+- `RvChat(executor=True)` runs the chip layers as the circuit itself (`tritonexec.Executor2`: the+ IR's arithmetic in fp32, one CUDA graph per token) instead of their HF modules.+- After the chip layers, a text token's residual is Qwen's embedding row plus the 23 kept dims+ (output code, tau, mu); the exit clears every other chip dim and the ballast. So `hc` is+ rebuilt from those and handed to Qwen's layers unchanged.+- **Checked** (`rv/execcheck.py`, 6 records including two mul_div ones):+ - "verify" mode runs both paths. The HF chip view's residual equals the executor's bit for+ bit at every position.+ - Executor-only decoding gives the records' tokens exactly.+- **Speed (L4):** about 250 chip tokens/s on the HF path, 3,200 on the executor (13x).+ - A calc call (1.3k to 3k chip tokens) now takes about 0.5 to 1 s.+ - Text tokens (about 68 ms each at batch 1) are most of the time.+ - mul_div problems: median 585 s per problem on the HF path, 242 s on the executor.++### Batching (`rvbatch.py`, `evalarith --batch N`)+- Each conversation keeps its own executor (batch 1, its own CUDA graph) for its chip bursts.+- The text tokens of all live rows go through Qwen's layers in one batched step, on q35's+ per-row-position cache (`batch._Rows`). Finished rows are refilled from the queue.+- **Invariance** (`rv/batchcheck.py --alone`, A10G, B = 8):+ - Each of 6 records (two of them mul_div) decoded alone in the 8-row decoder gives the same+ tokens as all 6 decoded together, so the results do not depend on the batch's other rows.+ - Not bit-identical to batch-1 `generate`: Qwen's GEMMs round differently at M = 1 and+ M = B, and the static KV cache with a mask differs from the dynamic one. Five of 6+ records still matched the L4 batch-1 records exactly; the sixth differs from token 8,390.+ - The circuit is exact either way: 0 lock errors, 0 head mismatches.+- **Throughput:**+ - On the same 6 mixed problems: 128 problems/h at batch 1 vs 199/h at batch 8. Six+ problems are limited by the longest.+ - Per text step: 62 ms for 1 row, about 90 ms for 8.+ - Full sets at batch 8 are in the calc v2 section below.+ - The rows' chip bursts run one after another, about a third of a batched run's wall time; a+ batched executor (Executor2 takes B rows) is the next speedup.+- **Base Qwen** (pytool, base_nudge): `--batch` uses q35's `BatchDecoder` (compiled steps). It+ is not token-identical to the batch-1 HF `generate` path: 1 of 6 pytool texts matched, with+ the same accuracy, 4/6 both ways.++### Checks+- **Text fidelity** (`rv/textcheck.py`, 16 prompts):+ - Mean KL of the merge vs base: 6.1e-4. The noise floor between base's own sdpa and eager+ kernels: 5.1e-4.+ - Greedy text identical on 10/16 prompts; the tokenizer is identical (4/4).+- **IR replay** (`rv/replay.py`): every circuit-owned position of an eval record re-run in the+ IR evaluator. All clean (0 wrong predictions, 0 owner mismatches):+ - arith: 40/40 records (89,453 owned positions);+ - DM: 6 records from each of 4 sets;+ - mul_div: 5 records, 4 of them from the executor path (up to 24 calls, 78k owned positions);+ - calc v2: 6 batched records.+- **Lockstep:** 0 lock errors and 0 head mismatches in every rv1 run (head: the stock lm_head's+ argmax equals the circuit's token at every owned text position).++### Gate results (n = 40, Qwen budget 4096 tokens; exact rational match)+calc v1 (fractions only as the final value), L4, batch 1. rv1 = the merge with the machine+inside; pytool = base Qwen3.5-4B with the same tool, answered by the host (`calc.expected`);+base_nudge = base Qwen with no tool.++| Set | rv1 | pytool | base_nudge |+|---|---|---|---|+| arith (evals' seeded set) | 39/40 | 39/40 | 30/40 |+| dmx_mul_big | 39/40 | 40/40 | 22/40 |+| dmx_add_or_sub_big | 39/40 | 39/40 | 35/40 |+| dmx_div_big | 38/40 | 39/40 | 29/40 |+| dmx_add_sub_multiple_longer | 40/40 | 40/40 | 39/40 |+| dmx_mul_div_multiple_longer | 20/40 | 19/40 | 36/40 |++- The DM base_nudge runs used q35's `BatchDecoder` (A10G, batch 16). The arith base_nudge run+ used batch-1 HF `generate`.+- **rv1's misses are Qwen's, not the machine's.**+ - Every result the machine returned was exact (lockstep and replay clean).+ - Misses: one re-subtraction after a correct result (arith); a dropped sign (add_or_sub,+ which pytool misses too); one 44-call loop (mul_big); two in div_big. In both div_big misses+ the machine had returned the right fraction; Qwen then answered a different number once and+ left no final answer the other time.+- **mul_div: the tool was the problem.**+ - 610 of the 1,040 calc calls returned `error`, because v1 allowed a fraction only as the final+ value (`1/2*1/2`, `(-27/4)/(-27)`). Qwen then retried in loops: 15 of the 20 misses made+ 20+ calls.+ - pytool used the same semantics and loops the same way.+ - calc v2 fixes this. Re-run on v2, Qwen's same 1,083 calls (mul_div and div_big) give 1,020+ values, 0 of them wrong; 41 errors (decimals mixed with `/`); 22 too complex.++### calc v2 (rationals) re-run (A10G, rv1 at batch 8, pytool at batch 16)+- pytool's host here is the firmware itself on a reference bridge Session (`--host bridge`).+ Its responses are byte for byte what rv1's machine answers: the same capture window, error+ messages and echo tokens.++| Set | rv1 | pytool |+|---|---|---|+| dmx_mul_div_multiple_longer | **40/40** (204 calls) | 39/40 (209 calls) |+| dmx_div_big | 39/40 | 38/40 |++- rv1's one div_big miss is a 44-call loop on a result the machine returned correctly.+- IR replay of 6 v2 batched records: clean.+- Throughput (A10G, batch 8): 109 problems/h on mul_div and 270/h on div_big. The L4 at batch+ 1 managed 14.5/h on mul_div (v1, executor) and 80/h on div_big (v1, HF chip path).+- Base Qwen's batched runs (`BatchDecoder`, fixed batch 16) don't depend on the batch's other+ rows either: 12/12 prompts decoded alone matched the full batch.++### Not done in P3+- The 80-token capture window was not raised. Its thermometer weights 8·2^m overflow bf16+ past 80 after the emitter's 2^36 fold, so a wider window needs a two-level counter.+- Text decoding is still the HF modules at batch 1 or B, about 68 ms per token on an L4. Two+ fallbacks are installed instead of the optimized kernels: torch's linear attention, as+ `flash-linear-attention` is missing, and the `causal_conv1d` reference.+- 0.8B was skipped (user decision). It has 1 value head per key head, and the core needs 2.+- MicroPython has run on the standalone core and executor (P2), not yet inside the merged+ model; it needs a bridge build (P4).## P2: RV32IM + Zba, a software stack (MicroPython, calc)diff --git a/experimental/riscv/tests/data/rv/calc-bridge.elf b/experimental/riscv/tests/data/rv/calc-bridge.elfindex 3c1dce8..32a4c30 100644Binary files a/experimental/riscv/tests/data/rv/calc-bridge.elf and b/experimental/riscv/tests/data/rv/calc-bridge.elf differdiff --git a/experimental/riscv/tests/data/rv/calc.elf b/experimental/riscv/tests/data/rv/calc.elfindex 7f80bdc..d9dea98 100644Binary files a/experimental/riscv/tests/data/rv/calc.elf and b/experimental/riscv/tests/data/rv/calc.elf differ