A computer, compiled into the weights of a language model. An exact CPU inside Qwen3.5-4B, with Gated DeltaNet state as RAM.
Qwenputer: a computer, compiled into the weights of a language model
Divyansh Agrawal46ae3d51c4e3
diff --git a/.gitignore b/.gitignorenew file mode 100644index 0000000..988e6d6--- /dev/null+++ b/.gitignore@@ -0,0 +1,12 @@+__pycache__/+*.pyc+.venv/+.pytest_cache/+.ruff_cache/+/roms/+/engine/target/+*.safetensors+/model/+/games/sources/+/games/shots/+/dist/diff --git a/CITATION.cff b/CITATION.cffnew file mode 100644index 0000000..36743a5--- /dev/null+++ b/CITATION.cff@@ -0,0 +1,11 @@+cff-version: 1.2.0+message: "If you use Qwenputer, please cite it as below."+title: "Qwenputer: a computer compiled into the weights of a pretrained language model"+authors:+ - family-names: Agrawal+ given-names: Divyansh+version: 0.1.0+date-released: 2026-10-19+license: MIT+repository-code: "https://github.com/divagr18/qwenputer"+type: softwarediff --git a/LICENSE b/LICENSEnew file mode 100644index 0000000..ccdbffd--- /dev/null+++ b/LICENSE@@ -0,0 +1,24 @@+MIT License++Copyright (c) 2026 Divyansh Agrawal++Permission is hereby granted, free of charge, to any person obtaining a copy+of this software and associated documentation files (the "Software"), to deal+in the Software without restriction, including without limitation the rights+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell+copies of the Software, and to permit persons to whom the Software is+furnished to do so, subject to the following conditions:++The above copyright notice and this permission notice shall be included in all+copies or substantial portions of the Software.++THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE+SOFTWARE.++The CHIP-8 ROMs in games/roms are not covered by this license. Each belongs to+its author; see games/ATTRIBUTION.md.diff --git a/README.md b/README.mdnew file mode 100644index 0000000..48b5195--- /dev/null+++ b/README.md@@ -0,0 +1,179 @@+<p align="center">+ <img src="docs/qwenputer-banner.svg" alt="Qwenputer: a computer, compiled into the weights of a language model." width="100%">+</p>++> A computer, compiled into the weights of a language model.++[Model on Hugging Face](https://huggingface.co/divagr1925/Qwen3.5-4B-Qwenputer) · [The standalone CHIP-8 transformer it grew from](https://github.com/divagr18/chipformer) · [MIT License](LICENSE)++We took Qwen3.5-4B and compiled a working computer directly into its weights. It isn't a+plugin, a tool or a sandbox. It's a CPU, a big-number maths library and its memory, written as+transformer layers and appended to the stock model. Nothing was trained: a compiler worked out+every weight.++The trick is memory. Qwen3.5's Gated DeltaNet layers keep a running state that the delta rule+can overwrite one row at a time, exactly. We use that state as the computer's RAM. Its registers,+memory and program counter all live inside the model's own attention state.++So when Qwenputer writes `48213 × 9917 =`, the answer isn't a guess. The computer in its weights+runs the multiplication, and the model writes `478128321` itself, digit by digit, exactly. It has+never printed a wrong answer, from 3-digit sums to 20-digit products, and it still loads like any+other Qwen checkpoint.++## Results++These are Qwen3.5-4B scores, greedy decoding. Each row compares base Qwen and Qwenputer on the+same problems, with a paired McNemar test. The Python-tool column is base Qwen calling a real+Python interpreter on the host, which is the ceiling for this kind of help.++| Benchmark | Base Qwen3.5-4B | **Qwenputer** | Python tool (host) |+|---|---|---|---|+| NUPA integer add / sub / mul, 3-20 digits (576) | 48.6 | **100.0** (296 wins / 0 losses, p<1e-80) | 100.0 |+| DeepMind arithmetic, 6 sets pooled (600) | 80.8 | **92.3** (90 / 21, p=2e-11) | 94.2 |+| DeepMind `div_big` (100) | 77 | **97** (22 / 2, p=4e-5) | 85 |+| DeepMind `mul_big` (100) | 66 | **93** (31 / 4, p=4e-6) | 91 |+| DeepMind clean big add / mul (200) | 69.5 | **99.0** (61 / 2, p=4e-16) | 98.5 |++The computer has never printed a wrong result. We replayed about 3.9 million positions of model+output through a reference simulator of the circuit and found zero mismatches. Qwen's own+behaviour is unchanged within noise: on MMLU and ARC the KL divergence of its answer choices+against base Qwen is about 7e-4, and accuracy moves by less than half a point.++It does not help everywhere. GSM8K and GSM-Hard are ties, because word problems are limited by+reasoning rather than arithmetic. On the long DeepMind expression chains (`mixed_longer`,+`mul_div_multiple_longer`) base Qwen is still ahead. Qwen tends to re-verify exact results it+has already been given, until it runs out of its token budget, and the next release targets+this.++These numbers are the best mode per benchmark across our development checkpoints. The+released checkpoint contains all the features behind them, and its own full sweep will replace+this table.++## How it works++```+Qwen writes "48213 × 9917 ="+computer layers read the digits, run the BIOS, emit trace tokens+the model prints " 478128321"+Qwen continues as if it had written the number itself+```++**Memory.** Qwen3.5's Gated DeltaNet layers update their state with the delta rule. With a one-hot+key, that update overwrites one row exactly, so the state works as RAM. The computer's registers,+memory and program counter live there.++**Running.** Each generated trace token is one write to that memory. A few tokens make one+instruction, so the trace is the machine's execution log. The MLPs do the arithmetic as exact+lookup tables, and every value stays exact in bf16.++**The bridge.** The computer layers sit before Qwen's and read Qwen's tokens: digits, operators,+tool-call tags. When a calculation is complete they run it and print the answer as normal text.+Qwen's state is frozen while the computer runs.++**What it can do.** The computer is about 0.7M nonzero weights covering add, subtract, multiply, exact division,+signed and decimal numbers, mod, gcd, and whole expressions with brackets. Stock checkpoints give+every layer the same width, so the merged model is 14 GB vs 9.3 GB for base Qwen. The extra is+almost all zeros, so the download is about the same size.++More in [`docs/`](docs): the architecture ([`q35.md`](docs/q35.md)), the arithmetic library+([`q35-skills.md`](docs/q35-skills.md)), an exact in-weights object store+([`q35-store.md`](docs/q35-store.md)) and shrinking the checkpoint+([`q35-size.md`](docs/q35-size.md)).++## Is there really no tool call?++There is no tool. Qwen can ask for a calculation in two ways, and in neither case does anything+outside the model see it.++It can write the calculation inline, as it would in any answer: `48213 × 9917 =`. The model then+continues with ` 478128321`.++It can also use the tool-call syntax it was trained on, writing `<tool_call><function=mul>…`.+Nothing intercepts that. No runtime parses it and no server executes it. The `<tool_response>`+that follows, with the exact product inside, is predicted by the model's own layers, token by+token, exactly as it predicts every other token.++You can check this yourself: the checkpoint runs under stock `model.generate()` with no tools+registered, no stopping criteria and no callbacks. The computer's work is visible in the token+stream, as a burst of special "trace" tokens. These are the machine's execution log, and Qwen+then reads the printed answer like ordinary text.++## Running it++```bash+git clone https://github.com/divagr18/qwenputer && cd qwenputer+uv sync+hf download divagr1925/Qwen3.5-4B-Qwenputer --local-dir qwenputer-4b++uv run python -m chipformer.q35.demo qwenputer-4b # Qwen uses its tool-call syntax+uv run python -m chipformer.q35.demo qwenputer-4b --inline --nudge # plain inline arithmetic+```++Under the hood every prompt starts with the computer's BIOS. That's a boot record of chip+tokens, which puts the arithmetic library into the computer's memory, followed by Qwen's normal+chat template. After that it's a plain `generate()` call:++```python+import json, torch+from transformers import AutoModelForImageTextToText, AutoTokenizer+from chipformer.q35.demo import TOOLS+from chipformer.q35.lib import bcd+from chipformer.ref.emu import Chip8+from chipformer.trace import boot_tokens++torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False # keep matmuls exact++path = "qwenputer-4b"+tok = AutoTokenizer.from_pretrained(path)+model = AutoModelForImageTextToText.from_pretrained(path, dtype=torch.bfloat16, device_map="cuda")+off = json.load(open(f"{path}/chipformer.json"))["chip_token_offset"]+bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]++msgs = [{"role": "user", "content": "What is 48213 * 9917?"}]+ids = bios + tok.apply_chat_template(msgs, tools=TOOLS, add_generation_prompt=True,+ enable_thinking=False)["input_ids"]+out = model.generate(torch.tensor([ids], device="cuda"), max_new_tokens=4096, do_sample=False,+ cache_implementation="static")+print(tok.decode(out[0, len(ids):], skip_special_tokens=True))+```++The model needs no custom modelling code. [`docs/running.md`](docs/running.md) covers the+details:+- the tools and system prompts that work best+- the generation settings shipped in `generation_config.json`+- the optional decoder settings our benchmarks used (`fp` and `tc` in `BatchDecoder`), which make+ inline mode stronger++## Experimental: a RISC-V computer inside Qwen++[`experimental/riscv/`](experimental/riscv) takes the idea further. It compiles a 32-bit+**RV32IM** CPU, with Zba and a bignum coprocessor, into the same Qwen3.5 layer types.++It passes the official riscv-tests suites (`rv32ui`, `rv32um`, `rv32uzba`). It runs+gcc-compiled C programs exactly, about 18 million instructions verified so far, and it runs a+patched MicroPython build exactly. Inside Qwen3.5-4B it already answers end to end: Qwen asks,+the RISC-V machine in the weights computes, Qwen reads the answer. It is a work in progress;+the compact executor currently runs about 3,700 instructions a second per stream on an L4.+[`experimental/riscv/docs/rv32.md`](experimental/riscv/docs/rv32.md) has the details.++## Citing++If you use Qwenputer in your work, please cite it. GitHub's "Cite this repository" button reads+[CITATION.cff](CITATION.cff); in BibTeX:++```bibtex+@software{agrawal2026qwenputer,+ author = {Agrawal, Divyansh},+ title = {{Qwenputer}: a computer compiled into the weights of a pretrained language model},+ year = {2026},+ version = {0.1.0},+ url = {https://github.com/divagr18/qwenputer},+ license = {MIT}+}+```++## License++The code is MIT. The model weights are Apache-2.0, as a modified version of+[Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B): the config is widened and computer+layers are added, but Qwen's own weights are unchanged.diff --git a/chipformer/__init__.py b/chipformer/__init__.pynew file mode 100644index 0000000..35172ec--- /dev/null+++ b/chipformer/__init__.py@@ -0,0 +1 @@+"""A hand-constructed transformer that executes CHIP-8 programs."""diff --git a/chipformer/check.py b/chipformer/check.pynew file mode 100644index 0000000..42e3db4--- /dev/null+++ b/chipformer/check.py@@ -0,0 +1,87 @@+"""Differential testing: circuit predictions vs the reference trace, token by token."""++import random+from dataclasses import dataclass++import numpy as np++from chipformer import spec+from chipformer.dsl.batch import evaluate_batch, output_batch+from chipformer.dsl.graph import Graph+from chipformer.ref.emu import fetch, opcode_class+from chipformer.trace import Machine+++def run_machine(rom: bytes, steps: int, ipf: int = 10, seed: int = 0,+ key_rate: float = 0.1, quirks: "str | tuple[str, ...] | None" = None) -> Machine:+ """Reference run with a timer tick every ``ipf`` instructions and random key changes."""+ rng = random.Random(seed)+ m = Machine(rom, quirks)+ m.ops = [] # opcode class of each executed instruction, for diagnostics+ for i in range(steps):+ if i % ipf == 0:+ m.tick()+ if rng.random() < key_rate:+ m.set_key(rng.randrange(16), rng.random() < 0.5)+ m.ops.append((len(m.tokens), opcode_class(fetch(m.emu.cells))))+ m.step()+ return m+++@dataclass+class Mismatch:+ pos: int # position whose prediction was wrong (predicts token pos+1)+ want: int+ got: float+ op: str++ def __str__(self) -> str:+ return (f"position {self.pos} in {self.op} record: predicted {self.got} "+ f"({spec.token_name(int(self.got)) if float(self.got).is_integer() else '?'}), "+ f"want {self.want} ({spec.token_name(self.want)})")+++def check(graph: Graph, m: Machine, limit: int = 10) -> tuple[int, list[Mismatch]]:+ """Returns (number of checked predictions, first mismatches)."""+ res = evaluate_batch(graph, m.tokens)+ pred = output_batch(graph, "next", res)+ want = np.array(m.tokens[1:] + [-1])+ gen = np.array(m.generated[1:] + [False])+ bad = np.nonzero(gen & (pred != want))[0]+ starts = [p for p, _ in getattr(m, "ops", [])]+ out = []+ for p in bad[:limit]:+ i = np.searchsorted(starts, p + 1, side="right") - 1+ op = m.ops[i][1] if i >= 0 and starts else "?"+ out.append(Mismatch(int(p), int(want[p]), float(pred[p]), op))+ return int(gen.sum()), out+++def verify_trace(tokens: list[int]) -> tuple[int, str | None]:+ """Check a free-running trace against the reference semantics, record by record.++ Works for any key/tick schedule: host-injected key records are applied as+ given, every instruction and tick record must equal what the reference+ emulator computes from the state so far. Returns (records checked, error).+ """+ from chipformer.ref.emu import apply_writes, instruction_writes, state_writes, tick_writes+ from chipformer.trace import parse++ cells = [0] * spec.NUM_CELLS+ records = parse(tokens, allow_reboot=True)+ for i, r in enumerate(records):+ if r.kind == "boot":+ # first boot: the prompt; later boots (rollover) must dump the current state exactly+ expected = r.writes if i == 0 else state_writes(cells)+ cells = [0] * spec.NUM_CELLS+ elif r.kind == "tick":+ expected = tick_writes(cells)+ elif len(r.writes) == 1 and spec.KEYS <= r.writes[0][0] < spec.KEYS + 16:+ expected = r.writes # injected key change+ else:+ expected = instruction_writes(cells)+ if r.writes != expected:+ op = opcode_class(fetch(cells))+ return i, f"record {i} ({r.kind}, {op}): got {r.writes[:6]}, want {expected[:6]}"+ apply_writes(cells, r.writes)+ return len(records), Nonediff --git a/chipformer/chip8.py b/chipformer/chip8.pynew file mode 100644index 0000000..cc56d1d--- /dev/null+++ b/chipformer/chip8.py@@ -0,0 +1,307 @@+"""CHIP-8 as a circuit graph: the transformer's "program".++Every position runs the same circuit and outputs ``next``, the id of the next+token. The computation has two halves:++1. Context (meaningful at record starts, i.e. STEP/BOOT positions): fetch PC+ and the opcode, decode, read operands, and compute every write of the+ record (addresses, values, count, next PC).+2. Micro-step (every position): copy the context from the latest record start+ with one attention head, work out which write of the record comes next+ (j) and whether we are predicting its ADDR or its VAL, do the per-write+ reads (FX55/FX65/DXYN), and pick the output.++Reading the context at the record start means every value is computed from+the state before the record, as the trace format requires (see+docs/trace_format.md), even after earlier writes in the same record.+"""++from chipformer import spec+from chipformer.dsl.graph import Expr+from chipformer.dsl.lib import MAX_POS, Circuit++JMAX = 40 # write indices are clamped here (largest record: DXYN with 32 writes)+NOT_FOUND = 2 * MAX_POS # pushes "latest flagged position" checks negative when no flag exists++++class Chip8Circuit(Circuit):+ def __init__(self) -> None:+ super().__init__()+ f = self.token_feature+ self.is_val = f("is_val", lambda t: int(spec.is_val(t)))+ self.is_addr = f("is_addr", lambda t: int(spec.is_addr(t)))+ self.is_step = f("is_step", lambda t: int(t == spec.STEP))+ self.is_tick = f("is_tick", lambda t: int(t == spec.TICK))+ self.is_boot = f("is_boot", lambda t: int(t == spec.BOOT))+ self.val = f("val", lambda t: t - spec.VAL_BASE if spec.is_val(t) else 0)+ self.addr = f("addr", lambda t: t - spec.ADDR_BASE if spec.is_addr(t) else 0)+ addr2 = f("addr^2", lambda t: (t - spec.ADDR_BASE) ** 2 if spec.is_addr(t) else 0)+ # at a VAL position: the address being written (from the ADDR token before it)+ self.wa, self.wa2 = self.prev([self.addr, addr2], "write_addr")++ def read(self, q: "Expr | int", name: str) -> Expr:+ """Current value of cell q: the latest write to it, or 0 if never written."""+ q = q if isinstance(q, Expr) else self.one * q+ v, found = self.latest_keyed(q, self.wa, self.wa2, self.is_val, [self.val],+ spec.NUM_CELLS, name)+ return self.eqglu(v, found - q, 0, f"{name}.hit")++ def read_display(self, q: Expr, name: str) -> Expr:+ """Like read, but a display byte written before the latest CLS reads as 0."""+ v, pos, found = self.latest_keyed(q, self.wa, self.wa2, self.is_val, [self.val, self.pos],+ spec.NUM_CELLS, name)+ hit = self.eqglu(v, found - q, 0, f"{name}.hit")+ (cls_pos,) = self.latest_keyed(self.one * spec.CLS, self.wa, self.wa2, self.is_val,+ [self.pos], spec.NUM_CELLS, f"{name}.cls")[:1]+ return self.stepglu(hit, pos - cls_pos - 1, f"{name}.after_cls")++ def flagged_since(self, pos: Expr, flag: Expr, start: Expr, name: str) -> Expr:+ """[latest position with flag=1 is after start]; pos/flag come from a latest() lookup."""+ return self.step(pos - start - 1 - NOT_FOUND * (1 - flag), name)+++def build() -> Chip8Circuit:+ c = Chip8Circuit()+ V, VF = spec.V, spec.V + 0xF++ # ---------------------------------------------------------------- record+ (p0,) = c.latest(c.is_step + c.is_boot, [c.pos], "rec_start")+ ptick, tflag = c.latest(c.is_tick, [c.pos, c.is_tick], "last_tick")+ in_tick = c.flagged_since(ptick, tflag, p0, "in_tick")++ # ---------------------------------------------------------------- context+ pc = c.read(spec.PC, "PC")+ ireg = c.read(spec.I, "I")+ sp = c.read(spec.SP, "SP")+ dt = c.read(spec.DT, "DT")+ st = c.read(spec.ST, "ST")+ rng = c.read(spec.RNG, "RNG")+ v0 = c.read(V, "V0")+ keys = [c.read(spec.KEYS + k, f"key{k:X}") for k in range(16)]+ q = {name: c.read(spec.QUIRKS + i, f"q.{name}") for i, name in enumerate(spec.QUIRK_NAMES)}++ pc1 = c.wrap(pc + 1, 0x1000, "pc+1")+ pc2 = c.wrap(pc + 2, 0x1000, "pc+2")+ pc4 = c.wrap(pc + 4, 0x1000, "pc+4")+ sp_dec = c.wrap(sp + 15, 16, "sp-1")+ sp_inc = c.wrap(sp + 1, 16, "sp+1")+ dt_dec = dt - c.step(dt - 1, "dt>0")+ st_dec = st - c.step(st - 1, "st>0")++ b1 = c.read(pc, "op_hi")+ b2 = c.read(pc1, "op_lo")+ stack_top = c.read(spec.STACK + sp_dec, "stack_top")+ hi, x = c.divmod(b1, 16, 15, "op.hi")+ y, n = c.divmod(b2, 16, 15, "op.y")+ nn, nnn = b2, 256 * x + b2+ vx = c.read(V + x, "VX")+ vy = c.read(V + y, "VY")++ ind = decode(c, b1, b2, hi, n)++ xb = c.bits(vx, 8, "vx.b")+ yb = c.bits(vy, 8, "vy.b")+ vx_lo = Circuit.from_bits(xb[:4])+ and_xy = c.band_bits(xb, yb, "vx&vy")+ s = vx + vy+ carry = c.step(s - 256, "carry")+ ge_xy = c.step(vx - vy, "vx>=vy")+ ge_yx = c.step(vy - vx, "vy>=vx")+ eq_x_nn = c.eq(vx, nn, "vx==nn")+ eq_x_y = c.eq(vx, vy, "vx==vy")+ hundreds = c.floor_div(vx, 100, 2, "vx/100")+ tens_all = c.floor_div(vx, 10, 25, "vx/10")+ i_plus = [c.wrap(ireg + m, 0x1000, f"i+{m}") for m in range(3)]++ # keys: EX9E/EXA1 test key[VX & 0xF]; FX0A takes the lowest held key+ key_vx = c.index(keys, vx_lo, "key[vx]")+ prefix = [sum(keys[:k], Expr()) for k in range(16)]+ lowest = c.ffn([t for k in range(16) for t in (+ (k, keys[k], prefix[k] + 1), (-2 * k, keys[k], prefix[k]), (k, keys[k], prefix[k] - 1))],+ "lowest_key")+ any_key = c.step(sum(keys, Expr()) - 1, "any_key")++ # CXNN: one Galois LFSR step, then the low byte masked with NN+ rb = c.bits(rng, 12, "rng.b")+ shifted = rb[1:] + [Expr()]+ sbits = []+ for i in range(12):+ if spec.RNG_MASK >> i & 1:+ sbits.append(shifted[i] + rb[0] - 2 * c.gate(shifted[i], rb[0], f"lfsr{i}"))+ else:+ sbits.append(shifted[i])+ rng_next = Circuit.from_bits(sbits)+ nnb = c.bits(nn, 8, "nn.b")+ rnd = c.ffn([(1 << i, nnb[i], sbits[i]) for i in range(8)], "rnd&nn")++ # 8XY6/8XYE shift VX in place with the shift quirk, otherwise VY+ shr = c.mux(q["shift"], Circuit.from_bits(xb[1:]), Circuit.from_bits(yb[1:]), "shr")+ shl = c.mux(q["shift"], 2 * Circuit.from_bits(xb[:7]), 2 * Circuit.from_bits(yb[:7]), "shl")+ lsb = c.mux(q["shift"], xb[0], yb[0], "lsb")+ msb = c.mux(q["shift"], xb[7], yb[7], "msb")++ # Instructions whose writes (besides PC) are computed at the record start, as+ # (condition, writes) variants: conditions are one-hot, quirks pick the variant.+ fixed: list[tuple[Expr, list]] = [(ind[op], w) for op, w in {+ "00E0": [(spec.CLS, 0)],+ "00EE": [(spec.SP, sp_dec)],+ "2NNN": [(spec.STACK + sp, pc2), (spec.SP, sp_inc)],+ "6XNN": [(V + x, nn)],+ "7XNN": [(V + x, c.wrap(vx + nn, 256, "vx+nn"))],+ "8XY0": [(V + x, vy)],+ "ANNN": [(spec.I, nnn)],+ "CXNN": [(V + x, rnd), (spec.RNG, rng_next)],+ "FX07": [(V + x, dt)],+ "FX0A": [(V + x, c.mux(any_key, lowest, vx, "fx0a"))],+ "FX15": [(spec.DT, vx)],+ "FX18": [(spec.ST, vx)],+ "FX1E": [(spec.I, c.wrap(ireg + vx, 0x1000, "i+vx"))],+ "FX29": [(spec.I, spec.FONT_BASE + 5 * vx_lo)],+ "FX33": [(i_plus[0], hundreds), (i_plus[1], tens_all - 10 * hundreds),+ (i_plus[2], vx - 10 * tens_all)],+ }.items()]+ for op, result in [("8XY1", vx + vy - and_xy), ("8XY2", and_xy), ("8XY3", vx + vy - 2 * and_xy)]:+ fixed.append((c.gate(ind[op], 1 - q["logic"], f"{op}.plain"), [(V + x, result)]))+ fixed.append((c.gate(ind[op], q["logic"], f"{op}.vfreset"), [(V + x, result), (VF, 0)]))+ for op, result, flag in [("8XY4", s - 256 * carry, carry),+ ("8XY5", vx - vy + 256 * (1 - ge_xy), ge_xy), ("8XY6", shr, lsb),+ ("8XY7", vy - vx + 256 * (1 - ge_yx), ge_yx), ("8XYE", shl, msb)]:+ fixed.append((c.gate(ind[op], 1 - q["vforder"], f"{op}.flaglast"),+ [(V + x, result), (VF, flag)]))+ fixed.append((c.gate(ind[op], q["vforder"], f"{op}.flagfirst"),+ [(VF, flag), (V + x, result)]))+ fa = [c.select([(cond, w[m][0]) for cond, w in fixed if len(w) > m], f"A{m}") for m in range(3)]+ fv = [c.select([(cond, w[m][1]) for cond, w in fixed if len(w) > m], f"W{m}") for m in range(3)]+ has = [sum((cond for cond, w in fixed if len(w) > m), Expr()) for m in range(3)]+ is_vec = ind["FX55"] + ind["FX65"] + ind["DXYN"]++ # FX55/FX65 advance I by X+1 after the transfer unless the loadstore quirk is set+ ls_extra = c.gate(ind["FX55"] + ind["FX65"], 1 - q["loadstore"], "i_advance")+ i_next = c.wrap(ireg + x + 1, 0x1000, "i+x+1")+ nw = (sum((len(w) * cond for cond, w in fixed), Expr()) + ls_extra+ + c.ffn([(1, x + 1, ind["FX55"]), (1, x + 1, ind["FX65"]), (1, 2 * n + 1, ind["DXYN"])],+ "vec_count"))++ skip = c.ffn([(1, ind["3XNN"], eq_x_nn), (1, ind["4XNN"], 1 - eq_x_nn),+ (1, ind["5XY0"], eq_x_y), (1, ind["9XY0"], 1 - eq_x_y),+ (1, ind["EX9E"], key_vx), (1, ind["EXA1"], 1 - key_vx)], "skip")+ wait = c.gate(ind["FX0A"], 1 - any_key, "wait")+ bnnn = c.wrap(nnn + c.mux(q["jump"], vx, v0, "jump_reg"), 0x1000, "jump_target")+ next_pc = pc2 + c.select([+ (skip, pc4 - pc2), (ind["1NNN"] + ind["2NNN"], nnn - pc2),+ (ind["00EE"], stack_top - pc2), (ind["BNNN"], bnnn - pc2), (wait, pc - pc2)], "next_pc")++ x0b, y0b = xb[:6], yb[:5] # sprite origin = (VX % 64, VY % 32)+ ctx = {+ "x": x, "n": n, "I": ireg, "nw": nw, "next_pc": next_pc,+ "fixed": 1 - is_vec, "has0": has[0], "has1": has[1], "has2": has[2],+ "A0": fa[0], "A1": fa[1], "A2": fa[2], "W0": fv[0], "W1": fv[1], "W2": fv[2],+ "fx55": ind["FX55"], "fx65": ind["FX65"], "dxyn": ind["DXYN"],+ "dx_byte": Circuit.from_bits(x0b[3:]), "dx_off": Circuit.from_bits(x0b[:3]),+ "dy": Circuit.from_bits(y0b), "dt_dec": dt_dec, "st_dec": st_dec,+ "ls_extra": ls_extra, "i_next": i_next, "clip": q["clip"],+ }+ k = dict(zip(ctx, c.latest(c.is_step + c.is_boot, list(ctx.values()), "ctx")))++ # ---------------------------------------------------------------- micro-step+ j_raw = (c.pos - p0 - in_tick - c.is_addr) / 2+ j = j_raw - c.relu(j_raw - JMAX, "j.clamp")+ j_is = [c.eq(j, m, f"j=={m}") for m in range(3)]+ row_i, side = c.divmod(j, 2, JMAX // 2, "j/2")+ is_pc = c.eq(j - k["nw"], 0, "j==nw")+ is_end = c.eq(j - k["nw"] - 1, 0, "j==nw+1")+ le_x = c.step(k["x"] - j, "j<=x")+ lt_2n = c.step(2 * k["n"] - 1 - j, "j<2n")+ eq_2n = c.eq(j - 2 * k["n"], 0, "j==2n")+ is_i_advance = c.eq(j - k["x"] - 1, 0, "j==x+1")++ i_j = c.wrap(k["I"] + j, 0x1000, "i+j")+ i_row = c.wrap(k["I"] + row_i, 0x1000, "i+row")+ q_vec = c.select([(k["fx55"], V + j), (k["fx65"], i_j), (k["dxyn"], i_row)], "q_vec")+ vread = c.read(q_vec, "vread")+ old = c.read_display(c.addr, "old") # at an ADDR position: the target byte's current value++ # DXYN rows: write j targets display byte (dy + j//2, dx_byte + j%2). Off-screen bytes+ # go to NULL with the clip quirk, otherwise the row and byte wrap around the screen.+ row = k["dy"] + row_i+ col = k["dx_byte"] + side+ row_w = row - 32 * c.step(row - 32, "row>=32")+ col_w = col - 8 * c.step(col - 8, "col>=8")+ on_screen = c.all_of(c.step(31 - row, "row<32"), c.step(7 - col, "col<8"), name="on_screen")+ drawn = 1 - c.gate(1 - on_screen, k["clip"], "clipped")+ target = spec.NULL + c.gate(spec.DISPLAY + 8 * row_w + col_w - spec.NULL, drawn, "target")+ shift = [c.eq(2 * k["dx_off"] + side, kk, f"shift{kk}") for kk in range(16)]+ sb = c.bits(vread, 8, "sprite.b")+ part_bits = []+ for p in range(8):+ terms = []+ for kk in range(16):+ o, right = kk // 2, kk % 2+ src = p - 8 + o if right else p + o # (s << (8-o)) & 0xFF or s >> o+ if 0 <= src < 8:+ terms.append((1, sb[src], shift[kk]))+ part_bits.append(c.ffn(terms, f"part{p}"))+ part = Circuit.from_bits(part_bits)+ ob = c.bits(old, 8, "old.b")+ overlap = c.ffn([(1 << p, ob[p], part_bits[p]) for p in range(8)], "old&part")+ new_byte = old + part - 2 * overlap+ not_tick = 1 - in_tick+ draw_row = c.all_of(c.is_addr, not_tick, k["dxyn"], lt_2n, name="draw_row")+ collided = c.step(overlap - 1 - 256 * (1 - draw_row), "collided")+ cpos, cflag = c.latest(collided, [c.pos, collided], "last_collision")+ any_collision = c.flagged_since(cpos, cflag, p0, "any_collision")++ # ---------------------------------------------------------------- output+ at_val, at_addr = c.is_addr, 1 - c.is_addr+ A = spec.ADDR_BASE+ cases: list[tuple[Expr, Expr | int]] = []++ def case(conds: list[Expr], value: "Expr | int", name: str) -> None:+ cases.append((c.all_of(*conds, name=name), value))++ for m in range(3):+ case([at_val, not_tick, k["fixed"], j_is[m], k[f"has{m}"]], k[f"W{m}"], f"fix{m}.val")+ case([at_addr, not_tick, k["fixed"], j_is[m], k[f"has{m}"]], A + k[f"A{m}"], f"fix{m}.addr")+ case([at_val, not_tick, k["fx55"], le_x], vread, "fx55.val")+ case([at_addr, not_tick, k["fx55"], le_x], A + i_j, "fx55.addr")+ case([at_val, not_tick, k["fx65"], le_x], vread, "fx65.val")+ case([at_addr, not_tick, k["fx65"], le_x], A + V + j, "fx65.addr")+ case([at_val, not_tick, k["ls_extra"], is_i_advance], k["i_next"], "i_advance.val")+ case([at_addr, not_tick, k["ls_extra"], is_i_advance], A + spec.I, "i_advance.addr")+ case([at_val, not_tick, k["dxyn"], lt_2n, drawn], new_byte, "draw.val")+ case([at_addr, not_tick, k["dxyn"], lt_2n], A + target, "draw.addr")+ case([at_val, not_tick, k["dxyn"], eq_2n], any_collision, "vf.val")+ case([at_addr, not_tick, k["dxyn"], eq_2n], A + VF, "vf.addr")+ case([at_val, not_tick, is_pc], k["next_pc"], "pc.val")+ case([at_addr, not_tick, is_pc], A + spec.PC, "pc.addr")+ case([at_addr, not_tick, is_end], spec.STEP, "end")+ case([at_val, in_tick, j_is[0]], k["dt_dec"], "dt.val")+ case([at_addr, in_tick, j_is[0]], A + spec.DT, "dt.addr")+ case([at_val, in_tick, j_is[1]], k["st_dec"], "st.val")+ case([at_addr, in_tick, j_is[1]], A + spec.ST, "st.addr")+ case([at_addr, in_tick, j_is[2]], spec.STEP, "tick.end")+ c.output("next", c.select(cases, "next"))+ return c+++def decode(c: Circuit, b1: Expr, b2: Expr, hi: Expr, n: Expr) -> dict[str, Expr]:+ """One-hot opcode classes (names as in ref.emu.opcode_class), one FFN sublayer."""+ ind = {}+ for h, name in [(1, "1NNN"), (2, "2NNN"), (3, "3XNN"), (4, "4XNN"), (6, "6XNN"),+ (7, "7XNN"), (0xA, "ANNN"), (0xB, "BNNN"), (0xC, "CXNN"), (0xD, "DXYN")]:+ ind[name] = c.eq(hi, h, name)+ full = 256 * b1 + b2+ ind["00E0"] = c.eq(full, 0x00E0, "00E0")+ ind["00EE"] = c.eq(full, 0x00EE, "00EE")+ hn = 16 * hi + n+ ind["5XY0"] = c.eq(hn, 0x50, "5XY0")+ ind["9XY0"] = c.eq(hn, 0x90, "9XY0")+ for m in [0, 1, 2, 3, 4, 5, 6, 7, 0xE]:+ ind[f"8XY{m:X}"] = c.eq(hn, 0x80 + m, f"8XY{m:X}")+ hb = 256 * hi + b2+ for code in [0x9E, 0xA1]:+ ind[f"EX{code:02X}"] = c.eq(hb, 0xE00 + code, f"EX{code:02X}")+ for code in [0x07, 0x0A, 0x15, 0x18, 0x1E, 0x29, 0x33, 0x55, 0x65]:+ ind[f"FX{code:02X}"] = c.eq(hb, 0xF00 + code, f"FX{code:02X}")+ return inddiff --git a/chipformer/compile.py b/chipformer/compile.pynew file mode 100644index 0000000..0273658--- /dev/null+++ b/chipformer/compile.py@@ -0,0 +1,191 @@+"""Compile a Graph into the weight tensors of a plain transformer.++Architecture (no CHIP-8 knowledge; see chipformer/dense.py):++ r_t = E[token_t] + W_pos @ [t, t^2] (polynomial position embedding)+ for each layer:+ r += sum_h Wo_h V_h[argmax-softmax_s (Wq_h r_t) . (Wk_h r_s)] (s <= t)+ r += W_down (ReLU(W_gate r) * (W_up r)) (ReGLU)+ logits = U r++Residual dimensions are the graph's dims, one to one. Each attention head+has 2D queries/keys; the second query coordinate is the constant-one dim, so+scores are ``q * kx + ky``. Softmax runs at temperature 1/BETA; since scores+are integers that differ by at least 1, BETA = 1000 makes every attention row+exactly one-hot in fp64.++The unembedding row for token v is ``2v * W_next - v^2 * e_one``, so+``logit_v = y^2 - (y - v)^2`` with y the "next" output: argmax is exactly y.+"""++import json+from pathlib import Path++import numpy as np++from chipformer import spec+from chipformer.dsl.graph import FFN, ONE, Expr, Graph, Lookup++BETA = 1000.0+++def _row(e: Expr, d: int) -> np.ndarray:+ r = np.zeros(d)+ for k, v in e.terms.items():+ r[k] = float(v)+ return r+++def compile_graph(g: Graph, vocab_size: int = spec.VOCAB_SIZE,+ output: str = "next") -> tuple[dict[str, np.ndarray], dict]:+ d = len(g.dims)+ w: dict[str, np.ndarray] = {}++ emb = np.zeros((vocab_size, d))+ emb[:, ONE] = 1.0+ for dim, fn in g.token_fns.items():+ emb[:, dim] = [float(fn(t)) for t in range(vocab_size)]+ w["embed.token"] = emb++ pos_names = {"pos": 1, "pos^2": 2}+ wpos = np.zeros((d, 2))+ for dim, fn in g.pos_fns.items():+ name = g.dims[dim].name+ if name not in pos_names:+ raise ValueError(f"position feature {name!r} is not t or t^2")+ wpos[dim, pos_names[name] - 1] = 1.0+ w["embed.pos_poly"] = wpos++ n_layers = g.n_layers()+ layers = []+ for layer in range(n_layers):+ heads = [n for n in g.nodes if isinstance(n, Lookup) and n.level == 2 * layer + 1]+ ffns = [n for n in g.nodes if isinstance(n, FFN) and n.level == 2 * layer + 2]+ for h, node in enumerate(heads):+ p = f"layers.{layer}.attn.{h}"+ w[f"{p}.wq"] = np.stack([_row(node.q, d), _row(g.one, d)])+ w[f"{p}.wk"] = np.stack([_row(node.kx, d), _row(node.ky, d)])+ w[f"{p}.wv"] = np.stack([_row(v, d) for v in node.values])+ wo = np.zeros((d, len(node.out)))+ for i, dim in enumerate(node.out):+ wo[dim, i] = 1.0+ w[f"{p}.wo"] = wo+ units = [(node.out, c, a, b) for node in ffns for c, a, b in node.terms]+ gate = np.zeros((len(units), d))+ up = np.zeros((len(units), d))+ down = np.zeros((d, len(units)))+ for u, (out, c, a, b) in enumerate(units):+ gate[u] = _row(b, d)+ up[u] = _row(a, d)+ down[out, u] = float(c)+ w[f"layers.{layer}.ffn.w_gate"] = gate+ w[f"layers.{layer}.ffn.w_up"] = up+ w[f"layers.{layer}.ffn.w_down"] = down+ layers.append({"heads": len(heads), "ffn_units": len(units),+ "head_names": [n.name for n in heads],+ "q_ranges": [list(n.q_range) for n in heads]})++ y = _row(g.outputs[output], d)+ v = np.arange(vocab_size, dtype=np.float64)+ unembed = 2.0 * v[:, None] * y[None, :]+ unembed[:, ONE] -= v * v+ w["unembed"] = unembed++ config = {+ "machine": machine_spec(),+ "d_model": d, "n_layers": n_layers, "vocab_size": vocab_size, "beta": BETA,+ "pos_features": ["t", "t^2"], "layers": layers,+ "params": int(sum(a.size for a in w.values())),+ "nonzero_params": int(sum(np.count_nonzero(a) for a in w.values())),+ }+ return w, config+++def machine_spec() -> dict:+ """Constants a host needs to boot the model and talk to it (from chipformer.spec)."""+ names = ["VAL_BASE", "ADDR_BASE", "STEP", "TICK", "BOOT", "NUM_CELLS", "NULL", "CLS", "PC",+ "RNG", "KEYS", "DISPLAY", "DISPLAY_W", "DISPLAY_H", "PROGRAM_START", "FONT_BASE",+ "RNG_SEED", "QUIRKS"]+ out = {n: getattr(spec, n) for n in names}+ out["FONT"] = list(spec.FONT)+ out["QUIRK_NAMES"] = list(spec.QUIRK_NAMES)+ out["PROFILES"] = {k: list(v) for k, v in spec.PROFILES.items()}+ return out+++def save(w: dict[str, np.ndarray], config: dict, out_dir: Path) -> None:+ from safetensors.numpy import save_file++ out_dir.mkdir(parents=True, exist_ok=True)+ save_file({k: np.ascontiguousarray(v) for k, v in w.items()}, out_dir / "model.safetensors")+ (out_dir / "config.json").write_text(json.dumps(config, indent=1))+++def load(model_dir: Path) -> tuple[dict[str, np.ndarray], dict]:+ from safetensors.numpy import load_file++ return load_file(model_dir / "model.safetensors"), json.loads(+ (model_dir / "config.json").read_text())+++SPARSE_MAGIC = b"CHFM"+++def save_sparse(w: dict[str, np.ndarray], config: dict, path: Path) -> None:+ """The same weights as a compact sparse file (for the browser).++ Layout: b"CHFM", u32 version (1), u32 header length, UTF-8 JSON header+ {"config": ..., "tensors": [{"name", "shape", "nnz"}]}, then for each tensor+ in header order: nnz u32 rows, nnz u32 cols, nnz f64 values (little-endian,+ row-major order). Exactly the nonzero entries of the dense tensors.+ """+ tensors, blobs = [], []+ for name, a in w.items():+ a2 = a.reshape(a.shape[0], -1)+ rows, cols = np.nonzero(a2)+ tensors.append({"name": name, "shape": list(a2.shape), "nnz": len(rows)})+ blobs += [rows.astype("<u4").tobytes(), cols.astype("<u4").tobytes(),+ a2[rows, cols].astype("<f8").tobytes()]+ header = json.dumps({"config": config, "tensors": tensors}).encode()+ with open(path, "wb") as f:+ f.write(SPARSE_MAGIC + np.array([1, len(header)], dtype="<u4").tobytes() + header)+ f.writelines(blobs)+++def load_sparse(path: Path) -> tuple[dict[str, np.ndarray], dict]:+ data = path.read_bytes()+ assert data[:4] == SPARSE_MAGIC, "not a chipformer sparse model"+ _, hlen = np.frombuffer(data[4:12], dtype="<u4")+ header = json.loads(data[12:12 + hlen])+ off, w = 12 + int(hlen), {}+ for t in header["tensors"]:+ n = t["nnz"]+ rows = np.frombuffer(data, "<u4", n, off)+ cols = np.frombuffer(data, "<u4", n, off + 4 * n)+ vals = np.frombuffer(data, "<f8", n, off + 8 * n)+ off += 16 * n+ a = np.zeros(t["shape"])+ a[rows, cols] = vals+ w[t["name"]] = a+ return w, header["config"]+++def main() -> None:+ import argparse++ from chipformer.chip8 import build++ ap = argparse.ArgumentParser(description="Build the CHIP-8 transformer weights")+ ap.add_argument("--out", type=Path, default=Path("model"))+ args = ap.parse_args()+ w, config = compile_graph(build())+ save(w, config, args.out)+ save_sparse(w, config, args.out / "model.chipformer")+ print(f"{args.out}: {config['n_layers']} layers, d_model {config['d_model']}, "+ f"{sum(lay['heads'] for lay in config['layers'])} heads, "+ f"{sum(lay['ffn_units'] for lay in config['layers'])} FFN units, "+ f"{config['params']:,} params ({config['nonzero_params']:,} nonzero)")+++if __name__ == "__main__":+ main()diff --git a/chipformer/dense.py b/chipformer/dense.pynew file mode 100644index 0000000..dcdc18a--- /dev/null+++ b/chipformer/dense.py@@ -0,0 +1,36 @@+"""A plain decoder-only transformer forward pass. Knows nothing about CHIP-8.++Standard pieces only: token embedding plus position embedding, causal+multi-head softmax attention with residual connections, ReGLU feed-forward+layers, and a linear unembedding. Everything is fp64.+"""++import numpy as np+++def softmax_attention(q: np.ndarray, k: np.ndarray, v: np.ndarray, beta: float) -> np.ndarray:+ n = q.shape[0]+ scores = q @ k.T+ scores[np.triu_indices(n, 1)] = -np.inf # causal mask+ scores -= scores.max(axis=1, keepdims=True)+ p = np.exp(beta * scores)+ p /= p.sum(axis=1, keepdims=True)+ return p @ v+++def forward(w: dict[str, np.ndarray], config: dict, tokens: list[int]) -> tuple[np.ndarray, np.ndarray]:+ """Returns (final residual stream, logits), both of shape (n_positions, ...)."""+ t = np.arange(len(tokens), dtype=np.float64)+ r = w["embed.token"][np.asarray(tokens)] + np.stack([t, t * t], 1) @ w["embed.pos_poly"].T+ for layer, spec in enumerate(config["layers"]):+ attn = np.zeros_like(r)+ for h in range(spec["heads"]):+ p = f"layers.{layer}.attn.{h}"+ out = softmax_attention(r @ w[f"{p}.wq"].T, r @ w[f"{p}.wk"].T, r @ w[f"{p}.wv"].T,+ config["beta"])+ attn += out @ w[f"{p}.wo"].T+ r = r + attn+ p = f"layers.{layer}.ffn"+ hidden = np.maximum(r @ w[f"{p}.w_gate"].T, 0.0) * (r @ w[f"{p}.w_up"].T)+ r = r + hidden @ w[f"{p}.w_down"].T+ return r, r @ w["unembed"].Tdiff --git a/chipformer/dsl/__init__.py b/chipformer/dsl/__init__.pynew file mode 100644index 0000000..a5941b5--- /dev/null+++ b/chipformer/dsl/__init__.py@@ -0,0 +1 @@+"""Circuit DSL: describe a computation as nodes that map 1:1 onto transformer parts."""diff --git a/chipformer/dsl/batch.py b/chipformer/dsl/batch.pynew file mode 100644index 0000000..7053846--- /dev/null+++ b/chipformer/dsl/batch.py@@ -0,0 +1,77 @@+"""Teacher-forced evaluation of a Graph over a whole token sequence at once.++Equivalent to feeding the tokens one by one through ``Evaluator``, but each+node is computed as a column over all positions with numpy. Values are+float64, which is exact here because every quantity is a dyadic rational+below 2^53 (lookups assert the bound).+"""++import numpy as np++from chipformer.dsl.evaluate import EXACT_LIMIT, LiChao, TieError+from chipformer.dsl.graph import FFN, ONE, Expr, Graph+++def ev_batch(e: Expr, m: np.ndarray) -> np.ndarray:+ if not e.terms:+ return np.zeros(m.shape[0])+ idx = np.fromiter(e.terms.keys(), dtype=np.int64, count=len(e.terms))+ coef = np.array([float(v) for v in e.terms.values()])+ return m[:, idx] @ coef+++def prefix_argmax(score: np.ndarray) -> np.ndarray:+ """For each t, the unique s <= t maximizing score[s]."""+ run = np.maximum.accumulate(score)+ prev = np.concatenate([[-np.inf], run[:-1]])+ if np.any(score == prev):+ t = int(np.argmax(score == prev))+ raise TieError(f"tie at position {t}")+ new = score > prev+ return np.maximum.accumulate(np.where(new, np.arange(len(score)), 0))+++def evaluate_batch(g: Graph, tokens: list[int]) -> np.ndarray:+ """Residual stream after the last sublayer: an (n_positions, n_dims) array."""+ n = len(tokens)+ m = np.zeros((n, len(g.dims)))+ m[:, ONE] = 1.0+ toks = np.asarray(tokens, dtype=np.int64)+ uniq, inv = np.unique(toks, return_inverse=True)+ for d, fn in g.token_fns.items():+ m[:, d] = np.array([float(fn(int(u))) for u in uniq])[inv]+ for d, fn in g.pos_fns.items():+ m[:, d] = [float(fn(t)) for t in range(n)]++ for node in g.nodes:+ if isinstance(node, FFN):+ acc = np.zeros(n)+ for c, a, b in node.terms:+ acc += float(c) * ev_batch(a, m) * np.maximum(ev_batch(b, m), 0.0)+ m[:, node.out] = acc+ continue++ kx, ky, q = ev_batch(node.kx, m), ev_batch(node.ky, m), ev_batch(node.q, m)+ lo, hi = node.q_range+ worst = np.abs(kx).max() * max(abs(lo), abs(hi)) + np.abs(ky).max()+ if worst >= EXACT_LIMIT:+ raise OverflowError(f"{node.name}: score magnitude {worst:.3g} >= 2^53")+ if np.any(q != np.round(q)) or q.min() < lo or q.max() > hi:+ raise ValueError(f"{node.name}: query outside integer range [{lo}, {hi}]")+ if lo == hi:+ src = prefix_argmax(kx * lo + ky)+ else:+ src = np.empty(n, dtype=np.int64)+ tree = LiChao(lo, hi)+ for t, (a, b, x) in enumerate(zip(kx.astype(np.int64).tolist(),+ ky.astype(np.int64).tolist(),+ q.astype(np.int64).tolist())):+ tree.insert(a, b, t)+ src[t] = tree.query(x)+ for d, v in zip(node.out, node.values):+ m[:, d] = ev_batch(v, m)[src]+ return m+++def output_batch(g: Graph, name: str, m: np.ndarray) -> np.ndarray:+ return ev_batch(g.outputs[name], m)diff --git a/chipformer/dsl/evaluate.py b/chipformer/dsl/evaluate.pynew file mode 100644index 0000000..a06905c--- /dev/null+++ b/chipformer/dsl/evaluate.py@@ -0,0 +1,148 @@+"""Exact autoregressive evaluation of a Graph.++Runs position by position, like a transformer with a KV cache: each lookup+head keeps the key (a line ``kx * q + ky``) and value vector of every past+position. Arithmetic uses Python ints/Fractions, so results are exact.++Two cache implementations:+ - "lichao": a Li Chao tree per head, O(log range) insert and query+ - "brute": a scan over all positions that also asserts the argmax is unique+ (softmax would average ties, so a tie means a broken circuit)+"""++from fractions import Fraction++from chipformer.dsl.graph import FFN, ONE, Expr, Graph, Lookup, Num++EXACT_LIMIT = 1 << 53 # every quantity must stay exactly representable in fp64+++class TieError(AssertionError):+ pass+++class LiChao:+ """Max over lines at integer points of [lo, hi]; nodes are created lazily."""++ def __init__(self, lo: int, hi: int):+ self.lo, self.hi = lo, hi+ self.nodes: dict[int, tuple[Num, Num, int]] = {}++ def insert(self, m: Num, b: Num, ident: int) -> None:+ new = (m, b, ident)+ idx, lo, hi = 1, self.lo, self.hi+ while True:+ cur = self.nodes.get(idx)+ if cur is None:+ self.nodes[idx] = new+ return+ mid = (lo + hi) // 2+ left = new[0] * lo + new[1] > cur[0] * lo + cur[1]+ center = new[0] * mid + new[1] > cur[0] * mid + cur[1]+ if center:+ self.nodes[idx], new = new, cur+ if lo == hi:+ return+ if left != center:+ idx, hi = 2 * idx, mid+ else:+ idx, lo = 2 * idx + 1, mid + 1++ def query(self, x: int) -> int:+ if not self.lo <= x <= self.hi:+ raise ValueError(f"query {x} outside [{self.lo}, {self.hi}]")+ idx, lo, hi = 1, self.lo, self.hi+ best_v, best_id = None, -1+ while idx in self.nodes:+ m, b, ident = self.nodes[idx]+ v = m * x + b+ if best_v is None or v > best_v:+ best_v, best_id = v, ident+ if lo == hi:+ break+ mid = (lo + hi) // 2+ if x <= mid:+ idx, hi = 2 * idx, mid+ else:+ idx, lo = 2 * idx + 1, mid + 1+ return best_id+++class BruteCache:+ def __init__(self, lo: int, hi: int):+ self.lo, self.hi = lo, hi+ self.lines: list[tuple[Num, Num]] = []++ def insert(self, m: Num, b: Num, ident: int) -> None:+ assert ident == len(self.lines)+ self.lines.append((m, b))++ def query(self, x: int) -> int:+ if not self.lo <= x <= self.hi:+ raise ValueError(f"query {x} outside [{self.lo}, {self.hi}]")+ scores = [m * x + b for m, b in self.lines]+ best = max(scores)+ winners = [i for i, s in enumerate(scores) if s == best]+ if len(winners) > 1:+ raise TieError(f"tie between positions {winners[:4]} at q={x}")+ return winners[0]+++def ev(e: Expr, vals: list[Num]) -> Num:+ return sum(c * vals[d] for d, c in e.terms.items())+++def relu(x: Num) -> Num:+ return max(0, x)+++class Evaluator:+ def __init__(self, graph: Graph, mode: str = "lichao"):+ self.g = graph+ cache_cls = {"lichao": LiChao, "brute": BruteCache}[mode]+ self.heads = [n for n in graph.nodes if isinstance(n, Lookup)]+ self.caches = {id(h): cache_cls(*h.q_range) for h in self.heads}+ self.head_values: dict[int, list[list[Num]]] = {id(h): [] for h in self.heads}+ self.pos = 0+ self.rows: list[list[Num]] | None = None # optionally keep every residual row++ def keep_rows(self) -> "Evaluator":+ self.rows = []+ return self++ def feed(self, token: int) -> list[Num]:+ """Process one position; returns its final residual row."""+ g, t = self.g, self.pos+ vals: list[Num] = [0] * len(g.dims)+ vals[ONE] = 1+ for d, fn in g.token_fns.items():+ vals[d] = fn(token)+ for d, fn in g.pos_fns.items():+ vals[d] = fn(t)+ for node in g.nodes:+ if isinstance(node, FFN):+ vals[node.out] = sum(c * ev(a, vals) * relu(ev(b, vals)) for c, a, b in node.terms)+ else:+ cache = self.caches[id(node)]+ store = self.head_values[id(node)]+ kx, ky, q = ev(node.kx, vals), ev(node.ky, vals), ev(node.q, vals)+ if isinstance(q, Fraction):+ if q.denominator != 1:+ raise ValueError(f"{node.name}: non-integer query {q}")+ q = int(q)+ lo, hi = node.q_range+ worst = abs(kx) * max(abs(lo), abs(hi)) + abs(ky)+ if worst >= EXACT_LIMIT:+ raise OverflowError(f"{node.name}: score magnitude {worst:.3g} >= 2^53")+ cache.insert(kx, ky, t)+ store.append([ev(v, vals) for v in node.values])+ src = store[cache.query(q)]+ for d, v in zip(node.out, src):+ vals[d] = v+ self.pos += 1+ if self.rows is not None:+ self.rows.append(vals)+ return vals++ def output(self, name: str, row: list[Num]) -> Num:+ return ev(self.g.outputs[name], row)diff --git a/chipformer/dsl/graph.py b/chipformer/dsl/graph.pynew file mode 100644index 0000000..cf2e341--- /dev/null+++ b/chipformer/dsl/graph.py@@ -0,0 +1,173 @@+"""Computation graph whose nodes map 1:1 onto parts of a transformer.++Every value is a scalar *dimension* of the residual stream, at every position.+Dimensions come from:++ - the embedding: the constant ``one``, any function of the input token, and+ any function of the position (a learned absolute position table)+ - ``ffn``: a sum of ReGLU units ``coef * a * ReLU(b)``, with a and b linear in+ existing dims (one dim per node, many hidden units per node)+ - ``lookup``: a hardmax attention head with 2D query/key. At position t it+ selects the unique s <= t maximizing ``q_t * kx_s + ky_s`` and copies the+ value expressions evaluated at s. The query is ``(q, 1)`` and the key is+ ``(kx, ky)``, so every head is a "max over lines at q" query, which+ allows an exact O(log n) cache.++Linear combinations are free: they fold into the weight matrices of whatever+reads them, so ``Expr`` is a linear form over dims rather than a node.++Each node is placed at the earliest legal sublayer (ASAP): attention sublayers+are odd (2L-1), FFN sublayers are even (2L), and a node may only read dims+produced at strictly earlier sublayers. Level 0 is the embedding.+"""++from collections.abc import Callable+from dataclasses import dataclass+from fractions import Fraction++Num = int | Fraction+ONE = 0 # dim id of the constant-one embedding feature+++class Expr:+ """A linear form ``sum(coef * dim)``; the constant term lives on dim ONE."""++ __slots__ = ("terms",)++ def __init__(self, terms: dict[int, Num] | None = None):+ self.terms = {k: v for k, v in (terms or {}).items() if v != 0}++ def __add__(self, other: "Expr | Num") -> "Expr":+ other = as_expr(other)+ t = dict(self.terms)+ for k, v in other.terms.items():+ t[k] = t.get(k, 0) + v+ return Expr(t)++ __radd__ = __add__++ def __neg__(self) -> "Expr":+ return Expr({k: -v for k, v in self.terms.items()})++ def __sub__(self, other: "Expr | Num") -> "Expr":+ return self + (-as_expr(other))++ def __rsub__(self, other: "Expr | Num") -> "Expr":+ return as_expr(other) - self++ def __mul__(self, c: Num) -> "Expr":+ if not isinstance(c, (int, Fraction)):+ raise TypeError("Expr can only be scaled by a constant (use ffn/gate for products)")+ return Expr({k: v * c for k, v in self.terms.items()})++ __rmul__ = __mul__++ def __truediv__(self, c: int) -> "Expr":+ return self * Fraction(1, c)++ def __repr__(self) -> str:+ return " + ".join(f"{v}*d{k}" for k, v in self.terms.items()) or "0"+++def as_expr(x: "Expr | Num") -> Expr:+ if isinstance(x, Expr):+ return x+ if isinstance(x, (int, Fraction)):+ return Expr({ONE: x})+ raise TypeError(f"cannot convert {type(x).__name__} to Expr")+++@dataclass+class Dim:+ name: str+ kind: str # "one" | "token" | "pos" | "ffn" | "attn"+ level: int+++@dataclass+class FFN:+ name: str+ terms: list[tuple[Num, Expr, Expr]] # coef * a * ReLU(b)+ out: int+ level: int+++@dataclass+class Lookup:+ name: str+ q: Expr+ kx: Expr+ ky: Expr+ values: list[Expr]+ out: list[int]+ q_range: tuple[int, int] # inclusive bounds on q, needed by the exact cache+ level: int+++class Graph:+ def __init__(self) -> None:+ self.dims: list[Dim] = [Dim("one", "one", 0)]+ self.token_fns: dict[int, Callable[[int], Num]] = {}+ self.pos_fns: dict[int, Callable[[int], Num]] = {}+ self.nodes: list[FFN | Lookup] = []+ self.outputs: dict[str, Expr] = {}+ self.one = Expr({ONE: 1})++ # --- embedding features --------------------------------------------------++ def token_feature(self, name: str, fn: Callable[[int], Num]) -> Expr:+ d = self._new_dim(name, "token", 0)+ self.token_fns[d] = fn+ return Expr({d: 1})++ def pos_feature(self, name: str, fn: Callable[[int], Num]) -> Expr:+ d = self._new_dim(name, "pos", 0)+ self.pos_fns[d] = fn+ return Expr({d: 1})++ # --- nodes ---------------------------------------------------------------++ def ffn(self, terms: list[tuple[Num, "Expr | Num", "Expr | Num"]], name: str = "ffn") -> Expr:+ terms = [(c, as_expr(a), as_expr(b)) for c, a, b in terms if c != 0]+ lvl = self.level(*(e for _, a, b in terms for e in (a, b)))+ sub = lvl + 1 if lvl % 2 == 1 else lvl + 2+ d = self._new_dim(name, "ffn", sub)+ self.nodes.append(FFN(name, terms, d, sub))+ return Expr({d: 1})++ def lookup(self, q: "Expr | Num", kx: "Expr | Num", ky: "Expr | Num",+ values: list["Expr | Num"], q_range: tuple[int, int],+ name: str = "lookup") -> list[Expr]:+ q, kx, ky = as_expr(q), as_expr(kx), as_expr(ky)+ values = [as_expr(v) for v in values]+ lvl = self.level(q, kx, ky, *values)+ sub = lvl + 1 if lvl % 2 == 0 else lvl + 2+ outs = [self._new_dim(f"{name}[{i}]", "attn", sub) for i in range(len(values))]+ self.nodes.append(Lookup(name, q, kx, ky, values, outs, q_range, sub))+ return [Expr({d: 1}) for d in outs]++ def output(self, name: str, e: "Expr | Num") -> None:+ self.outputs[name] = as_expr(e)++ # --- introspection --------------------------------------------------------++ def level(self, *exprs: Expr) -> int:+ return max((self.dims[d].level for e in exprs for d in e.terms), default=0)++ def n_layers(self) -> int:+ top = max((n.level for n in self.nodes), default=0)+ return (top + 1) // 2++ def stats(self) -> dict[str, int]:+ ffns = [n for n in self.nodes if isinstance(n, FFN)]+ heads = [n for n in self.nodes if isinstance(n, Lookup)]+ return {+ "layers": self.n_layers(),+ "d_model": len(self.dims),+ "heads": len(heads),+ "ffn_units": sum(len(n.terms) for n in ffns),+ }++ def _new_dim(self, name: str, kind: str, level: int) -> int:+ self.dims.append(Dim(name, kind, level))+ return len(self.dims) - 1diff --git a/chipformer/dsl/lib.py b/chipformer/dsl/lib.pynew file mode 100644index 0000000..1d6352b--- /dev/null+++ b/chipformer/dsl/lib.py@@ -0,0 +1,145 @@+"""Exact integer primitives built from ReGLU units and 2D hardmax lookups.++All inputs are assumed to be integers within the stated ranges; outputs are+then exact. Key identities (x integer):++ step(x) = [x >= 0] = ReLU(x + 1) - ReLU(x)+ hat(x) = [x == 0] = ReLU(x + 1) - 2 ReLU(x) + ReLU(x - 1)+ a * c = a * ReLU(c) for c in {0, 1}+ a * step(x), a * hat(x) same identities with the gate a+"""++from chipformer.dsl.graph import Expr, Graph, Num++MAX_POS = 1 << 22 # positions are kept below this by context rollover+++class Circuit(Graph):+ def __init__(self) -> None:+ super().__init__()+ self.pos = self.pos_feature("pos", lambda t: t)+ self.pos2 = self.pos_feature("pos^2", lambda t: t * t)++ # --- comparisons -----------------------------------------------------------++ def step(self, x: Expr, name: str = "step") -> Expr:+ """[x >= 0]"""+ return self.stepglu(self.one, x, name)++ def stepglu(self, a: "Expr | Num", x: Expr, name: str = "stepglu") -> Expr:+ """a * [x >= 0]"""+ return self.ffn([(1, a, x + 1), (-1, a, x)], name)++ def eq(self, x: Expr, k: "Expr | Num" = 0, name: str = "eq") -> Expr:+ """[x == k]"""+ return self.eqglu(self.one, x, k, name)++ def eqglu(self, a: "Expr | Num", x: Expr, k: "Expr | Num" = 0, name: str = "eqglu") -> Expr:+ """a * [x == k]"""+ d = x - k+ return self.ffn([(1, a, d + 1), (-2, a, d), (1, a, d - 1)], name)++ def ge(self, x: Expr, y: "Expr | Num", name: str = "ge") -> Expr:+ return self.step(x - y, name)++ def gate(self, a: "Expr | Num", c: Expr, name: str = "gate") -> Expr:+ """a * c for c in {0, 1}"""+ return self.ffn([(1, a, c)], name)++ def mux(self, c: Expr, a: "Expr | Num", b: "Expr | Num", name: str = "mux") -> Expr:+ """a if c else b, for c in {0, 1}"""+ return b + self.gate(a - b, c, name)++ def select(self, cases: list[tuple[Expr, "Expr | Num"]], name: str = "select") -> Expr:+ """sum(value * cond) for one-hot conds; one FFN node."""+ return self.ffn([(1, v, c) for c, v in cases], name)++ def all_of(self, *conds: "Expr | Num", name: str = "and") -> Expr:+ """Logical AND of binary conds in one FFN node."""+ return self.step(sum(conds, Expr()) - len(conds), name)++ def index(self, values: list["Expr | Num"], idx: Expr, name: str = "index") -> Expr:+ """values[idx] for integer 0 <= idx < len(values); one FFN node."""+ terms = []+ for i, v in enumerate(values):+ d = idx - i+ terms += [(1, v, d + 1), (-2, v, d), (1, v, d - 1)]+ return self.ffn(terms, name)++ def relu(self, x: Expr, name: str = "relu") -> Expr:+ return self.ffn([(1, self.one, x)], name)++ # --- integer arithmetic ----------------------------------------------------++ def floor_div(self, x: Expr, d: int, q_max: int, name: str = "div") -> Expr:+ """floor(x / d) for 0 <= x < d * (q_max + 1)"""+ terms = []+ for k in range(1, q_max + 1):+ terms += [(1, self.one, x - d * k + 1), (-1, self.one, x - d * k)]+ return self.ffn(terms, name)++ def divmod(self, x: Expr, d: int, q_max: int, name: str = "divmod") -> tuple[Expr, Expr]:+ q = self.floor_div(x, d, q_max, name)+ return q, x - d * q++ def wrap(self, x: Expr, m: int, name: str = "wrap") -> Expr:+ """x mod m for 0 <= x < 2m"""+ return x - m * self.step(x - m, name)++ def bits(self, x: Expr, n: int, name: str = "bits") -> list[Expr]:+ """Bits of 0 <= x < 2**n, LSB first, all computed in one FFN sublayer."""+ out = []+ for i in range(n):+ terms = []+ for m in range(1 << (n - 1 - i)):+ on, off = (2 * m + 1) << i, (2 * m + 2) << i+ terms += [(1, self.one, x - on + 1), (-1, self.one, x - on)]+ if off < (1 << n):+ terms += [(-1, self.one, x - off + 1), (1, self.one, x - off)]+ out.append(self.ffn(terms, f"{name}{i}"))+ return out++ @staticmethod+ def from_bits(bits: list[Expr]) -> Expr:+ return sum((b * (1 << i) for i, b in enumerate(bits)), Expr())++ def band_bits(self, xb: list[Expr], yb: list[Expr], name: str = "and") -> Expr:+ return self.ffn([(1 << i, x, y) for i, (x, y) in enumerate(zip(xb, yb))], name)++ def band(self, x: Expr, y: Expr, n: int = 8, name: str = "and") -> Expr:+ return self.band_bits(self.bits(x, n, f"{name}.x"), self.bits(y, n, f"{name}.y"), name)++ def bxor(self, x: Expr, y: Expr, n: int = 8, name: str = "xor") -> Expr:+ return x + y - 2 * self.band(x, y, n, name)++ def bor(self, x: Expr, y: Expr, n: int = 8, name: str = "or") -> Expr:+ return x + y - self.band(x, y, n, name)++ # --- lookups -----------------------------------------------------------------++ def prev(self, values: list[Expr], name: str = "prev") -> list[Expr]:+ """Values at position t-1 (at t=0: position 0).++ Score 2s(t-1) - s^2 = (t-1)^2 - (s-(t-1))^2 peaks exactly at s = t-1.+ """+ return self.lookup(self.pos - 1, 2 * self.pos, -self.pos2, values,+ (-1, MAX_POS), name)++ def latest(self, flag: Expr, values: list[Expr], name: str = "latest") -> list[Expr]:+ """Values at the latest position s <= t with flag(s) = 1 (flag in {0, 1})."""+ big = 2 * MAX_POS+ return self.lookup(0, 0, self.pos + big * (flag - 1), values, (0, 0), name)++ def latest_keyed(self, q: Expr, key: Expr, key_sq: Expr, is_entry: Expr,+ values: list[Expr], n_keys: int, name: str = "keyed") -> list[Expr]:+ """Values at the latest entry whose integer key equals q.++ Score T(2*key*q - key^2) + s = T(q^2 - (key-q)^2) + s: the key match+ dominates, recency breaks ties. Non-entries are pushed below every+ entry. Returns ``values + [found_key]``; if no entry has key q, the+ nearest key wins, so callers compare found_key with q.+ """+ t = MAX_POS+ big = 4 * t * n_keys * n_keys+ return self.lookup(q, 2 * t * key, -t * key_sq + self.pos + big * (is_entry - 1),+ values + [key], (0, n_keys - 1), name)diff --git a/chipformer/fuzz.py b/chipformer/fuzz.pynew file mode 100644index 0000000..6d80673--- /dev/null+++ b/chipformer/fuzz.py@@ -0,0 +1,59 @@+"""Random CHIP-8 programs that keep executing and cover every opcode.++Layout: a straight-line body of random instructions (skips allowed, calls to+one subroutine), a jump back to the start, then the subroutine. Stores+(FX33/FX55) may land anywhere in RAM, including the program itself, which+exercises self-modifying code.+"""++import random++SUB_LEN = 4+++def random_instruction(rng: random.Random, sub_addr: int) -> int:+ x, y, n, nn = rng.randrange(16), rng.randrange(16), rng.randrange(16), rng.randrange(256)+ kind = rng.choice([+ "00E0", "2NNN", "3XNN", "4XNN", "5XY0", "6XNN", "7XNN", "8XY", "9XY0", "ANNN",+ "BNNN", "CXNN", "DXYN", "DXYN", "EX9E", "EXA1", "FX07", "FX0A", "FX15", "FX18",+ "FX1E", "FX29", "FX33", "FX55", "FX65", "6XNN", "7XNN", "8XY",+ ])+ if kind == "00E0":+ return 0x00E0+ if kind == "2NNN":+ return 0x2000 | sub_addr+ if kind == "BNNN": # jump to the start of the body plus V0 (V0 small keeps it in range)+ return 0xB200+ if kind == "8XY":+ return 0x8000 | x << 8 | y << 4 | rng.choice([0, 1, 2, 3, 4, 5, 6, 7, 0xE])+ if kind == "ANNN":+ return 0xA000 | rng.choice([rng.randrange(0x200, 0x1000), rng.randrange(0x300, 0x400)])+ code = {+ "3XNN": 0x3000 | x << 8 | nn, "4XNN": 0x4000 | x << 8 | nn,+ "5XY0": 0x5000 | x << 8 | y << 4, "9XY0": 0x9000 | x << 8 | y << 4,+ "6XNN": 0x6000 | x << 8 | nn, "7XNN": 0x7000 | x << 8 | nn,+ "CXNN": 0xC000 | x << 8 | nn, "DXYN": 0xD000 | x << 8 | y << 4 | n,+ "EX9E": 0xE09E | x << 8, "EXA1": 0xE0A1 | x << 8,+ }.get(kind)+ if code is not None:+ return code+ return 0xF000 | x << 8 | int(kind.removeprefix("FX"), 16)+++def random_quirks(seed: int) -> tuple[str, ...]:+ """A random quirk profile (each flag on with probability 1/2)."""+ from chipformer.spec import QUIRK_NAMES++ rng = random.Random(seed * 7919 + 1)+ return tuple(q for q in QUIRK_NAMES if rng.random() < 0.5)+++def random_program(seed: int, body: int = 120) -> bytes:+ rng = random.Random(seed)+ sub_addr = 0x200 + 2 * (body + 1)+ words = [random_instruction(rng, sub_addr) for _ in range(body)]+ words.append(0x1200)+ words += [random_instruction(rng, sub_addr) for _ in range(SUB_LEN - 1)]+ words = [w if w != 0x2000 | sub_addr or i < body else 0x6000 for i, w in enumerate(words)]+ words.append(0x00EE)+ return b"".join(w.to_bytes(2, "big") for w in words)diff --git a/chipformer/games.py b/chipformer/games.pynew file mode 100644index 0000000..bcefa2e--- /dev/null+++ b/chipformer/games.py@@ -0,0 +1,387 @@+"""Game library: fetch ROMs and metadata, build the manifest, validate on the engine.++ uv run python -m chipformer.games fetch # download sources into games/sources/+ uv run python -m chipformer.games build # games/manifest.json + games/roms/+ uv run python -m chipformer.games validate # run every game on the engine, verify traces++Sources:+ - CHIP-8 Archive (github.com/JohnEarnest/chip8Archive): CC0 games with Octo quirk flags+ - classic games (github.com/kripod/chip8-roms, the Revival Studios pack: "can be freely+ distributed in its original form")+ - CHIP-8 database (github.com/chip-8/chip-8-database): title, authors, platform, tickrate+ and keys for known ROMs, keyed by SHA-1+"""++import argparse+import hashlib+import json+import urllib.parse+import urllib.request+from pathlib import Path++ROOT = Path(__file__).resolve().parent.parent+GAMES = ROOT / "games"+SOURCES = GAMES / "sources"++ARCHIVE = "https://raw.githubusercontent.com/JohnEarnest/chip8Archive/master"+DATABASE = "https://raw.githubusercontent.com/chip-8/chip-8-database/master/database"+CLASSICS_API = "https://api.github.com/repos/kripod/chip8-roms/contents/games"+CLASSICS_RAW = "https://raw.githubusercontent.com/kripod/chip8-roms/master/games"++LICENSES = {+ "archive": "CC0",+ "classics": "freely distributable (Revival Studios CHIP-8 pack)",+}+++def _get(url: str) -> bytes:+ req = urllib.request.Request(url, headers={"User-Agent": "chipformer"})+ with urllib.request.urlopen(req, timeout=60) as r:+ return r.read()+++def fetch() -> None:+ """Download every source into games/sources/ (data only; nothing is executed)."""+ (SOURCES / "database").mkdir(parents=True, exist_ok=True)+ for name in ["programs.json", "sha1-hashes.json", "platforms.json", "quirks.json"]:+ (SOURCES / "database" / name).write_bytes(_get(f"{DATABASE}/{name}"))++ arch = SOURCES / "archive"+ (arch / "roms").mkdir(parents=True, exist_ok=True)+ for name in ["programs.json", "authors.json"]:+ (arch / name).write_bytes(_get(f"{ARCHIVE}/{name}"))+ programs = json.loads((arch / "programs.json").read_text(encoding="utf-8"))+ for key, p in programs.items():+ if p.get("platform", "chip8") == "chip8":+ (arch / "roms" / f"{key}.ch8").write_bytes(_get(f"{ARCHIVE}/roms/{key}.ch8"))++ cls = SOURCES / "classics"+ cls.mkdir(parents=True, exist_ok=True)+ listing = json.loads(_get(CLASSICS_API))+ for entry in listing:+ if entry["name"].lower().endswith(".ch8"):+ url = f"{CLASSICS_RAW}/{urllib.parse.quote(entry['name'])}"+ (cls / entry["name"]).write_bytes(_get(url))+ print(f"archive: {len(list((arch / 'roms').glob('*.ch8')))} ROMs, "+ f"classics: {len(list(cls.glob('*.ch8')))} ROMs, database: "+ f"{len(json.loads((SOURCES / 'database' / 'programs.json').read_text(encoding='utf-8')))} programs")+++def sha1(data: bytes) -> str:+ return hashlib.sha1(data).hexdigest()+++# CHIP-8 database platforms we can run (lores CHIP-8 instruction set), best first+RUNNABLE_PLATFORMS = ["modernChip8", "originalChip8", "hybridVIP", "chip48", "superchip1",+ "superchip"]+# Octo's keyboard convention: WASD = keypad 5/7/8/9, E = keypad 6+OCTO_KEYS = {"up": 5, "left": 7, "down": 8, "right": 9, "a": 6}++# Games shown in the demo, in display order, with key roles where the database has none+# (from each game's own documentation) and a one-line control hint. Role names follow the+# CHIP-8 database: up/down/left/right/a/b, player2Up/player2Down.+FEATURED = {+ "tetris": ({}, "Left/Right move, Down drops faster, Space rotates"),+ "space-invaders": ({}, "Left/Right move, Space fires (press Space to start)"),+ "brix": ({"left": 4, "right": 6}, "Left/Right move the paddle"),+ "blinky": ({"up": 6, "down": 3, "left": 7, "right": 8}, "Arrow keys steer through the maze"),+ "pong": ({"up": 1, "down": 4, "player2Up": 12, "player2Down": 13},+ "Left paddle A/D (or W/S), right paddle Left/Right (or Up/Down)"),+ "super-pong": ({}, "WASD moves the paddles, Space (E) resets"),+ "ufo": ({"left": 4, "a": 5, "right": 6}, "Left/Space/Right fire left, up, right"),+ "missile-command": ({}, "Space fires"),+ "br8kout": ({}, "Left/Right move the paddle"),+ "cave-explorer": ({}, "Arrow keys move"),+ "outlaw": ({}, "Arrow keys move, Space shoots"),+ "flight-runner": ({}, "Up/Down steer"),+ "down8": ({}, "Left/Right move"),+ "dino-run": ({}, "Up or Space jumps"),+}++# Per-game play tuning. "ipf" overrides the platform's default speed. "tap" lists key roles+# delivered as one short pulse per physical press: games that act on every pass of their loop+# while a key is held (Tetris rotates once per pass) otherwise repeat a press several times.+# Measured on Tetris: at 30 instructions/frame a 150 ms press rotates twice; at 15 it rotates+# once even for a 200 ms hold.+#+# "game_over" lets the cabinet end games that never end themselves: Fran Dachille's Tetris+# (1991, HP48) has no game-over detection (its manual says "ENTER - restart"), so once the+# well is full it spawns pieces into the stack forever. The host declares the game over+# when any pixel in the given screen rows/columns stays lit for `seconds`. For Tetris, pieces+# spawn in rows 2-5; rows 6-7 are only crossed by falling pieces (at most 2.4 s in simulated+# play) until the stack reaches them. This is the host reading the screen, not the model.+TUNING = {+ "tetris": {"ipf": 15, "tap": ["a"],+ "game_over": {"rows": [6, 7], "cols": [27, 35], "seconds": 5,+ "message": "The well is full"}},+}+TAP_FRAMES = 3 # length of a tap pulse, in 60 Hz frames++# Ids for classics whose titles collide: the well-known original gets the plain name.+CLASSIC_IDS = {+ "Pong [Paul Vervalin, 1990].ch8": "pong",+ "Pong (1 player).ch8": "pong-1-player",+}+++def _slug(title: str) -> str:+ return "".join(ch if ch.isalnum() else "-" for ch in title.lower()).strip("-").replace("--", "-")+++def _short(text: str | None, limit: int = 280) -> str:+ text = " ".join((text or "").split())+ return text if len(text) <= limit else text[: limit - 1].rsplit(" ", 1)[0] + "…"+++def _db_quirks(platform: dict) -> tuple[list[str], list[str]]:+ """Map a CHIP-8 database platform's quirks onto our flags; returns (flags, notes)."""+ q = platform["quirks"]+ flags, notes = [], []+ if q.get("shift"):+ flags.append("shift")+ if q.get("memoryLeaveIUnchanged") or q.get("memoryIncrementByX"):+ flags.append("loadstore")+ if q.get("memoryIncrementByX"):+ notes.append("platform increments I by X after FX55/FX65; run with I unchanged")+ if not q.get("wrap"):+ flags.append("clip")+ if q.get("jump"):+ flags.append("jump")+ if q.get("logic"):+ flags.append("logic")+ if q.get("vblank"):+ notes.append("platform waits for vblank on draw; not modelled")+ return flags, notes+++def build() -> list[dict]:+ """Assemble games/manifest.json and games/roms/ from the downloaded sources."""+ db = json.loads((SOURCES / "database" / "programs.json").read_text(encoding="utf-8"))+ hashes = json.loads((SOURCES / "database" / "sha1-hashes.json").read_text(encoding="utf-8"))+ platforms = {p["id"]: p for p in json.loads(+ (SOURCES / "database" / "platforms.json").read_text(encoding="utf-8"))}+ archive = json.loads((SOURCES / "archive" / "programs.json").read_text(encoding="utf-8"))+ authors = json.loads((SOURCES / "archive" / "authors.json").read_text(encoding="utf-8"))+ old = {g["id"]: g for g in load_manifest()} if (GAMES / "manifest.json").exists() else {}++ games: list[dict] = []+ seen: set[str] = set()++ def add(entry: dict, rom: bytes, gid: str | None = None) -> None:+ base = gid = gid or _slug(entry["title"])+ n = 2+ while gid in seen:+ gid, n = f"{base}-{n}", n + 1+ seen.add(gid)+ entry = {"id": gid, **entry, "file": f"roms/{gid}.ch8", "sha1": sha1(rom)}+ for keep in ("status", "result"): # validation results survive rebuilds+ if keep in old.get(gid, {}):+ entry[keep] = old[gid][keep]+ if gid in FEATURED:+ keys, hint = FEATURED[gid]+ entry["keys"] = {**entry["keys"], **keys}+ entry["controls"] = hint+ entry["featured"] = list(FEATURED).index(gid)+ tuning = TUNING.get(gid, {})+ entry["ipf"] = tuning.get("ipf", entry["ipf"])+ entry["tap"] = tuning.get("tap", [])+ if "game_over" in tuning:+ entry["game_over"] = tuning["game_over"]+ (GAMES / "roms").mkdir(parents=True, exist_ok=True)+ (GAMES / "roms" / f"{gid}.ch8").write_bytes(rom)+ games.append(entry)++ def db_entry(rom: bytes):+ i = hashes.get(sha1(rom))+ if i is None:+ return None, None+ prog = db[int(i)]+ return prog, prog["roms"][sha1(rom)]++ for key, p in archive.items():+ if p.get("platform", "chip8") != "chip8":+ continue+ rom = (SOURCES / "archive" / "roms" / f"{key}.ch8").read_bytes()+ opts = p.get("options", {})+ flags = [name for name, opt in [("shift", "shiftQuirks"), ("loadstore", "loadStoreQuirks"),+ ("clip", "clipQuirks"), ("logic", "logicQuirks"),+ ("jump", "jumpQuirks"), ("vforder", "vfOrderQuirks")]+ if opts.get(opt) in (True, 1, "true")]+ _, dbrom = db_entry(rom)+ add({+ "title": p["title"],+ "authors": [authors.get(a, {}).get("name", a) if isinstance(authors.get(a), dict) else a+ for a in p.get("authors", [])],+ "year": (p.get("release") or "")[:4] or None,+ "description": _short(p.get("desc")),+ "source": f"{ARCHIVE.replace('raw.githubusercontent.com', 'github.com')}/roms/{key}.ch8",+ "license": LICENSES["archive"],+ "platform": "octo",+ "quirks": flags,+ "ipf": int(opts.get("tickrate", 20)),+ "keys": (dbrom or {}).get("keys") or OCTO_KEYS,+ "notes": [],+ }, rom)++ for path in sorted((SOURCES / "classics").glob("*.ch8")):+ if "(alt)" in path.name:+ continue+ rom = path.read_bytes()+ prog, dbrom = db_entry(rom)+ name = path.stem.split(" [")[0].split(" (")[0]+ plats = [pl for pl in (dbrom or {}).get("platforms", []) if pl in RUNNABLE_PLATFORMS]+ plat = plats[0] if plats else "originalChip8"+ flags, notes = _db_quirks(platforms[plat])+ if not plats:+ notes.append("not in the CHIP-8 database as a lores CHIP-8 game; COSMAC quirks assumed")+ add({+ "title": prog["title"] if prog else name,+ "authors": (prog or {}).get("authors", []),+ "year": (prog or {}).get("release"),+ "description": _short((dbrom or {}).get("description") or (prog or {}).get("description")),+ "source": f"https://github.com/kripod/chip8-roms/blob/master/games/{urllib.parse.quote(path.name)}",+ "license": LICENSES["classics"],+ "platform": plat,+ "quirks": flags,+ "ipf": int((dbrom or {}).get("tickrate") or platforms[plat].get("defaultTickrate", 15)),+ "keys": (dbrom or {}).get("keys") or {},+ "notes": notes,+ }, rom, CLASSIC_IDS.get(path.name))++ missing = set(FEATURED) - {g["id"] for g in games}+ if missing:+ raise ValueError(f"featured games not found: {sorted(missing)}")+ (GAMES / "manifest.json").write_text(json.dumps({"games": games}, indent=1), encoding="utf-8")+ write_attribution(games)+ print(f"manifest: {len(games)} games, {len(FEATURED)} featured")+ return games+++def write_attribution(games: list[dict], path: Path | None = None) -> None:+ lines = [+ "# Game credits",+ "",+ "ROMs in `roms/`. CHIP-8 Archive games are CC0. The classics come from the Revival",+ "Studios CHIP-8 pack, which \"can be freely distributed in its original form\"; each",+ "game belongs to its author.",+ "",+ "| Game | Authors | Year | License | Source |",+ "|---|---|---|---|---|",+ ]+ for g in sorted(games, key=lambda g: g["title"].lower()):+ lines.append(f"| {g['title']} | {', '.join(g['authors']) or 'unknown'} | {g.get('year') or ''}"+ f" | {g['license']} | [link]({g['source']}) |")+ (path or GAMES / "ATTRIBUTION.md").write_text("\n".join(lines) + "\n", encoding="utf-8")+++def load_manifest() -> list[dict]:+ return json.loads((GAMES / "manifest.json").read_text(encoding="utf-8"))["games"]+++# opcodes outside lores CHIP-8 (SCHIP / XO-CHIP); executing one means the game won't run right+def foreign_opcode(op: int) -> str | None:+ hi, n, nn = op >> 12, op & 0xF, op & 0xFF+ if hi == 0 and op not in (0x00E0, 0x00EE):+ return f"{op:04X} (machine code / SCHIP)"+ if hi == 0xD and n == 0:+ return "DXY0 (SCHIP 16x16 sprite)"+ if hi == 0x5 and n != 0:+ return f"5XY{n:X} (XO-CHIP)"+ if hi == 0xF and nn in (0x00, 0x01, 0x02, 0x30, 0x3A, 0x75, 0x85):+ return f"FX{nn:02X} (SCHIP/XO-CHIP)"+ return None+++def validate(ids: list[str] | None = None, instr: int = 20000) -> None:+ """Reference profile + free-running engine run + trace verification per game."""+ import os+ import random+ import subprocess++ import numpy as np++ from chipformer.check import verify_trace+ from chipformer.ref.emu import fetch as fetch_op+ from chipformer.trace import Machine++ exe = ROOT / "engine" / "target" / "release" / ("engine.exe" if os.name == "nt" else "engine")+ games = load_manifest()+ tmp = Path(os.environ.get("TEMP", "/tmp")) / "chipformer-validate"+ tmp.mkdir(exist_ok=True)+ shots = GAMES / "shots"+ shots.mkdir(exist_ok=True)+ for g in games:+ if ids and g["id"] not in ids:+ continue+ rom = (GAMES / g["file"]).read_bytes()+ quirks = ",".join(g["quirks"])+ # 1. reference: executed opcodes, tokens per instruction+ rng = random.Random(0)+ m = Machine(rom, tuple(g["quirks"]))+ foreign: dict[str, int] = {}+ for i in range(instr):+ if i % g["ipf"] == 0:+ m.tick()+ if rng.random() < 0.2:+ m.set_key(rng.randrange(16), rng.random() < 0.5)+ f = foreign_opcode(fetch_op(m.emu.cells))+ if f:+ foreign[f] = foreign.get(f, 0) + 1+ m.step()+ tok_per_instr = (len(m.tokens) - m.generated.index(True)) / instr+ need = tok_per_instr * g["ipf"] * 60+ # 2. the transformer, free-running, verified record by record+ dump = tmp / f"{g['id']}.u32"+ r = subprocess.run([str(exe), "run", str(ROOT / "model"), str(GAMES / g["file"]),+ "--instr", str(instr), "--ipf", str(g["ipf"]), "--quirks", quirks,+ "--dump", str(dump)], check=False, capture_output=True, text=True)+ if r.returncode != 0:+ verified, err = 0, "engine error: " + r.stderr.strip().splitlines()[-1]+ else:+ toks = np.fromfile(dump, dtype="<u4").tolist()+ verified, err = verify_trace(toks)+ save_screenshot(toks, shots / f"{g['id']}.png")+ status = "unsupported" if foreign else ("ok" if err is None else "mismatch")+ g["status"] = status+ g["result"] = {"verified_records": verified, "error": err, "foreign_opcodes": foreign,+ "tokens_per_instruction": round(tok_per_instr, 2),+ "realtime_tok_per_s": round(need)}+ print(f"{g['id']:32} {status:11} {verified:7,} records {need:8,.0f} tok/s needed"+ f" {', '.join(foreign) if foreign else ''}{err or ''}", flush=True)+ (GAMES / "manifest.json").write_text(json.dumps({"games": games}, indent=1), encoding="utf-8")+++def save_screenshot(tokens: list[int], path: Path, scale: int = 6) -> None:+ """The final screen of a (verified) trace, rendered from the model's own writes."""+ from PIL import Image++ from chipformer import spec+ from chipformer.ref.emu import apply_writes, display_pixels+ from chipformer.trace import parse++ cells = [0] * spec.NUM_CELLS+ for rec in parse(tokens, allow_reboot=True):+ if rec.kind == "boot":+ cells = [0] * spec.NUM_CELLS+ apply_writes(cells, rec.writes)+ img = Image.new("RGB", (spec.DISPLAY_W, spec.DISPLAY_H))+ img.putdata([(221, 245, 158) if p else (25, 34, 31) for row in display_pixels(cells) for p in row])+ img.resize((spec.DISPLAY_W * scale, spec.DISPLAY_H * scale), Image.NEAREST).save(path)+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("command", choices=["fetch", "build", "validate"])+ ap.add_argument("ids", nargs="*", help="validate only these game ids")+ ap.add_argument("--instr", type=int, default=20000)+ args = ap.parse_args()+ if args.command == "fetch":+ fetch()+ elif args.command == "build":+ build()+ else:+ validate(args.ids or None, args.instr)+++if __name__ == "__main__":+ main()diff --git a/chipformer/hf_export.py b/chipformer/hf_export.pynew file mode 100644index 0000000..377985d--- /dev/null+++ b/chipformer/hf_export.py@@ -0,0 +1,262 @@+"""Write a Hugging Face model repo for chipformer (transformers, trust_remote_code).++ uv run python -m chipformer.hf_export --out dist/hf++The weights are compiled fresh from the circuits (0.3 s), so the export never depends on a+stale model/ directory. The repo holds config.json (with auto_map), model.safetensors (fp64),+the configuration/modeling code from hf/, the sparse model.chipformer used by the browser+engine, a model card and a Tetris example.+"""++import argparse+import json+import shutil+from pathlib import Path++import numpy as np++from chipformer import spec+from chipformer.dsl.lib import MAX_POS++ROOT = Path(__file__).resolve().parent.parent+HF_SRC = ROOT / "hf"+GAMES = ROOT / "games"+ROLLOVER_AT = 250_000 # same as the engine+LINKS = {+ "github": "https://github.com/divagr18/chipformer",+ "space": "https://huggingface.co/spaces/divagr1925/chipformer",+ "demo": "https://divagr.com/chipformer",+}+EXAMPLE_GAME = "tetris"+++def hf_config(w: dict[str, np.ndarray], config: dict) -> dict:+ layers = []+ for i, lay in enumerate(config["layers"]):+ heads = [[int(w[f"layers.{i}.attn.{h}.wq"].shape[0]), int(w[f"layers.{i}.attn.{h}.wv"].shape[0])]+ for h in range(lay["heads"])]+ layers.append({"heads": heads, "ffn_units": lay["ffn_units"], "head_names": lay["head_names"]})+ return {+ "architectures": ["ChipformerForCausalLM"],+ "auto_map": {+ "AutoConfig": "configuration_chipformer.ChipformerConfig",+ "AutoModelForCausalLM": "modeling_chipformer.ChipformerForCausalLM",+ },+ "model_type": "chipformer",+ "d_model": config["d_model"],+ "vocab_size": config["vocab_size"],+ "beta": config["beta"],+ "layers": layers,+ "machine": config["machine"],+ "max_position": MAX_POS,+ "rollover_at": ROLLOVER_AT,+ "params": config["params"],+ "nonzero_params": config["nonzero_params"],+ "dtype": "float64",+ "torch_dtype": "float64",+ "tie_word_embeddings": False,+ }+++def example_game() -> dict:+ games = json.loads((GAMES / "manifest.json").read_text())+ games = games["games"] if isinstance(games, dict) else games+ return next(g for g in games if g["id"] == EXAMPLE_GAME)+++def model_card(config: dict, game: dict) -> str:+ heads = sum(lay["heads"] for lay in config["layers"])+ units = sum(lay["ffn_units"] for lay in config["layers"])+ keys = game["keys"]+ return f"""---+license: mit+library_name: transformers+pipeline_tag: text-generation+tags:+- chip-8+- emulator+- mechanistic-interpretability+- compiled-transformer+- custom_code+---++# chipformer++**Tetris, running entirely inside a transformer.**++This model *is* a CHIP-8 computer. Give it a game as the prompt and greedy decoding runs the game:+every token it predicts is one write to the machine's memory (a register, the program counter, a+row of pixels). Paint those writes onto a 64×32 screen and you are playing Tetris. Your keypresses+and the 60 Hz clock go into the context window. Everything else, every frame, every collision,+every point on the scoreboard, comes out of the model.++There is no emulator behind it. It is the same loop every language model runs: read the context,+predict the next token, append it, repeat.++- One set of weights plays Tetris, Space Invaders, Brix, Pong and 100+ other games. Switching games+ means switching the prompt.+- An ordinary decoder-only architecture: {config['n_layers']} layers, d_model {config['d_model']},+ {heads} attention heads, {units:,} ReGLU units, vocabulary {config['vocab_size']:,}, float64.++**Play it in your browser:** [{LINKS['demo'].removeprefix('https://')}]({LINKS['demo']}) ·+[Hugging Face Space]({LINKS['space']}) · Source: [GitHub]({LINKS['github']})++## How it works++- **The ROM is the prompt.** The prompt is `BOOT`, then one `ADDR a, VAL v` token pair per nonzero+ byte of machine state, then `STEP`.+- **Every output token is a write.** Each instruction becomes a short record of writes such as+ `@V3 =1F @PC =22A STEP`. The screen is drawn from these writes.+- **The host injects the outside world.** A `TICK` token arrives once per 60 Hz frame, and keypresses+ are injected as writes. Every other token is the model's greedy prediction.+- **Attention is memory.** To read register V3, a head attends to the most recent write to V3.+ Keys use a parabola over the address plus a recency term, so the hardmax lands exactly there.+- **ReLU is the ALU.** Addition, comparison, XOR and opcode decoding are all built from ReLUs,+ using identities like step(x) = ReLU(x+1) - ReLU(x).++The architecture is ordinary:+- token embedding plus a position embedding from [t, t²]+- causal softmax attention (β = {config['beta']:g}) with residual connections+- ReGLU feed-forward layers+- a linear unembedding+- no normalization++`modeling_chipformer.py` contains nothing CHIP-8 specific apart from the host loop, which only feeds+tokens in and reads the screen back.++**Where the weights come from.** They are compiled from a description of the CHIP-8 circuit+(written in a small DSL in the GitHub repo) rather than learned, which is what makes every+instruction exact. {config['nonzero_params']:,} of the {config['params']:,} parameters are nonzero.++## Usage++```python+import torch+from huggingface_hub import hf_hub_download+from transformers import AutoModelForCausalLM++repo = "divagr1925/chipformer"+model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True, dtype=torch.float64)++rom = open(hf_hub_download(repo, "roms/{game['id']}.ch8"), "rb").read()+game = model.play(rom, quirks="{','.join(game['quirks'])}", ipf={game['ipf']})+for frame in range(120):+ game.frame(keys={{{keys['left']}}} if frame % 20 < 4 else ()) # CHIP-8 key {keys['left']} = left+print(game.ascii())+```++`play_tetris.py` animates the game in a terminal: `python play_tetris.py`.++The model is a normal `PreTrainedModel`, so you can also call it directly:+- `model(input_ids, past_key_values=cache)` returns next-token logits over the {config['vocab_size']:,}-token vocabulary.+- `game.tokens` (with `keep_tokens=True`) gives you the full trace.++### Notes++- **Use float64.** The memory-read heads compare attention scores of about 10¹³, and these are only+ exact in float64. fp16 and bf16 will break the machine.+- **Speed.** Decoding is greedy, one token at a time with a KV cache. On a CPU in PyTorch that is+ about 200 tokens/s, or 2–3 frames per second of Tetris (the game wants 60).+ - `game = model.play(...)` multiplies by sparse copies of the big feed-forward matrices, since only+ 0.4% of the weights are nonzero. The outputs are identical; pass `sparse=False` to run the dense+ weights exactly as loaded.+ - The Rust/wasm engine in the GitHub repo uses the same weights with sparse matmuls and exact+ attention shortcuts. It runs the games at full speed in a browser.+- **Context.** Positions must stay below {MAX_POS:,} (2²²). `ChipformerGame` restarts the context+ from a state dump every {ROLLOVER_AT:,} tokens, as the engine does.+- **Coverage.** The weights implement the original CHIP-8 instruction set with Octo's quirk flags.+ SCHIP/XO-CHIP opcodes and the vBlank display-wait quirk are not supported.++## Files++| File | |+|---|---|+| `model.safetensors` | dense float64 weights (what `transformers` loads) |+| `model.chipformer` | the same weights, sparse (nonzeros only), for the Rust/wasm engine |+| `configuration_chipformer.py`, `modeling_chipformer.py` | the architecture plus the host loop |+| `play_tetris.py` | a terminal demo |+| `roms/{game['id']}.ch8` | {game['title']} by {', '.join(game['authors'])} ({game['year']}), {game['license']} |+"""+++EXAMPLE = '''"""Watch chipformer play Tetris in your terminal. Every pixel is a token the model predicted.++ python play_tetris.py # from the Hugging Face Hub+ python play_tetris.py path/to/repo # from a local copy+"""++import sys+import time+from pathlib import Path++import torch+from transformers import AutoModelForCausalLM++sys.stdout.reconfigure(encoding="utf-8") # the screen is drawn with block characters+REPO = sys.argv[1] if len(sys.argv) > 1 else "divagr1925/chipformer"+FRAMES = int(sys.argv[2]) if len(sys.argv) > 2 else 400+LEFT, RIGHT, DOWN, ROTATE = __KEYS__++model = AutoModelForCausalLM.from_pretrained(REPO, trust_remote_code=True, dtype=torch.float64)+if Path(REPO).is_dir():+ rom = Path(REPO, "roms/tetris.ch8").read_bytes()+else:+ from huggingface_hub import hf_hub_download++ rom = Path(hf_hub_download(REPO, "roms/tetris.ch8")).read_bytes()++game = model.play(rom, quirks="__QUIRKS__", ipf=__IPF__)+# a little scripted play: rotate (a tap), slide, drop+plan = {0: [ROTATE], 1: [], 6: [LEFT], 7: [], 8: [LEFT], 9: [], 14: [DOWN]}+t0, fed0 = time.time(), game.fed+print("\\x1b[2J", end="")+for f in range(FRAMES):+ keys = plan.get(f % 40, [] if f % 40 < 14 else [DOWN])+ game.frame(keys)+ rate = (game.fed - fed0) / (time.time() - t0)+ print(f"\\x1b[H{game.ascii()}\\nframe {f + 1}/{FRAMES} · {game.instructions} instructions · "+ f"{rate:,.0f} tokens/s · position {game.position:,}", flush=True)+'''+++def export(out: Path) -> None:+ from safetensors.numpy import save_file++ from chipformer.chip8 import build+ from chipformer.compile import compile_graph, save_sparse++ out.mkdir(parents=True, exist_ok=True)+ w, config = compile_graph(build())+ assert config["vocab_size"] == spec.VOCAB_SIZE+ save_file({k: np.ascontiguousarray(v) for k, v in w.items()}, out / "model.safetensors",+ metadata={"format": "pt"})+ save_sparse(w, config, out / "model.chipformer")+ (out / "config.json").write_text(json.dumps(hf_config(w, config), indent=1))+ for f in ("configuration_chipformer.py", "modeling_chipformer.py"):+ shutil.copy(HF_SRC / f, out / f)++ game = example_game()+ (out / "roms").mkdir(exist_ok=True)+ shutil.copy(GAMES / game["file"], out / "roms" / f"{game['id']}.ch8")+ keys = game["keys"]+ (out / "play_tetris.py").write_text(+ EXAMPLE.replace("__KEYS__", f"{keys['left']}, {keys['right']}, {keys['down']}, {keys['a']}")+ .replace("__QUIRKS__", ",".join(game["quirks"])).replace("__IPF__", str(game["ipf"])),+ encoding="utf-8")+ (out / "README.md").write_text(model_card(config, game), encoding="utf-8")+ (out / ".gitattributes").write_text(+ "*.safetensors filter=lfs diff=lfs merge=lfs -text\n"+ "*.chipformer filter=lfs diff=lfs merge=lfs -text\n")+++def main() -> None:+ ap = argparse.ArgumentParser(description="Export chipformer as a Hugging Face model repo")+ ap.add_argument("--out", type=Path, default=Path("dist/hf"))+ out = ap.parse_args().out+ export(out)+ files = [f for f in out.rglob("*") if f.is_file()]+ print(f"{out}: {len(files)} files, {sum(f.stat().st_size for f in files) / 1e6:.1f} MB")+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/__init__.py b/chipformer/q35/__init__.pynew file mode 100644index 0000000..ff50fef--- /dev/null+++ b/chipformer/q35/__init__.py@@ -0,0 +1,8 @@+"""CHIP-8 inside a stock Qwen3.5 checkpoint: bf16-exact circuits built from Gated DeltaNet and SwiGLU."""++try: # the chip is exact only without bf16 split-K reductions, which cuBLAS may pick for some shapes+ import torch++ torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False+except ImportError:+ passdiff --git a/chipformer/q35/arith_bench.py b/chipformer/q35/arith_bench.pynew file mode 100644index 0000000..b377664--- /dev/null+++ b/chipformer/q35/arith_bench.py@@ -0,0 +1,1224 @@+"""Standard arithmetic benchmarks: merged (the computer in the weights) vs base Qwen3.5-4B.++ python -m chipformer.q35.arith_bench run COND DATA [DATA ...] [--model PATH] [--batch 16]+ python -m chipformer.q35.arith_bench replay COND DATA [DATA ...] # chip exactness (IR)+ python -m chipformer.q35.arith_bench identity --model MERGED # BatchDecoder == generate+ python -m chipformer.q35.arith_bench summary++Datasets (problems sampled with a fixed seed, so every condition sees the same ones):++ dmx_add_or_sub_big, dmx_mul_big, dmx_mixed_longer, dmx_add_sub_multiple_longer+ dmi_add_or_sub, dmi_mul, dmi_add_sub_multiple, dmi_mixed+ dmxc_add_or_sub_big, dmxc_mul_big+ the same extrapolate test files, sampled from their clean problems only+ DeepMind Mathematics Dataset v1.0 test files (extrapolate / interpolate); the files+ alternate question / answer lines+ nupa_add, nupa_sub, nupa_mulh, nupa_mule+ NUPA_text test.json, integer add / sub / multiply_hard / multiply_easy, sampled evenly+ over digit lengths 3-20 (groups 3-6, 7-10, 11-15, 16-20). NUPA's "Directly return the+ answer as an integer ..." preamble and its "= answer" are dropped; the task sentence+ ("Add two numbers: 830 + 70") is kept verbatim+ bbh_mat2 BIG-Bench Hard multistep_arithmetic_two (the trailing " =" is dropped)+ eleu_<subset> EleutherAI/arithmetic (the "Q: ... A:" wrapper is dropped)++Conditions, prompts, decoding and judging are evals.py's (same SUFFIX, budgets, BIOS, bans,+BatchDecoder): this file only adds the loaders, a multi-dataset runner (one model load per+condition), an exact answer check (pred == gold as rationals: no tolerance, decimals and+fractions compared exactly), the circuit replay and the summary.++``clean`` = every number in the problem is a non-negative integer, there is no division and+the answer is a non-negative integer (what the computer's add / sub / mul can do end to end).+"""++import argparse+import json+import math+import random+import re+import sys+import time+from decimal import Decimal, InvalidOperation+from fractions import Fraction+from pathlib import Path++from chipformer.q35 import evals as E++_fmt = E.fmt+++def _safe_fmt(v) -> str:+ """evals.fmt with enough decimal precision for huge values (runaway digit strings in a+ transcript would otherwise raise InvalidOperation and end the run)."""+ import decimal++ with decimal.localcontext() as c:+ c.prec = 100_000+ try:+ return _fmt(v)+ except (decimal.InvalidOperation, ValueError, OverflowError):+ return str(v)+++E.fmt = _safe_fmt # judge() looks fmt up in its module at call time+ROOT = Path(__file__).resolve().parents[2]+HOME = Path.home()+DATA = HOME / "data"+OUT = ROOT / "dist" / "arith"+sys.set_int_max_str_digits(0)++DM_MODULES = {"dmx_add_or_sub_big": "extrapolate/arithmetic__add_or_sub_big",+ "dmx_mul_big": "extrapolate/arithmetic__mul_big",+ "dmx_mixed_longer": "extrapolate/arithmetic__mixed_longer",+ "dmx_add_sub_multiple_longer": "extrapolate/arithmetic__add_sub_multiple_longer",+ "dmx_div_big": "extrapolate/arithmetic__div_big",+ "dmx_mul_div_multiple_longer": "extrapolate/arithmetic__mul_div_multiple_longer",+ "dmi_add_or_sub": "interpolate/arithmetic__add_or_sub",+ "dmi_mul": "interpolate/arithmetic__mul",+ "dmi_add_sub_multiple": "interpolate/arithmetic__add_sub_multiple",+ "dmi_mixed": "interpolate/arithmetic__mixed"}+NUPA_TASKS = {"nupa_add": ("add_Integer_Integer_Integer", "add", "+"),+ "nupa_sub": ("sub_Integer_Integer_Integer", "sub", "-"),+ "nupa_mulh": ("multiply_hard_Integer_Integer_Integer", "mul", "*"),+ "nupa_mule": ("multiply_easy_Integer_Integer_Integer", "mul", "*")}+NUPA_GROUPS = [(3, 6), (7, 10), (11, 15), (16, 20)]+ELEU = ["two_digit_addition", "two_digit_subtraction", "two_digit_multiplication",+ "three_digit_addition", "three_digit_subtraction", "four_digit_addition",+ "four_digit_subtraction", "five_digit_addition", "five_digit_subtraction",+ "single_digit_three_ops"]+ELEU_OPS = {"addition": ("add", "plus"), "subtraction": ("sub", "minus"),+ "multiplication": ("mul", "times")}+DMXC = ["dmxc_add_or_sub_big", "dmxc_mul_big"]+DM_DIV = ["dmx_div_big", "dmx_mul_div_multiple_longer", "dmxc_div_big"] # v14: division+ALL = [*DM_MODULES, *DMXC, "dmxc_div_big", *NUPA_TASKS, "bbh_mat2", *(f"eleu_{s}" for s in ELEU)]+DEFAULT_N = {"dmx": 100, "dmxc": 100, "dmi": 100, "nupa": 400, "bbh": 250, "eleu": 50}+CONDS = ["base", "base_nudge", "merged_nb", "merged_inline_fp", "merged_inline_nudge_fp",+ "merged_inline_fp_tc", "merged_inline_nudge_fp_tc",+ "merged_inline", "merged_inline_nudge", "pytool"]+++def family(data: str) -> str:+ return data.split("_")[0]+++# ---------------------------------------------------------------------------------- datasets++def _neg_literal(q: str) -> bool:+ """A negative number literal: a '-' right before a digit whose previous non-space character+ is not a digit or ')' (DM / BBH write binary minus as 'a - b')."""+ for m in re.finditer(r"-(?=\d)", q):+ before = q[:m.start()].rstrip()+ if not before or not (before[-1].isdigit() or before[-1] == ")"):+ return True+ return False+++def is_clean(q: str, gold: str) -> bool:+ g = Fraction(gold) if "/" in gold else Fraction(Decimal(gold))+ if g < 0 or g.denominator != 1:+ return False+ return not (re.search(r"\d\.\d", q) or "/" in q or _neg_literal(q))+++def load_dm(data: str, n: int, seed: int) -> list[dict]:+ """dmx_/dmi_: a seeded sample of the module's test file; dmxc_: of its clean problems only+ (the add / sub / mul the computer covers, without decimals or negatives)."""+ clean_only = data.startswith("dmxc_")+ mod = DM_MODULES["dmx_" + data[5:] if clean_only else data]+ f = DATA / "dm" / "mathematics_dataset-v1.0" / (mod + ".txt")+ lines = f.read_text(encoding="utf-8").splitlines()+ pairs = [(lines[i].strip(), lines[i + 1].strip()) for i in range(0, len(lines) - 1, 2)]+ pool = [i for i, (q, a) in enumerate(pairs) if is_clean(q, a)] if clean_only else range(len(pairs))+ idx = sorted(random.Random(seed).sample(pool, min(n, len(pool))))+ return [{"id": f"{data}-{i}", "question": pairs[i][0], "gold": pairs[i][1],+ "group": data.split("_", 1)[1], "clean": is_clean(pairs[i][0], pairs[i][1])}+ for i in idx]+++_NUPA = None+++def load_nupa(data: str, n: int, seed: int) -> list[dict]:+ """n problems, spread evenly over the digit lengths 3-20 (n // 18 per length, at least 1)."""+ global _NUPA+ cache = DATA / "nupa" / "int_tasks.json"+ if _NUPA is None:+ if not cache.exists(): # keep only the integer tasks (test.json is 690 MB)+ full = json.loads((DATA / "nupa" / "test.json").read_text(encoding="utf-8"))+ keep = {k: full[k] for k, _, _ in NUPA_TASKS.values()}+ cache.write_text(json.dumps(keep), encoding="utf-8")+ _NUPA = json.loads(cache.read_text(encoding="utf-8"))+ key, op, sym = NUPA_TASKS[data]+ lens = list(range(3, 21))+ per = max(1, n // len(lens))+ out = []+ for L in lens:+ items = _NUPA[key][str(L)]+ order = list(range(len(items))) # per-length shuffle: a larger n extends a smaller one+ random.Random(f"{seed}-{data}-{L}").shuffle(order)+ for j in sorted(order[:per]):+ s = items[j]+ task = s.split(" . ", 1)[1] # drop "Directly return the answer ... like 123 ."+ m = re.fullmatch(r"(.*?:\s*)(\d+) (\S) (\d+) = (\d+)", task)+ assert m and m[3] == sym, s+ g = next(f"{lo}-{hi}" for lo, hi in NUPA_GROUPS if lo <= L <= hi)+ out.append({"id": f"{data}-L{L}-{j}", "question": f"{m[1]}{m[2]} {m[3]} {m[4]}",+ "gold": m[5], "group": f"len {g}", "length": L, "clean": True,+ "calls": [[op, m[2], m[4]]]})+ return out+++def load_bbh(data: str, n: int, seed: int) -> list[dict]:+ import pyarrow.parquet as pq++ t = pq.read_table(DATA / "bbh" / "mat2.parquet").to_pylist()+ idx = sorted(random.Random(seed).sample(range(len(t)), min(n, len(t))))+ out = []+ for i in idx:+ q = re.sub(r"\s*=\s*$", "", t[i]["input"].strip())+ out.append({"id": f"{data}-{i}", "question": q, "gold": t[i]["target"].strip(),+ "group": "multistep_arithmetic_two", "clean": is_clean(q, t[i]["target"])})+ return out+++def load_eleu(data: str, n: int, seed: int) -> list[dict]:+ sub = data[5:]+ rows = [json.loads(x) for x in (DATA / "eleuther" / f"{sub}.jsonl").read_text(+ encoding="utf-8").splitlines() if x.strip()]+ idx = sorted(random.Random(seed).sample(range(len(rows)), min(n, len(rows))))+ out = []+ for i in idx:+ q = re.sub(r"^\s*Q:\s*|\s*A:\s*$", "", rows[i]["context"]).strip()+ gold = rows[i]["completion"].strip()+ p = {"id": f"{data}-{i}", "question": q, "gold": gold, "group": sub,+ "clean": is_clean(q, gold) and "three_ops" not in sub}+ for word, (op, w) in ELEU_OPS.items():+ m = re.fullmatch(rf"What is (\d+) {w} (\d+)\?", q)+ if word in sub and m:+ p["calls"] = [[op, m[1], m[2]]]+ out.append(p)+ return out+++def load(data: str, n: int | None = None, seed: int = 0) -> list[dict]:+ n = n or DEFAULT_N[family(data)]+ fam = family(data)+ if fam in ("dmx", "dmi", "dmxc"):+ return load_dm(data, n, seed)+ if fam == "nupa":+ return load_nupa(data, n, seed)+ if fam == "bbh":+ return load_bbh(data, n, seed)+ if fam == "eleu":+ return load_eleu(data, n, seed)+ raise ValueError(data)+++# ---------------------------------------------------------------------------------- judging++def gold_value(gold: str) -> Fraction:+ gold = gold.replace(",", "").strip()+ return Fraction(gold) if "/" in gold else Fraction(Decimal(gold))+++def _norm(s: str) -> str:+ s = re.sub(r"\\(?:text|mathrm|textbf|mbox|mathbf)\{([^{}]*)\}", r" \1 ", s)+ s = s.replace("\u2212", "-").replace("\\left", "").replace("\\right", "").replace("{,}", ",")+ for x in ("\\$", "$", "\\,", "\\!", "\\;", "\\ ", "~", "**"):+ s = s.replace(x, " ")+ s = re.sub(r"(?<=\d),(?=\d{3}(?!\d))", "", s) # thousands separators+ s = re.sub(r"(?<=\d)\s+(?=\d{3}(?!\d))", "", s) # 1 234 567+ s = re.sub(r"\\d?frac\s*\{([^{}]*)\}\s*\{([^{}]*)\}", r"(\1)/(\2)", s)+ return s.strip()+++NUMR = r"-?\s*\d+(?:\.\d+)?"+++def parse_value(s: str):+ """A boxed answer as an exact rational (decimals and a/b fractions), else None."""+ s = _norm(s)+ if "=" in s:+ s = s.rsplit("=", 1)[1]+ s = s.strip().rstrip(".").strip()+ m = re.fullmatch(rf"(-)?\s*\(?\s*({NUMR})\s*\)?\s*/\s*\(?\s*({NUMR})\s*\)?", s)+ if m:+ a, b = Fraction(Decimal(m[2].replace(" ", ""))), Fraction(Decimal(m[3].replace(" ", "")))+ if b == 0:+ return None+ return (-1 if m[1] else 1) * a / b+ m = re.fullmatch(rf"\(?\s*({NUMR})\s*\)?", s)+ if m:+ return Fraction(Decimal(m[1].replace(" ", "")))+ nums = re.findall(NUMR, s)+ if len(nums) == 1:+ return Fraction(Decimal(nums[0].replace(" ", "")))+ return None+++def strict_pred(text: str):+ """The answer as evals.judge reads it (user-turn cut, last \\boxed{} after the thinking and+ outside tool calls, else the last number), parsed exactly (no rounding)."""+ m = E.USER_TURN.search(text)+ if m:+ text = text[:m.start()]+ t = re.sub(r"<think>.*?</think>", " ", text, flags=re.DOTALL)+ t = re.sub(r"<tool_call>.*?(</tool_call>|$)", " ", t, flags=re.DOTALL)+ t = re.sub(r"<tool_response>.*?(</tool_response>|$)", " ", t, flags=re.DOTALL)+ t = re.sub(r"<\|[^|]*\|>(assistant|user)?", " ", E.MARK.sub(" ", t))+ boxes = [E._balanced(t, b.end()) for b in re.finditer(r"\\boxed\s*\{", t)]+ if boxes:+ try:+ return parse_value(boxes[-1]), "boxed"+ except (InvalidOperation, ValueError, ZeroDivisionError):+ return None, "boxed_unparsed"+ nums = re.findall(r"-?\d+(?:\.\d+)?", _norm(t))+ if nums:+ return Fraction(Decimal(nums[-1])), "last_number"+ return None, "none"+++def exact_fields(p: dict, text: str) -> dict:+ v, src = strict_pred(text)+ g = gold_value(p["gold"])+ return {"pred_exact": (str(v.numerator) if v.denominator == 1 else+ f"{v.numerator}/{v.denominator}") if v is not None else None,+ "src_exact": src, "exact": v is not None and v == g}+++# ---------------------------------------------------------------------------------- running++def _judgeable(p: dict) -> dict:+ """evals.judge reads the gold with Decimal(): give it a fraction gold as a 30-digit decimal+ (it only feeds the tolerant ``correct``; ``exact`` uses the true gold)."""+ if "/" not in p["gold"]:+ return p+ import decimal++ g = gold_value(p["gold"])+ with decimal.localcontext() as c:+ c.prec = 40+ dec = Decimal(g.numerator) / Decimal(g.denominator)+ return p | {"gold": str(dec)}+++def rescore_errors(args) -> None:+ """Re-judge records whose scoring failed (score_error) from their stored ids / text."""+ out = Path(args.out)+ for f in sorted(out.glob("*.jsonl")):+ parts = f.name.split(".")+ if len(parts) != 3:+ continue+ recs = [json.loads(x) for x in f.read_text(encoding="utf-8").splitlines() if x.strip()]+ bad = [r for r in recs if r.get("score_error")]+ if not bad:+ continue+ probs = {p["id"]: p for p in load(parts[0], 100_000)}+ for r in bad:+ p = probs[r["id"]]+ j = E.judge(_judgeable(p), r["text"], False)+ r.update(j)+ r.update(exact_fields(p, r["text"]))+ r["source"] = j["source"]+ r["gold"] = p["gold"]+ r.pop("score_error")+ r["rescored"] = True+ f.write_text("".join(json.dumps(r, ensure_ascii=False) + "\n" for r in recs),+ encoding="utf-8")+ print(f"{f.name}: rescored {len(bad)}", flush=True)+++def snapshot() -> str:+ snap = HOME / "snap.txt"+ if snap.exists():+ return snap.read_text().strip().splitlines()[-1].strip()+ return E.default_model("base")+++def setup(cond: str, mpath: str, batch: int, max_len: int, fast: bool, compile_: bool,+ view: int = 0):+ """evals.run's model setup: tokenizer, model, BIOS, Ctx, hook, ban, BatchDecoder."""+ import torch+ from transformers import AutoTokenizer++ kind, flags = E.parse_cond(cond)+ tok = AutoTokenizer.from_pretrained(mpath)+ model = E.load_model(mpath)+ model = model.cuda().eval()+ chip, bios, bios_id = None, [], None+ meta_path = Path(mpath) / "chipformer.json"+ if meta_path.exists():+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ meta = json.loads(meta_path.read_text())+ chip = (meta["chip_token_offset"], meta["chip_vocab"])+ if kind in E.BIOS_KINDS:+ import hashlib++ bios = [t + chip[0] for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ bios_id = f"{hashlib.sha1(bcd.ROM).hexdigest()[:10]} ({len(bcd.ROM)} B, OPS {bcd.OPS})"+ print(f"BIOS {bios_id}, boot record {len(bios)} tokens", flush=True)+ ctx = E.Ctx(tok, chip)+ hook = {"pytool_inline": E.executor("inline", ctx), "pytool": E.executor("template", ctx)}.get(kind)+ banned = []+ if "nb" in flags or "ne" in flags or kind == "merged_inline":+ gc = Path(mpath) / "generation_config.json"+ bad = json.loads(gc.read_text()).get("bad_words_ids", []) if gc.exists() else []+ # ne: <|endoftext|> too (Qwen otherwise ends a tool-mode turn right after the response)+ banned = sorted({w[0] for w in bad if len(w) == 1} | {E.IM_START}+ | ({getattr(E, "ENDOFTEXT", 248044)} if "ne" in flags else set()))+ model.lm_head = E.BannedHead(model.lm_head, banned, model.lm_head.weight.shape[0])+ print(f"banned token ids {banned}", flush=True)+ import torch._inductor.config++ from chipformer.q35.batch import BatchDecoder++ torch._inductor.config.shape_padding = False+ if fast and chip:+ from chipformer.q35 import fast as F++ print("compact", F.compact(model, qwen=False), flush=True)+ bd = BatchDecoder(model, batch=batch, max_len=max_len, eos=sorted(ctx.stop), sizes=[batch],+ compile=compile_, view=view,+ **({"skip_qwen": True} if fast and chip else {}),+ **({"freeze_chip_positions": True} if "fp" in flags else {}),+ **({"transparent_chip": True} if "tc" in flags else {}))+ return tok, model, ctx, hook, bios, bios_id, banned, bd+++def decode_rows(bd, ctx, prompts: list[list[int]], budget: int, hook, on_done,+ chip_per_full: int = 64, chip_min: int = 2, prefetch: int = 2,+ views_by_text: bool = False) -> None:+ """decode_views (same rows, stop rules, forcing, KV views), scheduled for throughput:++ - **Chip steps for some rows.** With skip_qwen, rows whose input is a chip token take the+ cheap chip step (the chip layers only) while the rows on text wait, whenever at least+ ``chip_min`` rows are on chip tokens (a chip token then costs a fraction of its slot in a+ full step), for up to ``chip_per_full`` chip steps in a row. In the tail (no prompt+ queued) that only delays the last rows, unless the full step reads KV views: then rows+ that finish their traces sooner shrink the view (on v15, batch 16: 298 s with chip steps+ in the tail vs 366 s without; with one view, 245 s without vs 335 s with). A waiting+ row's state is untouched (the chip step restores its chip-layer states and writes no+ Qwen KV for it), so every row sees the same token sequence as with full steps only: only+ the interleaving across rows changes, and rows are independent (the batch size, hence+ every GEMM kernel, stays the same). On v15, 40 of 40 problems came out token-identical+ to decode_views, and 20 of 20 to evals.decode_all.+ - **Prefill ahead.** Prompts are prefilled on a second CUDA stream (batch.BatchDecoder's+ prefetcher: same modules and kernels as ``bd.prefill``, so the same bits) while the batch+ decodes, instead of stalling every row at each refill.+ - **Views by text length** (``views_by_text``). With transparent_chip a row's KV holds only+ its text tokens, so the attention view can cover just those, however long the trace. Not+ bit-identical: the attention kernel's rounding depends on the view, so Qwen's logits move+ by up to a bf16 ulp and exact or 1-ulp ties can flip (dist/spikes/s5_tf.py: 2 of 40 v15+ problems changed, each at a top-2 gap of 0 or 0.125). Without it the views are sized as+ decode_views sizes them (by every token).+ """+ from collections import deque++ import torch++ from chipformer.q35.batch import _Prefetcher++ queue = deque(range(len(prompts)))+ n = bd.batch+ bd.prefill(prompts[0]) # the batched cache copies a prefilled cache's shapes+ level = bd._prepare(n)+ rows, tok, pos = level[:3]+ act = level[6]+ graphs = bd.view_graphs.get(n)+ views = graphs is not None and len(bd.views) > 1+ has_chip = level[4] is not None+ rpos = bd.rpos[n] if getattr(bd, "freeze", False) else None+ pre = _Prefetcher(bd, prompts, range(len(prompts)), max(1, prefetch))+ live: list = [None] * n+ last = [0] * n+ # KV slots each row's attention must read at its next step: its tokens so far, or with+ # transparent_chip only its text tokens (chip tokens write the next text token's slot)+ length = [0] * n+ tc = getattr(bd, "transparent", False) and views_by_text+ stats = bd.stats+ stats.setdefault("partial_chip_steps", 0)++ def admit(b):+ while queue:+ i = queue.popleft()+ cache, t = pre.get(i)+ r = E._Row(i, len(prompts[i]))+ if r.push(t, False, ctx, budget, bd.max_len, hook):+ pre.release(cache)+ on_done(i, r.result())+ continue+ rows.load_row(b, cache, len(prompts[i]))+ pre.release(cache)+ bd.mark_prompt(n, b, prompts[i]) # _tc: its chip slots+ tok[b, 0] = t+ pos[b] = len(prompts[i])+ if rpos is not None:+ rpos[b] = sum(not ctx.is_chip(x) for x in prompts[i])+ n_kv = sum(not ctx.is_chip(x) for x in prompts[i]) if tc else len(prompts[i])+ live[b], last[b], length[b] = r, t, n_kv + 1+ return+ live[b] = None+ tok[b, 0] = 0+ pos[b] = 0+ length[b] = 0++ credit = 0 # chip steps left before the rows on text get a full step+ try:+ for b in range(n):+ admit(b)+ with torch.no_grad():+ while any(r is not None for r in live):+ busy = [b for b in range(n) if live[b] is not None]+ on = [live[b] is not None and ctx.is_chip(last[b]) for b in range(n)]+ partial = False+ if has_chip and all(on[b] for b in busy):+ use_chip = True+ elif has_chip and (queue or views) and sum(on) >= chip_min and credit > 0:+ use_chip = partial = True+ credit -= 1+ else:+ use_chip = False+ credit = chip_per_full+ if graphs is not None:+ need = max(length[b] for b in busy)+ v = next(x for x in bd.views if x >= need)+ step_fn, chip_fn = graphs[v]+ stats["view_sum"] += v+ else:+ step_fn, chip_fn = level[3], level[4]+ if use_chip:+ act.copy_(torch.tensor(on, device=act.device))+ pred = chip_fn(tok, pos)+ pos.add_(act)+ stats["chip_steps"] += 1+ stats["partial_chip_steps"] += partial+ step_rows = [b for b in busy if on[b]]+ else:+ pred = step_fn(tok, pos)+ pos.add_(1)+ if rpos is not None: # the input was text: the next RoPE position moves on+ rpos.add_(bd._is_text(tok[:, 0]))+ step_rows = busy+ stats["steps"] += 1+ pred = pred.tolist()+ nt = list(last) if use_chip else [0] * n+ done = []+ for b in step_rows:+ r = live[b]+ forced = bool(r.forced)+ t = r.forced.popleft() if forced else pred[b]+ nt[b] = t+ length[b] += not (tc and ctx.is_chip(last[b]))+ stats["tokens"] += 1+ if r.push(t, forced, ctx, budget, bd.max_len, hook):+ done.append(b)+ if use_chip: # waiting rows keep their input token+ for b in range(n):+ if live[b] is None:+ nt[b] = 0+ tok.copy_(torch.tensor(nt, device=tok.device)[:, None])+ last = nt+ idle = [b for b in range(n) if live[b] is None]+ if idle:+ pos[idle] = 0+ if rpos is not None:+ rpos[idle] = 0+ for b in done:+ on_done(live[b].i, live[b].result())+ admit(b)+ finally:+ pre.close()+++def decode_views(bd, ctx, prompts: list[list[int]], budget: int, hook, on_done) -> None:+ """evals.decode_all (same rows, stop rules, forcing, chip steps), except that each step's+ attention reads only the smallest captured KV view covering every live row (BatchDecoder+ ``view``), instead of the whole static cache. Without views it is decode_all."""+ from collections import deque++ import torch++ if not getattr(bd, "views", None) or len(bd.views) < 2:+ return E.decode_all(bd, ctx, prompts, budget, hook, on_done)+ queue = deque(range(len(prompts)))+ n = bd.batch+ bd.prefill(prompts[0])+ level = bd._prepare(n)+ rows, tok, pos = level[:3]+ has_chip = level[4] is not None+ graphs = bd.view_graphs[n]+ # fp: RoPE positions frozen on chip tokens (a separate per-row buffer in BatchDecoder)+ rpos = bd.rpos[n] if getattr(bd, "freeze", False) else None+ first = bd.prefill(prompts[queue[0]])+ last = [0] * n+ live: list = [None] * n+ length = [0] * n # tokens in each row's cache after its next input++ def admit(b, first=None):+ while queue:+ i = queue.popleft()+ t = bd.prefill(prompts[i]) if first is None else first+ first = None+ r = E._Row(i, len(prompts[i]))+ if r.push(t, False, ctx, budget, bd.max_len, hook):+ on_done(i, r.result())+ continue+ rows.load_row(b, bd.small, len(prompts[i]))+ if hasattr(bd, "mark_prompt"): # _tc: its chip slots (no-op otherwise)+ bd.mark_prompt(n, b, prompts[i])+ tok[b, 0] = t+ pos[b] = len(prompts[i])+ if rpos is not None:+ rpos[b] = sum(not ctx.is_chip(x) for x in prompts[i])+ live[b], last[b], length[b] = r, t, len(prompts[i]) + 1+ return+ live[b] = None+ tok[b, 0] = 0+ pos[b] = 0+ length[b] = 0++ admit(0, first)+ for b in range(1, n):+ admit(b)+ with torch.no_grad():+ while any(r is not None for r in live):+ busy = [b for b in range(n) if live[b] is not None]+ use_chip = has_chip and all(ctx.is_chip(last[b]) for b in busy)+ need = max(length[b] for b in busy)+ v = next(x for x in bd.views if x >= need)+ step_fn, chip_fn = graphs[v]+ pred = (chip_fn if use_chip else step_fn)(tok, pos).tolist()+ bd.stats["view_sum"] += v+ pos.add_(1)+ if rpos is not None: # the input was text: the next RoPE position moves on+ rpos.add_(bd._is_text(tok[:, 0]))+ nt, done = [], []+ for b, p in enumerate(pred):+ r = live[b]+ if r is None:+ nt.append(0)+ continue+ forced = bool(r.forced)+ t = r.forced.popleft() if forced else p+ nt.append(t)+ length[b] += 1+ if r.push(t, forced, ctx, budget, bd.max_len, hook):+ done.append(b)+ tok.copy_(torch.tensor(nt, device=tok.device)[:, None])+ last = nt+ idle = [b for b in range(n) if live[b] is None]+ if idle:+ pos[idle] = 0+ if rpos is not None:+ rpos[idle] = 0+ for b in done:+ on_done(live[b].i, live[b].result())+ admit(b)+++def default_model(cond: str) -> str:+ return str(ROOT / "dist" / "q4-v10") if cond.startswith("merged") else snapshot()+++def run(args) -> None:+ import torch++ cond = args.cond + (f"-{args.tag}" if args.tag else "")+ kind, _ = E.parse_cond(cond)+ out = Path(args.out)+ out.mkdir(parents=True, exist_ok=True)+ todo = []+ for data in args.data:+ path = out / f"{data}.{cond}.jsonl"+ done = set()+ if path.exists():+ recs = [json.loads(x) for x in path.read_text(encoding="utf-8").splitlines()+ if x.strip()]+ # --redo-context: rerun problems that hit the cache length (the newer record is+ # appended; readers keep the last record per id)+ done = {r["id"] for r in recs if not (args.redo_context and r["stop"] == "context")}+ probs = load(data, args.n, args.seed)+ if args.per_data: # the first K of the (sorted, seeded) sample: a subset of the full run+ probs = probs[:args.per_data]+ probs = [p for p in probs if p["id"] not in done]+ print(f"{path.name}: {len(done)} done, {len(probs)} to go", flush=True)+ todo += [(data, p) for p in probs]+ if args.limit:+ todo = todo[:args.limit]+ if not todo:+ return+ mpath = args.model or default_model(cond)+ budget = args.budget+ batch = min(args.batch, len(todo))+ # the cache length is fixed per condition (one compile); prompts here are short+ max_len = args.max_len or (budget + 1024 + (4096 if cond.startswith("merged") else 512))+ max_len = -(-max_len // 1024) * 1024+ tok, _model, ctx, hook, bios, bios_id, banned, bd = setup(+ cond, mpath, batch, max_len, args.fast, not args.no_compile, args.view)+ prompts = [E.build_prompt(tok, p["question"], cond, bios) for _, p in todo]+ assert max(map(len, prompts)) + budget + 512 < max_len, "prompt too long for max_len"+ if kind == "merged" and not args.no_selftest:+ E.selftest(bd, ctx, tok, bios)+ print(f"batch {bd.batch}, max_len {max_len}, budget {budget}, {len(todo)} problems",+ flush=True)+ t0 = time.time()+ counts = {}++ def done(i, r):+ data, p = todo[i]+ try:+ rec = E.score(_judgeable(p), cond, ctx, r, r["seconds"], False)+ rec |= exact_fields(p, rec["text"])+ rec["gold"] = p["gold"]+ except (ArithmeticError, ValueError, TypeError, KeyError, IndexError) as e:+ # never lose the run to one transcript's scoring+ print(f"scoring error {p['id']}: {e!r}", flush=True)+ rec = {"id": p["id"], "cond": cond, "question": p["question"], "gold": p["gold"],+ "pred": None, "source": "score_error", "inline": [], "correct": False,+ "calls": [], "stop": r["stop"], "n_qwen": r["n_qwen"],+ "n_chip": sum(1 for t in r["out"] if ctx.is_chip(t)), "n_out": len(r["out"]),+ "seconds": round(r["seconds"], 2), "text": ctx.show(r["out"]),+ "pred_exact": None, "src_exact": "score_error", "exact": False,+ "score_error": repr(e)}+ rec |= {"data": data, "group": p["group"], "clean": p["clean"],+ "n_prompt": len(prompts[i]), "batch": bd.batch, "model": mpath,+ "bios": bios_id, "banned": banned, "ids": r["out"], "inj_at": r.get("inj_at")}+ if "length" in p:+ rec["length"] = p["length"]+ with (out / f"{data}.{cond}.jsonl").open("a", encoding="utf-8") as f:+ f.write(json.dumps(rec, ensure_ascii=False) + "\n")+ counts[data] = counts.get(data, 0) + 1+ print(json.dumps({k: rec[k] for k in ("id", "gold", "pred_exact", "exact", "stop",+ "n_qwen", "n_chip", "seconds")}+ | {"calls": len(rec["calls"]), "fires": len(rec["inline"])}), flush=True)++ if args.chip_per_full and bd.skip_qwen:+ decode_rows(bd, ctx, prompts, budget, hook, done, chip_per_full=args.chip_per_full,+ chip_min=args.chip_min, views_by_text=args.views_by_text)+ else:+ decode_views(bd, ctx, prompts, budget, hook, done)+ wall = time.time() - t0+ with (out / "runs.log").open("a", encoding="utf-8") as f:+ f.write(json.dumps({"cond": cond, "data": counts, "n": len(todo), "batch": bd.batch,+ "wall_s": round(wall, 1), "s_per_problem": round(wall / len(todo), 2),+ "view": args.view, "chip_per_full": args.chip_per_full,+ "views_by_text": args.views_by_text, "stats": bd.stats,+ "model": mpath, "bios": bios_id, "budget": budget,+ "gpu": torch.cuda.get_device_name()}) + "\n")+ print(f"done {len(todo)} in {wall:.0f}s ({wall / len(todo):.1f} s/problem)", flush=True)+++# ---------------------------------------------------------------------------------- checks++def identity(args) -> None:+ """BatchDecoder (the config the runs use) vs HF generate, greedy, on a few problems."""+ import torch++ cond = args.cond+ probs = load("dmx_mul_big", 100)[:2] + load("nupa_mulh", 400)[100:102] + load("bbh_mat2")[:1]+ mpath = args.model or default_model(cond)+ max_len = 1024 + 4096 + args.max_new+ max_len = -(-max_len // 1024) * 1024+ tok, model, ctx, _hook, bios, _, _banned, bd = setup(cond, mpath, args.batch, max_len,+ True, True)+ prompts = [E.build_prompt(tok, p["question"], cond, bios) for p in probs]+ got = {}+ E.decode_all(bd, ctx, prompts, args.max_new, None, lambda i, r: got.__setitem__(i, r["out"]))+ for i, p in enumerate(prompts):+ x = torch.tensor([p], device="cuda")+ with torch.no_grad():+ g = model.generate(x, max_new_tokens=len(got[i]), do_sample=False,+ cache_implementation="static")[0, len(p):].tolist()+ a = got[i]+ k = next((j for j in range(min(len(a), len(g))) if a[j] != g[j]), min(len(a), len(g)))+ n_chip = sum(ctx.is_chip(t) for t in a)+ print(json.dumps({"id": probs[i]["id"], "batch_len": len(a), "gen_len": len(g),+ "identical": a == g[:len(a)], "first_diff": k, "chip_tokens": n_chip,+ "diff_ctx": ctx.show(a[max(0, k - 20):k + 5]) if a != g[:len(a)] else ""}),+ flush=True)+++def tfcheck(args) -> None:+ """Teacher-forced check of stored records against HF's plain forward (eager, batch 1): at+ every generated position, is the stored token the HF model's argmax? Where it is not, the+ top-2 logit gap shows whether it is a bf16 near-tie (kernel rounding) or a real+ disagreement. Positions with a forced token (pytool) are skipped."""+ import torch+ from transformers import AutoTokenizer++ cond = args.cond+ mpath = args.model or default_model(cond)+ tok = AutoTokenizer.from_pretrained(mpath)+ model = E.load_model(mpath).cuda().eval()+ from chipformer.q35.fast import exact_matmuls++ exact_matmuls()+ kind, flags = E.parse_cond(cond)+ chip, bios = None, []+ meta_path = Path(mpath) / "chipformer.json"+ if meta_path.exists():+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ meta = json.loads(meta_path.read_text())+ chip = (meta["chip_token_offset"], meta["chip_vocab"])+ if kind in E.BIOS_KINDS:+ bios = [t + chip[0] for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ banned = []+ if "nb" in flags or "ne" in flags or kind == "merged_inline":+ gc = Path(mpath) / "generation_config.json"+ bad = json.loads(gc.read_text()).get("bad_words_ids", []) if gc.exists() else []+ # ne: <|endoftext|> too (Qwen otherwise ends a tool-mode turn right after the response)+ banned = sorted({w[0] for w in bad if len(w) == 1} | {E.IM_START}+ | ({getattr(E, "ENDOFTEXT", 248044)} if "ne" in flags else set()))+ out = Path(args.out)+ for data in args.data:+ src = out / f"{data}.{cond}.jsonl"+ recs = [json.loads(x) for x in src.read_text(encoding="utf-8").splitlines() if x.strip()]+ for r in recs[:args.k]:+ prompt = E.build_prompt(tok, r["question"], cond, bios)+ gen = r["ids"] if r.get("ids") is not None else tok(+ r["text"], add_special_tokens=False).input_ids+ ids = prompt + gen+ with torch.no_grad():+ lg = model(torch.tensor([ids], device="cuda")).logits[0].float()+ if banned:+ lg[:, banned] = float("-inf")+ lg = lg[len(prompt) - 1:len(ids) - 1]+ top2 = lg.topk(2, dim=-1)+ want = torch.tensor(gen, device="cuda")+ arg = top2.indices[:, 0]+ miss = (arg != want).nonzero().flatten().tolist()+ gaps = [round(float(lg[p, arg[p]] - lg[p, want[p]]), 3) for p in miss]+ chipmiss = [p for p in miss if chip and chip[0] <= gen[p] < chip[0] + chip[1]]+ print(json.dumps({"id": r["id"], "cond": cond, "n_gen": len(gen),+ "argmax_agree": len(gen) - len(miss), "disagree": len(miss),+ "first": miss[:1], "gap_at_disagree": gaps[:5],+ "max_gap": max(gaps) if gaps else 0.0,+ "chip_token_disagree": len(chipmiss)}), flush=True)+++def replay(args) -> None:+ """Replay every merged record through the circuit IR: at every position the circuit owns,+ the model's token must be the IR's prediction."""+ import numpy as np+ from transformers import AutoTokenizer++ from chipformer.q35 import bridge as B+ from chipformer.q35 import check as Q+ from chipformer.q35 import chip8_q35, ir+ from chipformer.q35.lib import bcd+ from chipformer.q35.spikes_replay import class_map+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ mpath = args.model or default_model("merged")+ meta = json.loads((Path(mpath) / "chipformer.json").read_text())+ off, nchip = meta["chip_token_offset"], meta["chip_vocab"]+ tok = AutoTokenizer.from_pretrained(mpath)+ cmap = class_map(tok, meta)+ circuit = chip8_q35.build()+ bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ out = Path(args.out)+ for data in args.data:+ src = out / f"{data}.{args.cond}.jsonl"+ if not src.exists():+ continue+ dst = out / f"{data}.{args.cond}.replay.jsonl"+ have = set()+ if dst.exists():+ have = {json.loads(x)["id"] for x in dst.read_text(encoding="utf-8").splitlines() if x}+ recs = []+ for x in src.read_text(encoding="utf-8").splitlines():+ try: # the run may be appending the last line right now+ recs.append(json.loads(x))+ except json.JSONDecodeError:+ pass+ if args.shard:+ k, m = map(int, args.shard.split("/"))+ recs = recs[k::m]+ for r in recs:+ if r["id"] in have or r.get("ids") is None:+ continue+ gen = r["ids"]+ if not any(off <= t < off + nchip for t in gen):+ res = {"id": r["id"], "chip_tokens": 0, "owned": 0, "mismatch": 0, "skipped": True}+ else:+ t0 = time.time()+ prompt = E.build_prompt(tok, r["question"], r["cond"], bios)+ ids = prompt + gen+ stream = [t - off if off <= t < off + nchip else cmap.get(t, B.OTHER)+ for t in ids]+ pred, m = Q.predictions(circuit, stream)+ owner = ir.ev(circuit.owner, m)+ rng = range(len(prompt) - 1, len(ids) - 1)+ owned = [p for p in rng if owner[p] == 1]+ bad = [p for p in owned if pred[p] != stream[p + 1]]+ # chip tokens Qwen wrote where the circuit does not own the position+ stray = [p for p in rng if owner[p] != 1 and off <= ids[p + 1] < off + nchip]+ res = {"id": r["id"], "chip_tokens": sum(off <= t < off + nchip for t in gen),+ "owned": len(owned), "mismatch": len(bad), "stray_chip": len(stray),+ "first": [(p - len(prompt), int(stream[p + 1]), int(pred[p]))+ for p in bad[:3]],+ "seconds": round(time.time() - t0, 1)}+ del m+ with dst.open("a", encoding="utf-8") as f:+ f.write(json.dumps(res) + "\n")+ print(data, json.dumps(res), flush=True)+ np.zeros(1)+++# ---------------------------------------------------------------------------------- summary++def wilson(k, n):+ return E.wilson(k, n)+++def paired(ra: dict, rb: dict, key="exact"):+ ids = sorted(set(ra) & set(rb))+ if not ids:+ return None+ xa = [ra[i][key] for i in ids]+ xb = [rb[i][key] for i in ids]+ n = len(ids)+ bo = sum(x and not y for x, y in zip(xa, xb))+ co = sum(y and not x for x, y in zip(xa, xb))+ d = (bo - co) / n+ se = math.sqrt(max(0.0, (bo + co) / n - d * d) / n)+ return n, d, d - 1.96 * se, d + 1.96 * se, bo, co, E.mcnemar(bo, co)+++def machine_stats(rs: list[dict]):+ """Every result the machine printed: tool responses of valid calls and parsed inline fires.+ exact = printed == A op B."""+ tool = [c["exact"] for r in rs for c in r["calls"] if c["valid"] and c["response"] is not None]+ inl = [f["exact"] for r in rs for f in r.get("inline", []) if "exact" in f]+ unparsed = sum("exact" not in f for r in rs for f in r.get("inline", []))+ return tool, inl, unparsed+++def load_all(out: Path):+ res, rep = {}, {}+ for f in sorted(out.glob("*.jsonl")):+ parts = f.name.split(".")+ if len(parts) == 3:+ recs = [json.loads(x) for x in f.read_text(encoding="utf-8").splitlines() if x.strip()]+ res[(parts[0], parts[1])] = {r["id"]: r for r in recs}+ elif len(parts) == 4 and parts[2] == "replay":+ recs = [json.loads(x) for x in f.read_text(encoding="utf-8").splitlines() if x.strip()]+ rep[(parts[0], parts[1])] = {r["id"]: r for r in recs}+ return res, rep+++def _pct(k, n):+ return f"{100 * k / n:.1f}%" if n else "-"+++def _table(res, rep, keys: list[tuple], conds: list[str], title: str, wall: dict) -> list[str]:+ """One table over the pooled records of ``keys`` (datasets), per condition."""+ L = [f"\n### {title}\n",+ ("| cond | N | exact acc | 95% CI | clean acc (n) | vs base (pts, 95% CI, McNemar p) | "+ "vs base_nudge | tool-call rate | inline-fire rate | answer = a printed result | "+ "machine exact (tool / inline) | IR replay mismatches | cutoffs | loops | Qwen tok | "+ "chip tok | s/prob |"),+ "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|"]+ pooled = {}+ for c in conds:+ d = {}+ for k in keys:+ d |= res.get((k, c), {})+ pooled[c] = d+ for c in conds:+ rs = list(pooled[c].values())+ if not rs:+ continue+ n, k = len(rs), sum(r["exact"] for r in rs)+ lo, hi = wilson(k, n)+ cl = [r["exact"] for r in rs if r["clean"]]+ cmp = []+ for b in ("base", "base_nudge"):+ pr = paired(pooled[c], pooled.get(b, {})) if c != b else None+ cmp.append(f"{100 * pr[1]:+.1f} ({100 * pr[2]:+.1f}..{100 * pr[3]:+.1f}), "+ f"p={pr[6]:.2g} [{pr[4]}/{pr[5]}]" if pr else "-")+ tool, inl, unp = machine_stats(rs)+ rp = [rep.get((key, c), {}) for key in keys]+ rpr = [x for dct in rp for i, x in dct.items() if i in pooled[c]]+ rp_s = "-"+ if rpr:+ checked = [x for x in rpr if not x.get("skipped")]+ rp_s = (f"{sum(x['mismatch'] for x in checked)} in "+ f"{sum(x['owned'] for x in checked)} owned pos. ({len(checked)} recs"+ f"{'' if len(rpr) == n else f', {len(rpr)}/{n} done'})")+ ws = [wall[(key, c)] for key in keys if (key, c) in wall]+ L.append(+ f"| {c} | {n} | {_pct(k, n)} | {100 * lo:.0f}-{100 * hi:.0f} | "+ f"{_pct(sum(cl), len(cl))} ({len(cl)}) | {cmp[0]} | {cmp[1]} | "+ f"{_pct(sum(bool(r['calls']) for r in rs), n)} | "+ f"{_pct(sum(bool(r.get('inline')) for r in rs), n)} | "+ f"{_pct(sum(_from_machine(r) for r in rs), n)} | "+ f"{sum(tool)}/{len(tool)} / {sum(inl)}/{len(inl)}{f' (+{unp} unparsed)' if unp else ''} | "+ f"{rp_s} | {sum(r['stop'] != 'eos' for r in rs)} | "+ f"{sum(E._looping(r.get('inline', [])) for r in rs)} | "+ f"{sum(r['n_qwen'] for r in rs) / n:.0f} | {sum(r['n_chip'] for r in rs) / n:.0f} | "+ f"{f'{sum(ws) / len(ws):.1f}' if ws else '-'} |")+ return L+++def _cause(r: dict) -> str:+ """Why a wrong answer is wrong (the first that applies)."""+ tool, inl, _ = machine_stats([r])+ if not all(tool) or not all(inl):+ return "wrong print"+ loop = E._looping(r.get("inline", []))+ if r["stop"] != "eos":+ return "cutoff (looping)" if loop else "cutoff"+ if r.get("pred_exact") is None:+ return "no answer"+ if loop:+ return "looping, then wrong"+ g = str(gold_value(r["gold"]))+ printed = {f.get("true") for f in r.get("inline", []) if f.get("exact")} | {+ (c.get("response") or "").strip() for c in r["calls"] if c.get("exact")}+ if g in printed:+ return "gold was printed, answer differs"+ return "Qwen's own error"+++CAUSES = ["wrong print", "cutoff (looping)", "cutoff", "looping, then wrong", "no answer",+ "gold was printed, answer differs", "Qwen's own error"]+++def _fail_table(res, keys, conds) -> list[str]:+ L = ["\nWrong answers by cause (pooled; loops = 3+ identical inline fires in a row):\n",+ "| cond | wrong | " + " | ".join(CAUSES) + " |", "|---|---|" + "---|" * len(CAUSES)]+ for c in conds:+ rs = [r for k in keys for r in res.get((k, c), {}).values()]+ if not rs:+ continue+ bad = [_cause(r) for r in rs if not r["exact"]]+ L.append(f"| {c} | {len(bad)}/{len(rs)} | " + " | ".join(+ str(bad.count(x)) for x in CAUSES) + " |")+ return L+++def _from_machine(r: dict) -> bool:+ """The final answer is a number the machine (or the Python tool) printed."""+ p = r.get("pred_exact")+ if p is None:+ return False+ return (any(f.get("is_answer") for f in r.get("inline", []))+ or any((c.get("response") or "").strip() == p for c in r["calls"]))+++def _wall(out: Path) -> dict:+ """(data, cond) -> wall seconds per problem of the run(s) that made it (a run that covers+ several datasets shares its wall time by problem count)."""+ tot = {}+ log = out / "runs.log"+ if log.exists():+ for line in log.read_text(encoding="utf-8").splitlines():+ x = json.loads(line)+ for d, n in x["data"].items():+ w, m = tot.get((d, x["cond"]), (0.0, 0))+ tot[(d, x["cond"])] = (w + x["s_per_problem"] * n, m + n)+ return {k: w / n for k, (w, n) in tot.items() if n}+++def _groups(res, keys, conds):+ """Accuracy per group (DM module / NUPA length bucket / Eleuther subset)."""+ groups = {}+ for k in keys:+ for c in conds:+ for r in res.get((k, c), {}).values():+ groups.setdefault(f"{k}:{r['group']}", set())+ L = ["\n| group | " + " | ".join(conds) + " |", "|---|" + "---|" * len(conds)]+ for g in sorted(groups, key=lambda s: (s.split(":")[0], [int(x) if x.isdigit() else x+ for x in re.split(r"(\d+)", s)])):+ k, name = g.split(":", 1)+ cells = []+ for c in conds:+ rs = [r for r in res.get((k, c), {}).values() if r["group"] == name]+ cells.append(f"{sum(r['exact'] for r in rs)}/{len(rs)}" if rs else "-")+ L.append(f"| {g} | " + " | ".join(cells) + " |")+ return L+++BENCH = [("DeepMind Mathematics, extrapolate (big / longer) test modules",+ [k for k in DM_MODULES if k.startswith("dmx") and k not in DM_DIV]),+ (("DeepMind Mathematics, extrapolate big modules, clean problems only (non-negative "+ "integers, no decimals)"), DMXC),+ ("DeepMind Mathematics, extrapolate division modules (div_big, mul_div_multiple_longer, "+ "clean div_big)", DM_DIV),+ ("DeepMind Mathematics, interpolate test modules", [k for k in DM_MODULES if k.startswith("dmi")]),+ ("NUPA integer add / sub / multiply (lengths 3-20)", list(NUPA_TASKS)),+ ("BBH multistep_arithmetic_two", ["bbh_mat2"]),+ ("EleutherAI/arithmetic", [f"eleu_{s}" for s in ELEU])]+++def summary(args) -> None:+ out = Path(args.out)+ res, rep = load_all(out)+ wall = _wall(out)+ conds = [c for c in CONDS if any(cc == c for _, cc in res)] + sorted(+ {cc for _, cc in res if cc not in CONDS})+ L = ["# Arithmetic benchmarks: merged (computer in the weights) vs base Qwen3.5-4B\n",+ PREAMBLE]+ for title, keys in BENCH:+ if not any((k, c) in res for k in keys for c in conds):+ continue+ L += _bench(res, rep, wall, conds, title, keys)+ text = "\n".join(L)+ print(text)+ (out / "SUMMARY.md").write_text(text + "\n", encoding="utf-8")+++PREAMBLE = (+ "Non-thinking, greedy, 2048 Qwen-written tokens per problem (the machine's tokens are not "+ "counted), evals.py's prompt suffix (final answer in \\boxed{}). Problems are seeded "+ "samples, identical across conditions (paired). exact acc = boxed (else last) number equals "+ "the gold exactly as a rational (no tolerance). clean = all numbers in the problem and the "+ "answer are non-negative integers and there is no division. Paired deltas: A - B in points "+ "with a 95% CI, exact McNemar p, [A-only/B-only wins]. machine exact = printed result == "+ "A op B over every valid tool call / parsed inline fire. IR replay = positions owned by the "+ "circuit where the model's token differs from the circuit IR's prediction (spikes_replay). "+ "cutoffs = budget or context stops; loops = 3+ identical inline fires in a row. s/prob = "+ "wall time per problem at the run's batch size (base / base_nudge / pytool / merged_nb: L4, "+ "batch 16 / 16 / 16 / 8; v11 merged_inline*: L40S, batch 32; v13: L4 / A10G, batch 8). "+ "Models: merged_nb = q4-v12 (commit 62ed528); merged_inline* = q4-v11 (afd8230; v12 only "+ "changes tool-call capture); -v13 = q4-v13 (f270d1b: inline A - B < 0 prints its negative "+ "result, inline add/sub need >= 7 digits). _fp = RoPE positions frozen on chip tokens; "+ "_tc = chip tokens invisible to Qwen's layers; _mix = NUDGE_MIX (products inline, big "+ "add/sub through the tools).\n")+++def _bench(res, rep, wall, conds, title, keys) -> list[str]:+ L = [f"\n## {title}"]+ L += _table(res, rep, keys, conds, "all modules pooled" if len(keys) > 1 else title, wall)+ L += _fail_table(res, keys, conds)+ if len(keys) > 1:+ for k in keys:+ if any((k, c) in res for c in conds):+ L += _table(res, rep, [k], conds, k, wall)+ L += _fail_table(res, [k], conds)+ L.append("\nPer group (exact correct / N):")+ L += _groups(res, keys, conds)+ return L+++def _tags(r: dict) -> set:+ fires = r.get("inline", [])+ t = set()+ if any(f.get("true", "0").startswith("-") for f in fires):+ t.add("negative fire")+ if E._looping(fires):+ t.add("loop")+ if r["stop"] != "eos":+ t.add("cutoff")+ return t+++def losses(args) -> None:+ """Paired wins / losses of A vs B on the given datasets, each problem tagged with what+ happened in A's transcript (negative inline fire, loop, cutoff)."""+ res, _ = load_all(Path(args.out))+ L = [f"\n{args.a} vs {args.b}\n",+ ("| data | N | A | B | A-only | B-only (losses) | losses with negative fire | with loop | "+ "with cutoff | none of these | McNemar p |"),+ "|---|---|---|---|---|---|---|---|---|---|---|"]+ for data in [*args.data, "pooled"]:+ keys = args.data if data == "pooled" else [data]+ ra = {i: r for k in keys for i, r in res.get((k, args.a), {}).items()}+ rb = {i: r for k in keys for i, r in res.get((k, args.b), {}).items()}+ ids = sorted(set(ra) & set(rb))+ if not ids:+ continue+ win = [i for i in ids if ra[i]["exact"] and not rb[i]["exact"]]+ lose = [i for i in ids if rb[i]["exact"] and not ra[i]["exact"]]+ tags = [_tags(ra[i]) for i in lose]+ L.append(f"| {data} | {len(ids)} | {sum(ra[i]['exact'] for i in ids)} | "+ f"{sum(rb[i]['exact'] for i in ids)} | {len(win)} | {len(lose)} | "+ f"{sum('negative fire' in t for t in tags)} | {sum('loop' in t for t in tags)} | "+ f"{sum('cutoff' in t for t in tags)} | {sum(not t for t in tags)} | "+ f"{E.mcnemar(len(win), len(lose)):.2g} |")+ print("\n".join(L))+++def main() -> None:+ ap = argparse.ArgumentParser()+ sub = ap.add_subparsers(dest="cmd", required=True)+ r = sub.add_parser("run")+ r.add_argument("cond", type=E.cond_arg)+ r.add_argument("data", nargs="+", choices=ALL)+ r.add_argument("--n", type=int, help="problems per dataset (default per family)")+ r.add_argument("--seed", type=int, default=0)+ r.add_argument("--model")+ r.add_argument("--out", default=str(OUT))+ r.add_argument("--budget", type=int, default=2048)+ r.add_argument("--max-len", type=int)+ r.add_argument("--batch", type=int, default=16)+ r.add_argument("--limit", type=int, help="only the first N problems still to do")+ r.add_argument("--per-data", type=int, help="only the first K problems of each dataset")+ r.add_argument("--redo-context", action="store_true",+ help="rerun problems whose stored record stopped at the cache length")+ r.add_argument("--no-compile", action="store_true")+ r.add_argument("--no-selftest", action="store_true")+ r.add_argument("--fast", action="store_true", default=True)+ r.add_argument("--tag")+ r.add_argument("--view", type=int, default=1024,+ help="attention KV view granularity (0 = whole cache; the view can move "+ "Qwen's attention rounding by a bf16 ulp, dist/spikes/s5_tf.py)")+ r.add_argument("--chip-per-full", type=int, default=64,+ help="decode_rows: chip steps in a row for the rows on chip tokens before a "+ "full step (same tokens as decode_views; 0 = decode_views)")+ r.add_argument("--chip-min", type=int, default=2,+ help="decode_rows: rows on chip tokens needed for a chip step")+ r.add_argument("--views-by-text", action="store_true",+ help="decode_rows under tc: size KV views by text tokens only (faster; Qwen's "+ "attention rounds differently per view: bf16 logit near-ties can flip)")+ i = sub.add_parser("identity")+ i.add_argument("--cond", default="merged_inline")+ i.add_argument("--model")+ i.add_argument("--batch", type=int, default=8)+ i.add_argument("--max-new", type=int, default=1024)+ t = sub.add_parser("tfcheck")+ t.add_argument("cond")+ t.add_argument("data", nargs="+", choices=ALL)+ t.add_argument("--model")+ t.add_argument("--out", default=str(OUT))+ t.add_argument("--k", type=int, default=5)+ p = sub.add_parser("replay")+ p.add_argument("cond")+ p.add_argument("data", nargs="+", choices=ALL)+ p.add_argument("--model")+ p.add_argument("--out", default=str(OUT))+ p.add_argument("--shard")+ lo = sub.add_parser("losses", help="paired wins / losses of A vs B, losses tagged")+ lo.add_argument("a")+ lo.add_argument("b")+ lo.add_argument("data", nargs="+", choices=ALL)+ lo.add_argument("--out", default=str(OUT))+ e = sub.add_parser("rescore-errors")+ e.add_argument("--out", default=str(OUT))+ s = sub.add_parser("summary")+ s.add_argument("--out", default=str(OUT))+ ld = sub.add_parser("show", help="print a few loaded problems")+ ld.add_argument("data", nargs="+", choices=ALL)+ ld.add_argument("--k", type=int, default=3)+ args = ap.parse_args()+ if args.cmd == "show":+ for d in args.data:+ ps = load(d)+ print(d, len(ps), "clean", sum(p["clean"] for p in ps))+ for p in ps[:args.k]:+ print(" ", json.dumps(p))+ return+ {"run": run, "summary": summary, "replay": replay, "identity": identity,+ "tfcheck": tfcheck, "rescore-errors": rescore_errors,+ "losses": losses}[args.cmd](args)+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/batch.py b/chipformer/q35/batch.pynew file mode 100644index 0000000..97d1614--- /dev/null+++ b/chipformer/q35/batch.py@@ -0,0 +1,839 @@+"""Batched greedy decoding of q35 checkpoints with stock modules, token-identical to decoding+each prompt on its own.++ outs = generate_batch(model, tok, prompts, max_new=4500, batch=16)++How it stays equal to ``model.generate(do_sample=False)`` on one prompt:++- **Prefill alone.** Each prompt is prefilled on its own (batch 1, no padding) through the same+ modules ``generate`` uses, so its states and first token are the batch-1 ones. The states are+ then copied into one row of the batched cache. There is no padding anywhere, so nothing+ depends on ``apply_mask_to_padding_states`` or on DeltaNet chunk alignment, and the circuit+ never sees a pad token.+- **Per-row positions.** Every row keeps its own position: full attention writes key/value ``t``+ of row ``b`` at slot ``t`` (like the batch-1 static cache), RoPE gets the row's own position,+ and the mask of row ``b`` admits slots ``<= pos[b]``. Gated DeltaNet rows are independent.+- **Continuous batching.** A row that emits an end token (or reaches ``max_new``) is refilled+ with the next prompt at once, so the batch stays full while the queue lasts. When the queue+ is empty and few rows are left, the live rows are compacted into a smaller batch.++What can still differ from batch 1 is floating-point rounding in Qwen's own text arithmetic+(GEMM kernels for M = 1 and M = B accumulate in different orders); the circuit's arithmetic is+exact either way (``torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction`` must be+False, see ``fast.exact_matmuls``).++Decoding modes (each step is a CUDA graph per batch size, captured once):++- ``compile=True`` (default): Inductor kernels. Fastest; at batch 1 it matches+ ``generate(cache_implementation="static")`` (which also compiles) only while the graph is the+ same as HF's (no compaction), since any change shifts Inductor's fusion and Qwen's rounding.+- ``compile=False, kv_len="generate"``: eager kernels, attending over exactly generate's static+ cache length (``len(prompt) + max_new``; the attention result depends on it). At batch 1 this+ is bit-identical to ``generate(cache_implementation="static", disable_compile=True)``, also+ with ``fast.compact`` and ``skip_qwen`` (dist/spikes/b22_eager.py).++Options: ``skip_qwen`` (chip-token inputs run only the chip layers; Qwen's layers are exact+no-ops there), ``freeze_chip_positions`` (RoPE counts only non-chip tokens), ``run(hook=...)``+(token forcing / custom stops), generation_config ``bad_words_ids`` are honored. Prompts are+prefilled ahead on a second stream (``prefetch``) while the batch decodes.+"""++import time+from collections import deque++import torch+++def _text_model(model):+ inner = model.model+ return getattr(inner, "language_model", inner)+++def _mark(t):+ torch._dynamo.mark_static_address(t)+ return t+++def _fold_sdpa(module, query, key, value, attention_mask, dropout=0.0, scaling=None, **kw):+ """HF's SDPA attention, except that a single-token query attends without ``repeat_kv``:+ the GQA group goes into the query axis (heads ``j*G .. j*G+G-1`` share KV head ``j``), so+ the cache is not copied G times per layer and step. Bit-identical to the repeat_kv path+ (dist/spikes/b14_fold.py); prompts (more than one query) take HF's path unchanged."""+ from transformers.integrations.sdpa_attention import sdpa_attention_forward++ b, h, q, d = query.shape+ kv = key.shape[1]+ if q != 1 or h == kv or dropout:+ return sdpa_attention_forward(module, query, key, value, attention_mask,+ dropout=dropout, scaling=scaling, **kw)+ out = torch.nn.functional.scaled_dot_product_attention(+ query.view(b, kv, h // kv, d), key, value, attn_mask=attention_mask, scale=scaling)+ return out.reshape(b, h, 1, d).transpose(1, 2).contiguous(), None+++def _use_fold(model) -> None:+ from transformers import AttentionInterface++ AttentionInterface.register("q35_fold", _fold_sdpa)+ cfg = getattr(model.config, "text_config", model.config)+ if cfg._attn_implementation in ("sdpa", "q35_fold"):+ cfg._attn_implementation = "q35_fold"+++class _Rows:+ """Batched cache: linear-attention layers (stock ``LinearAttentionLayer``), and full-attention+ layers whose ``update`` writes each row at its own position."""++ def __init__(self, cfg, small, batch: int, max_len: int, pos: torch.Tensor,+ kvpos: torch.Tensor | None = None):+ from transformers.cache_utils import Cache, LinearAttentionLayer, StaticLayer++ self.view = max_len # attention reads slots [0, view): a compile-time constant+ owner = self+ slot = pos if kvpos is None else kvpos # KV slot per row (transparent: text count)++ class RowKV(StaticLayer):+ def update(self, key_states, value_states, *args, **kwargs):+ # decode only: one new position per row+ rows = torch.arange(slot.shape[0], device=slot.device)+ self.keys[rows, :, slot] = key_states[:, :, 0]+ self.values[rows, :, slot] = value_states[:, :, 0]+ return self.keys[:, :, :owner.view], self.values[:, :, :owner.view]++ layers = []+ for kind, s in zip(cfg.layer_types, small.layers):+ if kind == "linear_attention":+ la = LinearAttentionLayer()+ conv, rec = s.conv_states[0], s.recurrent_states[0]+ la.dtype, la.device = conv.dtype, conv.device+ la.conv_states[0] = _mark(conv.new_zeros(batch, *conv.shape[1:]))+ la.recurrent_states[0] = _mark(rec.new_zeros(batch, *rec.shape[1:]))+ la.conv_kernel_size[0] = conv.shape[-1]+ la.is_conv_states_initialized[0] = True+ la.is_recurrent_states_initialized[0] = True+ la.has_previous_state[0] = True+ layers.append(la)+ else:+ kv = RowKV(max_cache_len=max_len)+ kv.dtype, kv.device = s.keys.dtype, s.keys.device+ kv.batch_size, kv.num_heads = batch, s.keys.shape[1]+ kv.k_head_dim, kv.v_head_dim = s.keys.shape[-1], s.values.shape[-1]+ kv.keys = _mark(s.keys.new_zeros(batch, s.keys.shape[1], max_len, s.keys.shape[-1]))+ kv.values = _mark(s.values.new_zeros(batch, s.values.shape[1], max_len,+ s.values.shape[-1]))+ kv.cumulative_length = kv.cumulative_length.to(pos.device)+ kv.is_initialized = True+ layers.append(kv)+ self.cache = Cache(layers=layers)+ self.kinds = list(cfg.layer_types)++ @torch.no_grad()+ def load_row(self, b: int, small, n: int) -> None:+ """Copy a batch-1 prefill cache holding ``n`` positions into row ``b``."""+ for kind, big, s in zip(self.kinds, self.cache.layers, small.layers):+ if kind == "linear_attention":+ big.conv_states[0][b].copy_(s.conv_states[0][0])+ big.recurrent_states[0][b].copy_(s.recurrent_states[0][0])+ else:+ big.keys[b, :, :n].copy_(s.keys[0, :, :n])+ big.values[b, :, :n].copy_(s.values[0, :, :n])++ @torch.no_grad()+ def take(self, idx: torch.Tensor, into: "_Rows") -> None:+ """Copy rows ``idx`` of this cache into rows 0.. of ``into`` (a smaller batch)."""+ for big, small in zip(self.cache.layers, into.cache.layers):+ if hasattr(big, "keys") and big.keys is not None:+ small.keys[: len(idx)].copy_(big.keys[idx])+ small.values[: len(idx)].copy_(big.values[idx])+ else:+ small.conv_states[0][: len(idx)].copy_(big.conv_states[0][idx])+ small.recurrent_states[0][: len(idx)].copy_(big.recurrent_states[0][idx])+++class _Prefetcher:+ """Prefills the queued prompts ahead of time on a second CUDA stream, from a worker thread,+ into a few spare batch-1 caches; the decode loop then only copies a ready cache into a row.+ Same modules and kernels as ``BatchDecoder.prefill``, so the states are the same bits; the+ GPU overlaps the (launch-bound) prefill with the decode steps."""++ def __init__(self, dec: "BatchDecoder", prompts, order, depth: int, lens=None):+ import threading++ from transformers import StaticCache++ self.dec, self.prompts, self.lens = dec, prompts, lens+ self.order = list(order)+ self.ready: dict[int, tuple] = {}+ self.cv = threading.Condition()+ self.stream = torch.cuda.Stream()+ while len(dec.spare) < depth:+ dec.spare.append(StaticCache(config=dec.cfg, max_cache_len=dec.max_len))+ self.free = [(c, None) for c in dec.spare[:depth]] # (cache, event: free after it)+ self.stop = False+ self.error = None+ self.thread = threading.Thread(target=self._work, daemon=True)+ self.thread.start()++ def _work(self):+ try:+ with torch.no_grad(), torch.cuda.stream(self.stream):+ for i in self.order:+ with self.cv:+ while not self.free and not self.stop:+ self.cv.wait()+ if self.stop:+ return+ cache, ev = self.free.pop()+ if ev is not None:+ self.stream.wait_event(ev) # the row copy out of it is done+ if self.lens is not None: # a cache of generate's length for this prompt+ from transformers import StaticCache++ cache = StaticCache(config=self.dec.cfg, max_cache_len=self.lens[i])+ t0 = time.perf_counter()+ t = self.dec._prefill_into(cache, self.prompts[i])+ done = torch.cuda.Event()+ done.record(self.stream)+ with self.cv:+ self.dec.stats["prefills"] += 1+ self.dec.stats["prefill_s"] += time.perf_counter() - t0+ self.ready[i] = (cache, t, done)+ self.cv.notify_all()+ except BaseException as e: # noqa: BLE001 - surfaced in get()+ with self.cv:+ self.error = e+ self.cv.notify_all()++ def get(self, i: int):+ """(cache, first token) for prompt ``i``; call ``release`` after copying from it."""+ with self.cv:+ while i not in self.ready:+ if self.error is not None:+ raise self.error+ self.cv.wait()+ cache, t, done = self.ready.pop(i)+ torch.cuda.current_stream().wait_event(done)+ return cache, t++ def release(self, cache):+ ev = torch.cuda.Event()+ ev.record()+ with self.cv:+ self.free.append((cache, ev))+ self.cv.notify_all()++ def close(self):+ with self.cv:+ self.stop = True+ self.cv.notify_all()+ self.thread.join()+++class BatchDecoder:+ """Greedy decoding of many prompts through one model, ``batch`` rows at a time.++ ``sizes`` are the batch sizes with a compiled step (the largest is the full batch); when the+ queue is empty and the live rows fit a smaller size, they are compacted into it.+ """++ def __init__(self, model, batch: int = 16, max_len: int = 8192, eos=None,+ compile: bool = True, sizes=None, mode: str = "graphs",+ fullgraph: bool = True, skip_qwen: bool = False, chip_per_full: int = 3,+ fold_gqa: bool = False, view: int = 0, prefetch: int = 2,+ freeze_chip_positions: bool = False, kv_len: str | None = None,+ transparent_chip: bool = False):+ from transformers import StaticCache++ from chipformer.q35.fast import _n_chip_layers, chip_meta, exact_matmuls++ exact_matmuls() # batch sizes change cuBLAS's kernel choice; keep the circuit exact+ if fold_gqa:+ _use_fold(model)++ self.model = model+ self.lm = _text_model(model)+ self.head = model.lm_head # an nn.Linear, or fast.SparseLinear after fast.compact+ self.cfg = getattr(model.config, "text_config", model.config)+ self.dev = model.lm_head.weight.device+ self.max_len = max_len+ if eos is None:+ eos = model.generation_config.eos_token_id+ eos = [eos] if isinstance(eos, int) else list(eos or [])+ self.eos = set(eos)+ # generate's bad_words_ids / suppress_tokens (single tokens): never predicted+ gc = model.generation_config+ ban = set(getattr(gc, "suppress_tokens", None) or [])+ for w in getattr(gc, "bad_words_ids", None) or []:+ assert len(w) == 1, f"multi-token bad word {w} is not supported"+ ban.add(w[0])+ self.banned = torch.tensor(sorted(ban), dtype=torch.long, device=self.dev) if ban else None+ self.small = StaticCache(config=self.cfg, max_cache_len=max_len)+ self.sizes = sorted(set(sizes or [batch]) | {batch})+ self.batch = batch+ self.compile = compile+ self.mode, self.fullgraph = mode, fullgraph+ self.levels = {} # batch size -> (rows cache, tok, pos, step, chip step, chip keys)+ self.ready = set()+ self.pool, self.graphs = None, []+ # attention views: KV slots read per step, by the longest live row+ self.views = sorted({*range(view, max_len, view), max_len}) if view else [max_len]+ self.view_graphs = {} # batch size -> {view: [text step, chip step]}+ self.raw = {} # batch size -> compiled (uncaptured) steps+ self.prefetch = prefetch # prompts prefilled ahead, on a second stream+ self.spare = [] # their batch-1 caches+ torch._dynamo.config.cache_size_limit = max(torch._dynamo.config.cache_size_limit, 64)+ self.slots = torch.arange(max_len, device=self.dev)+ self.chip_per_full = chip_per_full+ # kv_len="generate": attention reads exactly generate's static cache length+ # (prompt + max_new) in the prefill and every step. The attention kernel's result+ # depends on that length, so this is what makes batch 1 bit-identical to+ # generate(cache_implementation="static", disable_compile=True). Eager kernels only.+ self.kv_exact = kv_len == "generate"+ assert not (self.kv_exact and compile), "kv_len='generate' needs compile=False"+ meta = chip_meta(model)+ self.rpos = {} # batch size -> RoPE position buffer (``pos`` itself unless frozen)+ self.freeze = (freeze_chip_positions or transparent_chip) and meta is not None+ if meta is not None:+ self.chip_lo = meta["chip_token_offset"]+ self.chip_hi = self.chip_lo + meta["chip_vocab"]+ # transparent_chip (implies freeze_chip_positions): chip tokens are invisible to Qwen's+ # layers. A chip-token input leaves their conv windows as they were (the stock model+ # shifts in a zero), and Qwen's full attention keeps its KV by text position: a chip+ # token's key goes to the slot the next text token overwrites, so text keys are+ # contiguous and no query ever reads a chip key. Qwen's decode steps then compute what+ # they would on the same text with the chip tokens deleted (dist/spikes/b26_*.py).+ self.transparent = transparent_chip and meta is not None+ if meta is not None:+ self.n_chip = _n_chip_layers(model)+ self.skip_qwen = skip_qwen and meta is not None+ if self.skip_qwen:+ self.n_chip, self.chip_dims = _n_chip_layers(model), meta["chip_dims_offset"]+ self.chip_lo = meta["chip_token_offset"]+ self.chip_hi = self.chip_lo + meta["chip_vocab"]+ self.stats = {"view_sum": 0, "chip_steps": 0, "steps": 0, "row_steps": 0, "tokens": 0, "prefills": 0, "prefill_s": 0.0}++ # --- one batch size ---------------------------------------------------------------------++ def _level(self, n: int):+ if n not in self.levels:+ pos = _mark(torch.zeros(n, dtype=torch.long, device=self.dev))+ tok = _mark(torch.zeros(n, 1, dtype=torch.long, device=self.dev))+ # RoPE position per row: the slot, or with freeze_chip_positions the number of+ # non-chip tokens before it (chip layers have no RoPE; Qwen sees contiguous text)+ rpos = _mark(torch.zeros(n, dtype=torch.long, device=self.dev)) if self.freeze else pos+ self.rpos[n] = rpos+ transparent = self.transparent+ kvpos = rpos if transparent else pos # the KV slot each row writes+ rows = _Rows(self.cfg, self.small, n, self.max_len, pos, kvpos)+ cache, slots, lm, head = rows.cache, self.slots, self.lm, self.head+ if transparent:+ kinds = self.cfg.layer_types+ qlin = [cache.layers[i] for i in range(self.n_chip, len(kinds))+ if kinds[i] == "linear_attention"]+ lo, hi = self.chip_lo, self.chip_hi++ def step(tok, pos):+ # slots past the view are masked anyway: reading fewer of them is bit-identical+ # (fully masked key blocks add exact zeros; dist/spikes/b14_fold.py)+ mask = slots[None, :rows.view] <= kvpos[:, None]+ if transparent:+ ischip = (tok[:, 0] >= lo) & (tok[:, 0] < hi)+ saved = [L.conv_states[0].clone() for L in qlin]+ h = lm(input_ids=tok, position_ids=rpos[:, None], past_key_values=cache,+ attention_mask={"full_attention": mask[:, None, None, :],+ "linear_attention": None},+ use_cache=True).last_hidden_state[:, -1]+ if transparent: # chip inputs: Qwen's conv windows stay as they were+ for L, c in zip(qlin, saved):+ L.conv_states[0].copy_(torch.where(ischip[:, None, None], c,+ L.conv_states[0]))+ return self._argmax(head(h))++ chip = kconst = None+ act = _mark(torch.ones(n, dtype=torch.bool, device=self.dev))+ if self.skip_qwen:+ chip, kconst = self._chip_step(rows, n, act, kvpos)+ if self.compile:+ # "graphs": inductor kernels, captured below as one plain CUDA graph per step+ # kind (no cudagraph-tree bookkeeping between the two kinds of step)+ mode = None if self.mode == "graphs" else self.mode+ step = torch.compile(step, mode=mode, fullgraph=self.fullgraph)+ if chip is not None:+ chip = torch.compile(chip, mode=mode, fullgraph=self.fullgraph)+ self.levels[n] = (rows, tok, pos, step, chip, kconst, act)+ return self.levels[n]++ def _chip_step(self, rows, n, act, kvpos):+ """The step for rows whose input is a chip token. Qwen's layers are exact no-ops there+ (muted: residual unchanged, DeltaNet state unchanged, conv window shifts in a zero,+ full attention stores a value of 0 under a fixed key; dist/spikes/b5_skip.py checks all+ of it bitwise), so they are replaced by those cache writes, and the head reads only the+ chip dims (Qwen's dims of the hidden state are exactly 0)."""+ from chipformer.q35.fast import SparseLinear++ lm, cache, kinds = self.lm, rows.cache, self.cfg.layer_types+ n_chip, d0 = self.n_chip, self.chip_dims+ attn = [cache.layers[i] for i in range(n_chip, len(kinds)) if kinds[i] == "full_attention"]+ lin = [cache.layers[i] for i in range(n_chip, len(kinds))+ if kinds[i] == "linear_attention"]+ kconst = [_mark(L.keys.new_zeros(L.keys.shape[1], L.keys.shape[-1])) for L in attn]+ w = self.head.weight+ c0 = self.head.c0 if isinstance(self.head, SparseLinear) else 0+ c1 = c0 + w.shape[1]+ if not isinstance(self.head, SparseLinear):+ from chipformer.q35.fast import _hull, nonzero_cols++ c1 = _hull(nonzero_cols(w))[1]+ block = w[:, d0 - c0:c1 - c0].contiguous()+ chip_layers = lm.layers[:n_chip]+ chip_lin = [cache.layers[i] for i in range(n_chip) if kinds[i] == "linear_attention"]+ transparent = self.transparent+ def chip(tok, pos):+ # rows with act False (text rows waiting for a full step) come out untouched;+ # ``act`` is the level's buffer, all True unless the caller sets it+ saved = [(L.conv_states[0].clone(), L.recurrent_states[0].clone()) for L in chip_lin]+ h = lm.embed_tokens(tok)+ for layer in chip_layers:+ h = layer(h, position_embeddings=None, past_key_values=cache, use_cache=True)+ a3 = act[:, None, None]+ for L, (c, r) in zip(chip_lin, saved):+ L.conv_states[0].copy_(torch.where(a3, L.conv_states[0], c))+ L.recurrent_states[0].copy_(torch.where(act[:, None, None, None],+ L.recurrent_states[0], r))+ idx = torch.arange(pos.shape[0], device=pos.device)+ for L, k in zip(attn, kconst):+ L.keys[idx, :, kvpos] = torch.where(a3, k, L.keys[idx, :, kvpos])+ L.values[idx, :, kvpos] = torch.where(a3, 0.0, L.values[idx, :, kvpos])+ if not transparent: # transparent: Qwen's conv windows stay as they were+ for L in lin:+ c = L.conv_states[0]+ c.copy_(torch.where(a3, torch.nn.functional.pad(c[..., 1:], (0, 1)), c))+ h = lm.norm(h)[:, -1, d0:c1]+ return self._argmax(torch.nn.functional.linear(h, block))++ return chip, kconst++ def _argmax(self, logits):+ if self.banned is not None: # as generate's logits processors do: score -inf+ logits = logits.index_fill(-1, self.banned, float("-inf"))+ return logits.argmax(-1)++ @torch.no_grad()+ def prefill(self, ids: list[int]) -> int:+ """Batch-1 prefill into ``self.small`` (as ``generate`` does it); returns the first token."""+ t0 = time.perf_counter()+ t = self._prefill_into(self.small, ids)+ self.stats["prefills"] += 1+ self.stats["prefill_s"] += time.perf_counter() - t0+ return t++ def _rope_positions(self, ids: list[int]) -> torch.Tensor:+ if not self.freeze:+ return torch.arange(len(ids), device=self.dev)+ lo, hi = self.chip_lo, self.chip_hi+ p, out = 0, []+ for t in ids:+ out.append(p)+ p += not lo <= t < hi+ return torch.tensor(out, device=self.dev)++ def _is_text(self, tok):+ return (tok < self.chip_lo) | (tok >= self.chip_hi)++ def chip_mask(self, ids: list[int]) -> torch.Tensor:+ """[len(ids)] bool on the device: which tokens are chip tokens."""+ x = torch.tensor(ids, device=self.dev)+ return (x >= self.chip_lo) & (x < self.chip_hi)++ def mark_prompt(self, n: int, b: int, ids: list[int]) -> None:+ """Call after loading a prefilled prompt into row ``b`` of the batch-``n`` level. With+ transparent_chip it moves the row's full-attention KV to text positions (the prefill+ wrote one slot per token); a no-op otherwise."""+ if not self.transparent:+ return+ text = (~self.chip_mask(ids)).nonzero().flatten()+ if len(text) == len(ids):+ return+ for L in self.levels[n][0].cache.layers:+ if getattr(L, "keys", None) is not None:+ L.keys[b, :, :len(text)] = L.keys[b, :, text]+ L.values[b, :, :len(text)] = L.values[b, :, text]++ def _prefill_into(self, cache, ids: list[int]) -> int:+ cache.reset()+ x = torch.tensor([ids], device=self.dev)+ mask = torch.ones_like(x)+ if self.transparent:+ ch = self.chip_mask(ids)+ if bool(ch.any()):+ # chip keys are invisible (a chip query keeps its own). A leading run of chip+ # tokens (the BIOS) also matches Qwen's zero-padded conv window exactly; chip+ # tokens later in a prompt still shift zeros into the conv windows.+ n = len(ids)+ size = next(L.max_cache_len for L in cache.layers if hasattr(L, "max_cache_len"))+ q = torch.arange(n, device=self.dev)[:, None]+ k = torch.arange(size, device=self.dev)[None, :]+ vis = (k <= q) & (~torch.nn.functional.pad(ch, (0, size - n))[None, :] | (k == q))+ mask = {"full_attention": vis[None, None], "linear_attention": None}+ h = self.lm(input_ids=x, position_ids=self._rope_positions(ids)[None],+ attention_mask=mask, past_key_values=cache,+ use_cache=True).last_hidden_state[:, -1:]+ return int(self._argmax(self.head(h)[0, -1]))++ @torch.no_grad()+ def _prepare(self, n: int):+ """The level of batch size ``n``, compiled and warmed; for the chip step, its fixed+ keys are taken from a real full step on a chip-token input."""+ level = self._level(n)+ if n in self.ready:+ return level+ rows, tok, pos, step, chip, kconst, _act = level+ tok.zero_()+ pos.zero_()+ for _ in range(3):+ step(tok, pos)+ if chip is not None:+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ boot = [t + self.chip_lo for t in boot_tokens(Chip8(bytes([0x12, 0x00])).cells)]+ k = len(boot) - 1+ self.prefill(boot[:k])+ rows.load_row(0, self.small, k)+ tok[0, 0], pos[0] = boot[k], k+ step(tok, pos)+ kinds = self.cfg.layer_types+ attn = [rows.cache.layers[i] for i in range(self.n_chip, len(kinds))+ if kinds[i] == "full_attention"]+ for L, kc in zip(attn, kconst):+ kc.copy_(L.keys[0, :, int((self.rpos[n] if self.transparent else pos)[0])])+ tok.fill_(boot[k])+ pos.zero_()+ for _ in range(3):+ chip(tok, pos)+ tok.zero_()+ pos.zero_()+ if self.mode == "graphs": # compiled or eager kernels, captured as CUDA graphs+ # the compiled functions own buffers their kernels read (e.g. inductor's copies of+ # closure tensors): they must outlive the graphs, or replays read freed memory+ self.raw[n] = (step, chip)+ graphs = {}+ for view in self.views:+ rows.view = view+ tok.zero_()+ pos.zero_()+ for _ in range(2):+ step(tok, pos)+ g = [self._capture(step, tok, pos), None]+ if chip is not None: # the chip step reads no keys: one graph serves all views+ if not graphs:+ tok.fill_(self.chip_lo)+ for _ in range(2):+ chip(tok, pos)+ g[1] = self._capture(chip, tok, pos)+ else:+ g[1] = next(iter(graphs.values()))[1]+ graphs[view] = g+ rows.view = self.max_len+ tok.zero_()+ pos.zero_()+ self.view_graphs[n] = graphs+ level = self.levels[n] = (rows, tok, pos, *graphs[self.max_len], kconst, _act)+ torch.cuda.synchronize()+ self.ready.add(n)+ return level++ def _view_graph(self, n: int, v: int):+ """[text step, chip step] graphs of batch size ``n`` whose attention reads ``v`` slots."""+ graphs = self.view_graphs[n]+ if v not in graphs:+ rows, tok, pos = self.levels[n][:3]+ rows.view = v+ g = self._capture(self.raw[n][0], tok, pos, warm=False)+ rows.view = self.max_len+ graphs[v] = [g, graphs[self.max_len][1]]+ return graphs[v]++ def _capture(self, fn, tok, pos, warm: bool = True):+ """``fn(tok, pos)`` as a CUDA graph over the level's static buffers; the returned+ callable takes the same (ignored: always those buffers) arguments. ``warm=False``+ records without running it first (safe mid-run: recording runs no kernel)."""+ torch.cuda.synchronize()+ s = torch.cuda.Stream()+ s.wait_stream(torch.cuda.current_stream())+ with torch.cuda.stream(s): # warm up on the capture stream (lazy cuBLAS state)+ for _ in range(2 if warm else 0):+ fn(tok, pos)+ torch.cuda.current_stream().wait_stream(s)+ torch.cuda.synchronize()+ g = torch.cuda.CUDAGraph()+ # thread_local: the prefetch thread may be running a prefill on its own stream+ with torch.cuda.graph(g, pool=self.pool, stream=s, capture_error_mode="thread_local"):+ out = fn(tok, pos)+ self.pool = g.pool()+ self.graphs.append(g)++ def replay(tok_=None, pos_=None):+ g.replay()+ return out++ return replay++ @torch.no_grad()+ def warmup(self, prompt: list[int] | None = None) -> None:+ """Compile (and record the CUDA graphs of) every batch size's steps ahead of time."""+ if prompt is not None:+ self.prefill(prompt)+ for n in self.sizes:+ self._prepare(n)++ # --- the loop -----------------------------------------------------------------------------++ @torch.no_grad()+ def run(self, prompts: list[list[int]], max_new: int, on_done=None,+ hook=None) -> list[list[int]]:+ """Greedy continuations (each ending with its end token, or ``max_new`` long).++ ``hook(i, t) -> (t_used, stop)`` sees every token of prompt ``i`` as it is produced: it+ may replace it (token forcing: ``t_used`` is what is kept and fed back) or end the row+ (``stop``). With a hook the loop reads every step before queueing the next one.+ ``on_done(i, tokens)`` is called as each prompt finishes."""+ for p in prompts:+ assert len(p) + max_new <= self.max_len, (len(p), max_new, self.max_len)+ if not prompts:+ return []+ if not self.small.layers[0].is_conv_states_initialized[0]:+ self.prefill(prompts[0]) # the batched cache copies a prefilled cache's shapes+ self._prepare(self.batch)+ lens = [len(p) + max_new for p in prompts] if self.kv_exact else None+ # record every graph now (all batch sizes, and generate-length views): capturing+ # mid-run would race the prefetch thread's allocations+ for m in self.sizes:+ self._prepare(m)+ for v in sorted(set(lens or [])) if m in self.view_graphs else []:+ self._view_graph(m, min(v, self.max_len))+ pre = _Prefetcher(self, prompts, range(len(prompts)), max(1, self.prefetch), lens)+ try:+ return self._run(prompts, max_new, on_done, pre, hook)+ finally:+ pre.close()++ def _run(self, prompts, max_new, on_done, pre, hook=None):+ out: list[list[int] | None] = [None] * len(prompts)+ queue = deque(range(len(prompts)))+ n = self.batch+ rows, tok, pos, step, chip, _, act = self._prepare(n)+ rpos = self.rpos[n]+ live: list[int | None] = [None] * n # prompt index per row+ gen: list[list[int]] = [[] for _ in range(n)]+ plen = [0] * n # prompt length per row++ def finish(b):+ i = live[b]+ out[i] = gen[b]+ live[b] = None+ if on_done is not None:+ on_done(i, gen[b])++ def admit(b):+ while queue:+ i = queue.popleft()+ cache, t = pre.get(i)+ stop = False+ if hook is not None:+ t, stop = hook(i, t)+ live[b], gen[b] = i, [t]+ if stop or t in self.eos or max_new <= 1:+ pre.release(cache)+ finish(b)+ continue+ rows.load_row(b, cache, len(prompts[i]))+ pre.release(cache)+ tok[b, 0] = t+ pos[b] = len(prompts[i])+ plen[b] = len(prompts[i])+ if self.freeze:+ rpos[b] = sum(not self.chip_lo <= x < self.chip_hi for x in prompts[i])+ self.mark_prompt(n, b, prompts[i])+ return+ tok[b, 0] = 0 # an idle row decodes garbage at slot 0 until it is reused+ pos[b] = 0++ for b in range(n):+ admit(b)+ # The host runs one step ahead: step t+1 is queued before the tokens of step t are read,+ # so the GPU never waits for Python. A row that finishes at step t has its (stale) step+ # t+1 token dropped; a row refilled then starts with step t+2.+ host = [torch.zeros(self.batch, dtype=torch.long, pin_memory=True) for _ in range(2)]+ skip = [0] * n+ flip = 0++ lo, hi = (self.chip_lo, self.chip_hi) if chip is not None else (0, 0)+ budget = 0 # chip steps left before the waiting text rows get a full step++ def launch():+ """One step. With skip_qwen, rows whose input is a chip token take the cheap chip+ step while text rows wait (up to ``chip_per_full`` chip steps per full step); a full+ step advances every row (it is exact for chip rows too)."""+ nonlocal flip, budget+ on = None+ if chip is not None:+ on = [live[b] is not None and lo <= gen[b][-1] < hi for b in range(n)]+ text = any(live[b] is not None and not on[b] for b in range(n))+ if any(on) and (not text or budget > 0):+ budget -= 1+ else:+ on = None+ budget = self.chip_per_full+ step_fn, chip_fn = step, chip+ if n in self.view_graphs and self.kv_exact: # generate's cache length+ v = max((plen[b] + max_new for b in range(n) if live[b] is not None), default=0)+ step_fn, chip_fn = self._view_graph(n, min(max(v, 1), self.max_len))+ elif n in self.view_graphs: # the smallest attention view covering every live row+ need = 1 + max((plen[b] + len(gen[b]) for b in range(n) if live[b] is not None),+ default=0)+ v = next(v for v in self.views if v >= need)+ step_fn, chip_fn = self.view_graphs[n][v]+ self.stats["view_sum"] += v+ if on is not None:+ act.copy_(torch.tensor(on), non_blocking=True)+ nxt = chip_fn(tok, pos)+ tok.copy_(torch.where(act[:, None], nxt[:, None], tok))+ pos.add_(act)+ self.stats["chip_steps"] += 1+ else:+ nxt = step_fn(tok, pos)+ if self.freeze: # the input was text: the next RoPE position moves on+ rpos.add_(self._is_text(tok[:, 0]))+ tok.copy_(nxt[:, None])+ pos.add_(1)+ pos.clamp_(max=self.max_len - 1) # idle rows count up; live rows never get there+ buf = host[flip][:n]+ flip ^= 1+ buf.copy_(nxt, non_blocking=True)+ ev = torch.cuda.Event()+ ev.record()+ self.stats["steps"] += 1+ self.stats["row_steps"] += n+ return ev, buf, on++ def collect(res, inflight: bool):+ ev, buf, on = res+ ev.synchronize()+ for b, t in enumerate(buf.tolist()):+ if skip[b]:+ skip[b] -= 1+ continue+ if live[b] is None or (on is not None and not on[b]):+ continue+ stop = False+ if hook is not None:+ t2, stop = hook(live[b], t)+ if t2 != t: # forced: feed the forced token instead+ tok[b, 0] = t2+ t = t2+ gen[b].append(t)+ self.stats["tokens"] += 1+ if stop or t in self.eos or len(gen[b]) >= max_new:+ finish(b)+ skip[b] = int(inflight)+ admit(b)++ pending = None+ while any(i is not None for i in live):+ cur = launch()+ if chip is not None or hook is not None: # the next step depends on them: no lag+ collect(cur, False)+ cur = None+ elif pending is not None:+ collect(pending, True)+ pending = cur+ if not queue:+ alive = [b for b in range(n) if live[b] is not None]+ smaller = [s for s in self.sizes if s < n and s >= len(alive)]+ if smaller:+ if pending is not None:+ collect(pending, False)+ pending = None+ alive = [b for b in range(n) if live[b] is not None]+ if not alive:+ break+ m = smaller[0]+ rows2, tok2, pos2, step2, chip2, _, act2 = self._prepare(m)+ idx = torch.tensor(alive, device=self.dev)+ rows.take(idx, rows2)+ tok2[: len(alive)].copy_(tok[idx])+ if self.freeze:+ self.rpos[m][: len(alive)].copy_(rpos[idx])+ self.rpos[m][len(alive):] = 0+ pos2[: len(alive)].copy_(pos[idx])+ pos2[len(alive):] = 0+ tok2[len(alive):] = 0+ live = [live[b] for b in alive] + [None] * (m - len(alive))+ gen = [gen[b] for b in alive] + [[] for _ in range(m - len(alive))]+ plen = [plen[b] for b in alive] + [0] * (m - len(alive))+ skip = [0] * m+ n, rows, tok, pos, step, chip, act = m, rows2, tok2, pos2, step2, chip2, act2+ rpos = self.rpos[m]+ if pending is not None:+ collect(pending, False)+ return out+++_DECODERS: dict = {}+++def generate_batch(model, tok, prompts: list[list[int]], max_new: int, batch: int | None = None,+ max_len: int | None = None, **kw) -> list[list[int]]:+ """Greedy continuations of ``prompts`` (token id lists), equal to running each alone.++ ``tok`` is unused (end tokens come from ``model.generation_config``); it is accepted so the+ call reads like the usual ``generate`` helpers. The decoder (and its compiled steps) is+ reused by later calls with the same model, batch, ``max_len`` and options; a call with+ different ones replaces it (one decoder per model, to bound memory).+ """+ del tok+ if not prompts:+ return []+ batch = batch or min(16, len(prompts))+ if max_len is None:+ need = max(len(p) for p in prompts) + max_new+ max_len = -(-need // 1024) * 1024+ sizes = kw.pop("sizes", None) or sorted({batch, *(s for s in (1, 2, 4, 8, 16, 32)+ if s < batch)})+ key = (batch, max_len, tuple(sizes), tuple(sorted(kw.items())))+ old = _DECODERS.get(id(model))+ if old is None or old[0] != key:+ _DECODERS.pop(id(model), None)+ torch.cuda.empty_cache()+ _DECODERS[id(model)] = (key, BatchDecoder(model, batch=batch, max_len=max_len,+ sizes=sizes, **kw))+ return _DECODERS[id(model)][1].run(prompts, max_new)+++def generate_fast(model, prompts: list[list[int]], max_new: int, batch: int | None = None,+ compact_qwen: bool = False, skip_qwen: bool = True, **kw) -> list[list[int]]:+ """Fast greedy decoding of a merged q35 checkpoint: the drop-in for++ [model.generate(torch.tensor([p]), do_sample=False, max_new_tokens=max_new)[0, len(p):]+ .tolist() for p in prompts]++ Each continuation ends with its end token (generation_config.eos_token_id) or is+ ``max_new`` long. On first use it compacts the chip layers in place (``fast.compact``: only+ their nonzero weight blocks are read, every logit stays bit-identical), and with+ ``skip_qwen`` chip-token inputs run only the chip layers (Qwen's layers are exact no-ops+ there). ``compact_qwen=True`` also compacts Qwen's layers and the head: faster at text+ steps, but Qwen's sums round differently (1 of 12 demo answers changed its final period). Prompts are+ decoded ``batch`` at a time with continuous batching (``batch=1`` is the latency setting).+ The compacted model still works with ``generate`` and plain forwards.+ """+ from chipformer.q35.fast import compact++ done = getattr(model, "_q35_compact", None)+ if done is None:+ compact(model, qwen=compact_qwen)+ model._q35_compact = compact_qwen+ elif compact_qwen and not done:+ compact(model, qwen=True)+ model._q35_compact = True+ return generate_batch(model, None, prompts, max_new, batch=batch, skip_qwen=skip_qwen, **kw)diff --git a/chipformer/q35/bridge.py b/chipformer/q35/bridge.pynew file mode 100644index 0000000..fa19c61--- /dev/null+++ b/chipformer/q35/bridge.py@@ -0,0 +1,485 @@+"""The read bridge: the computer prints text that Qwen reads, inside one token stream.++Token space of the v2 circuit: the 8708 chip tokens, then a few *text classes* standing for+Qwen's own text tokens (every other text token is OTHER). In a merged model each Qwen text token+row carries the features of its class, so the circuit sees text but only acts on these.++SCHIP's 00FD ("exit") is redefined as print-and-halt: the record that would execute it prints++ \\n <tool_response> \\n h(mem[I]) h(mem[I+1]) ... h(mem[I+V0-1]) \\n </tool_response> \\n++as real text tokens (h = the low nibble as a hex digit, so BCD digits print as decimals), then+the machine stops and Qwen continues. Nothing in the machine state changes.+"""++from chipformer import spec+from chipformer.ref.emu import fetch+from chipformer.trace import Machine, encode_writes++TEXT_BASE = spec.VOCAB_SIZE # 8708+HEX = list(range(TEXT_BASE, TEXT_BASE + 16)) # '0'..'9', 'a'..'f'+NL, TR_OPEN, TR_CLOSE, TC_OPEN, TC_CLOSE, OTHER, FN_ADD, FN_MUL, FN_SUB = range(+ TEXT_BASE + 16, TEXT_BASE + 25)+# inline arithmetic ("37 \\times 43 =", see inline_scan)+SPACE, EQ, PLUS, MINUS, TIMES, BSL, TIMESW, COMMA, DOLLAR, LPAREN, WS = range(TEXT_BASE + 25,+ TEXT_BASE + 36)+# "$" splits by spelling: the circuit prints "$", "$$" and "." to close an inline result, so those+# classes may have no other spellings; " $", " $$" and "\$" (currency) are their own classes+SDOLLAR, DDOLLAR, SDDOLLAR, CURR, PERIOD = range(TEXT_BASE + 36, TEXT_BASE + 41)+DOLLARS = (DOLLAR, SDOLLAR, DDOLLAR, SDDOLLAR, CURR) # all neutral, all may precede a number+# " -" starts a negative inline result, so its other spellings ("-", "−") are their own class+MINUS2 = TEXT_BASE + 41+# division: "/" (" /", "÷"), the tool name "div" (also "\div"), and "..." after an inexact quotient+DIV, FN_DIV, ELLIPSIS = range(TEXT_BASE + 42, TEXT_BASE + 45)+# signs: "(-" (a sign after the same tokens as "-") and " (-" (always one); ")" after a digit+# (closing a signed number); mod (" mod", " %"; "mod" names the tool and follows "\\" or "\\b");+# gcd ("gcd", " gcd": the tool name and an operator written first, "gcd(A, B) =")+# (for expressions "(-", " (-" and ")" are also stored on the tape: see _codes)+LPMINUS, SLPMINUS, RPAREN, MODOP, FN_MOD, FN_GCD = range(TEXT_BASE + 45, TEXT_BASE + 51)+# a tool response ends with an empty thinking block, as the next assistant turn of Qwen's chat+# template starts: without it Qwen often ends its turn right after the response+THINK_O, THINK_C = range(TEXT_BASE + 51, TEXT_BASE + 53)+# " (" is its own class (the space makes it a safe start for an expression)+SLPAREN = TEXT_BASE + 53+# Expressions (v18): Qwen fuses brackets with operators (")/(", "*(-", "))"), so every token an+# expression is made of is a class of its own, which the circuit stores on the tape as one code;+# the BIOS evaluator expands a code into its symbols (EXPR_SYMS). Input-only: several spellings+# are fine. (expansion, spellings, spaced) - spaced spellings start with a space, which makes them+# a safe start without a safe token before them.+SYMS = "0123456789.+-*/()^" # the evaluator's symbols, codes 0..17+EXPR_CLASSES = [+ (")", [" )"]), ("^", ["^", " ^", "**"]), ("((", ["(("]), ("((", [" (("], True),+ ("(((", ["((("]), ("(((", [" ((("], True),+ ("((-", ["((-"]), ("))", ["))", "})", "}}"]), (")))", [")))"]), (")^", [")^", ")**"]),+ (")+", [")+"]), (")-", [")-"]), (")*", [")*"]), (")/", [")/"]), ("))*", ["))*"]),+ ("))/", ["))/"]), ("))+", ["))+"]), ("))-", ["))-"]), ("))^", ["))^"]),+ ("^(", ["^(", "**(", "^{"]), ("^-", ["^-", "**-"]), ("^(-", ["^{-"]), ("-(", ["-("]),+ ("-(", [" -("], True), ("+(", ["+("]), ("*(", ["*("]), ("/(", ["/("]), ("*(-", ["*(-"]),+ ("*((", ["*(("]), (")*(", [")*(", ")("]), (")/(", [")/("]), (")-(", [")-("]),+ (")+(", [")+("]), ("))/(", ["))/("]), ("))*(", ["))*("]), ("/-", ["/-"]), ("+-", ["+-"]),+ ("*-", ["*-"]), ("--", ["--"]), ("(", ["{"]), (")", ["}"]), ("(", ["$(", " $("], True),+]+EXPR_BASE = TEXT_BASE + 54+EXPR = list(range(EXPR_BASE, EXPR_BASE + len(EXPR_CLASSES)))+DLPAREN = EXPR[-1] # "$(": also opens the math the result closes ("$")+RPDIV = EXPR[[e[1][0] for e in EXPR_CLASSES].index(")/")] # ")/": also the single-op path's "/"+FN_CALC = EXPR_BASE + len(EXPR_CLASSES) # the tool calc(expression)+# "/" is printed (a fraction p/q), so its other spellings (" /", "\u00f7") are their own+# class+DIV2 = FN_CALC + 1+N_TEXT = 56 + len(EXPR_CLASSES)+FUNCS = {"add": FN_ADD, "mul": FN_MUL, "sub": FN_SUB, "div": FN_DIV, "mod": FN_MOD,+ "gcd": FN_GCD} # tool names it answers+PRINT_OP = 0x00FD+# after "</tool_response>\n": "<think>\n\n</think>\n\n", with each "\n\n" printed as two "\n"+# ("\n\n" is WS, a class with many spellings, which the circuit cannot print)+TOOL_TAIL = [THINK_O, NL, NL, THINK_C, NL, NL]+MAX_PRINT = 58 - len(TOOL_TAIL) # digits per print (the write counter tops out at 64)+MAX_ARG = 24 # digits per tool-call argument++# Qwen3.5 token ids of the classes (single tokens in its tokenizer)+QWEN_TEXT = {**{TEXT_BASE + i: c for i, c in enumerate("0123456789abcdef")},+ NL: "\n", TR_OPEN: "<tool_response>", TR_CLOSE: "</tool_response>",+ TC_OPEN: "<tool_call>", TC_CLOSE: "</tool_call>", FN_ADD: "add", FN_MUL: "mul",+ FN_SUB: "sub", SPACE: " ", EQ: " =", PLUS: " +", MINUS: " -", TIMES: " \u00d7",+ BSL: " \\", TIMESW: "times", COMMA: ",", DOLLAR: "$", LPAREN: "(",+ WS: "\n\n", SDOLLAR: " $", DDOLLAR: "$$", SDDOLLAR: " $$", CURR: " \\$",+ PERIOD: ".", MINUS2: "-", DIV: "/", FN_DIV: "div", ELLIPSIS: "...",+ LPMINUS: "(-", SLPMINUS: " (-", RPAREN: ")", MODOP: " mod", FN_MOD: "mod",+ FN_GCD: "gcd", THINK_O: "<think>", THINK_C: "</think>", SLPAREN: " (",+ FN_CALC: "calc",+ **{cls: e[1][0] for cls, e in zip(EXPR, EXPR_CLASSES)}}+# other single tokens Qwen generates for the same thing: it writes <function=add> as "=add" and+# <function=mul> as "=m" "ul", not the way its tokenizer splits the text+# (never for a class the circuit prints: the tied lm_head rows would tie)+# (and <function=gcd> as "=g" "cd")+EXTRA_TEXT = {FN_ADD: ["=add"], FN_MUL: ["ul"], FN_SUB: ["=sub", "ub"], EQ: ["="], PLUS: ["+"],+ MINUS2: [" \u2212", "\u2212"], DIV2: [" /", "\u00f7"], TIMES: ["\u00d7", " *", "*"],+ BSL: ["\\"], MODOP: [" %"], FN_GCD: [" gcd", "cd"], TIMESW: ["cdot"],+ CURR: ["\\$"], FN_CALC: ["alc"],+ WS: ["\n\n\n", "\n\n\n\n", " ", " ", " ", " ", " \n", " \n\n"],+ **{cls: e[1][1:] for cls, e in zip(EXPR, EXPR_CLASSES) if len(e[1]) > 1}}+INLINE_OPS = {PLUS: "add", MINUS: "sub", MINUS2: "sub", TIMES: "mul", DIV: "div", DIV2: "div", RPDIV: "div",+ MODOP: "mod", FN_GCD: "gcd"}+# and after a backslash: TIMESW ("\times", "\cdot") mul, FN_DIV ("\div") div, FN_MOD ("\mod", also+# "\bmod": "b" has the hex digit's class) mod+BSL_OPS = {TIMESW: "mul", FN_DIV: "div", FN_MOD: "mod"}+PREFIX = ("gcd",) # written before A: "gcd(A, B) ="+SIGNED = ("mul", "div", "gcd") # the others only answer A and B without signs+SIGNS = (MINUS, MINUS2, LPMINUS) # a sign when it follows SIGN_PREV (" (-" always is one)+# may precede a first number (and so may chip tokens)+SAFE = (SPACE, NL, WS, LPAREN, SLPAREN, *DOLLARS)+OPISH = (PLUS, MINUS, MINUS2, TIMES, DIV, DIV2, RPDIV, TIMESW, FN_DIV, MODOP, FN_MOD, FN_GCD)+SIGN_PREV = SAFE + (EQ, COMMA) + OPISH+++def _codes():+ """Tape code of every class an expression is made of, and the expansions of the codes."""+ single = {**{HEX[d]: d for d in range(10)}, PERIOD: 10, PLUS: 11, MINUS: 12, MINUS2: 12,+ TIMES: 13, TIMESW: 13, DIV: 14, DIV2: 14, FN_DIV: 14, LPAREN: 15, SLPAREN: 15, RPAREN: 16}+ multi = ["(-"] # v16's sign classes LPMINUS / SLPMINUS+ single[LPMINUS] = single[SLPMINUS] = len(SYMS)+ for cls, e in zip(EXPR, EXPR_CLASSES):+ if len(e[0]) == 1:+ single[cls] = SYMS.index(e[0])+ else:+ if e[0] not in multi:+ multi.append(e[0])+ single[cls] = len(SYMS) + multi.index(e[0])+ return single, multi+++TAPE_CODE, MULTI = _codes() # class -> code; code len(SYMS) + i expands to MULTI[i]+EXPANSION = {**{i: s for i, s in enumerate(SYMS)}, **{len(SYMS) + i: s for i, s in enumerate(MULTI)}}+SPACED = {SLPAREN, SLPMINUS} | {cls for cls, e in zip(EXPR, EXPR_CLASSES) if len(e) > 2}+# may start an expression: a number, a bracket or a minus+STARTS = {*HEX[:10], MINUS, MINUS2, LPAREN, SLPAREN, LPMINUS, SLPMINUS} | {+ cls for cls, e in zip(EXPR, EXPR_CLASSES) if e[0][0] in "(-"}+TAPE_MAX = 48 # tape cells (the CALL record writes TAPE_MAX + 3 <= JMAX)+EXPR_MIN_DIGITS = 4 # an expression fires with at least this many digits+CLASS_OF = {**{s: cls for cls, s in QWEN_TEXT.items()},+ **{s: cls for cls, ss in EXTRA_TEXT.items() for s in ss}} # Qwen token -> class+assert len(CLASS_OF) == len(QWEN_TEXT) + sum(map(len, EXTRA_TEXT.values())), "a token in 2 classes"+# Inline calls need at least this many digits in A and B together. Qwen gets small ones right,+# and right after a trace its conv window holds chip tokens instead of the "=", so a short+# printed result can read to it like the start of a new expression. Qwen also adds and+# subtracts numbers of a few digits reliably, so those stay its own.+INLINE_MIN_DIGITS = {"mul": 4, "add": 7, "sub": 7, "div": 4, "mod": 4, "gcd": 4}+++def is_text(t: int) -> bool:+ return t >= TEXT_BASE+++def readout(cells: list[int]) -> list[int]:+ from chipformer.q35.lib import bcd++ i, n = cells[spec.I], min(cells[spec.V], MAX_PRINT)+ # 10 and 11 print as "." and "..." (a quotient's decimals), so no hex digits a / b+ shown = {**dict(enumerate(HEX)), 10: PERIOD, 11: ELLIPSIS, 12: DIV}+ digits = [shown[cells[(i + k) & 0xFFF] & 15] for k in range(n)]+ mode = cells[bcd.MODE]+ if mode & 1: # an inline call: " digits" (" -digits" below 0) and a closer (the math's+ close = DDOLLAR if mode & 4 else DOLLAR if mode & 2 else PERIOD # "$"/"$$", else ".")+ return [MINUS if mode & 8 else SPACE, *digits, close] if digits else [SPACE]+ # a tool call: " -" instead of the newline before a negative result+ return [NL, TR_OPEN, MINUS if mode & 8 and digits else NL, *digits, NL, TR_CLOSE, NL,+ *TOOL_TAIL]+++def prev_safe(prev) -> bool:+ """May a first number (or an expression) start right after this token?"""+ return prev is None or not is_text(prev) or prev in (SPACE, NL, LPAREN, SLPAREN, WS, *DOLLARS)+++def opener_kind(prev, prev2) -> int:+ """The math a first number sits in: 1 right after "$" (maybe and a space), 2 after "$$"."""++ def kind(x):+ return 1 if x in (DOLLAR, SDOLLAR) else 2 if x in (DDOLLAR, SDDOLLAR) else 0++ return kind(prev) or (kind(prev2) if prev == SPACE else 0)+++class InlineScan:+ """Which "A op B =" the circuit answers: the reference for its capture logic.++ Text since the last *breaker* token (anything but digits, spaces, operators, signs, "=",+ "$", "(", a backslash, or a comma or ")" right after a digit) must be exactly: digits, one+ operator, digits, then "=" (for gcd: the operator first, "gcd(A, B) ="), with at least+ INLINE_MIN_DIGITS[op] digits in the two numbers together. A comma between digits joins them+ (1,234). The first number must start after a space, newline, other whitespace, "$", "(" or+ a chip token, so "2.5 * 3 =" and "x^2 + 1 =" are left alone.++ A sign is "-" (any spelling) or "(-" right after one of SIGN_PREV, or " (-"; it is no+ operator, and the number right after it is negative. mul, div and gcd take signed numbers+ (the result is negative when exactly one of A, B is, for mul and div); the others do not+ fire on them. Chip tokens are breakers too. Fed every token of the stream, in order.+ """++ def __init__(self):+ self.prev = self.prev2 = None # None: no token yet (treated like a chip token)+ self.p = self.nops = self.d = 0+ self.lens = {1: 0, 2: 0}+ self.neg = {1: False, 2: False}+ self.digits = {}+ self.op, self.op_ok, self.a_safe, self.in_call = None, False, False, False+ self.opener = 0 # what the first number follows: 1 "$", 2 "$$" (maybe with a space)++ def negative(self) -> bool:+ """Whether the result of the call being made prints with a minus (MODE bit 8)."""+ return self.op in ("mul", "div") and self.neg[1] != self.neg[2]++ def mode(self) -> int:+ """The CALL record's MODE cell for an inline call: 1 | 2 inside "$" | 4 inside "$$"+ | 8 negative."""+ return (1 | (2 if self.opener else 0) | (4 if self.opener == 2 else 0)+ | (8 if self.negative() else 0))++ def feed(self, t: int):+ """Returns (op, a, b) if the circuit takes over after token t, else None."""+ prev, prev2 = self.prev, self.prev2+ self.prev, self.prev2 = t, prev+ text = is_text(t)+ dig = text and t in HEX[:10]++ def isd(x):+ return x is not None and x in HEX[:10]++ def safe(x):+ return x is not None and (not is_text(x) or x in SAFE)++ def sign_prev(x):+ return safe(x) or x in SIGN_PREV++ if t in (TC_OPEN, TC_CLOSE) or t in (spec.STEP, spec.BOOT):+ self.in_call = t == TC_OPEN+ if t == TC_OPEN:+ self.op = None # a tool call needs its own function name+ cont = dig and (isd(prev) or (prev == COMMA and isd(prev2)))+ run_start = dig and not cont+ neg = run_start and ((prev in SIGNS and sign_prev(prev2)) or prev == SLPMINUS)+ bs_op = t in BSL_OPS and (prev == BSL or (t == FN_MOD and prev == HEX[11]+ and prev2 == BSL))+ sign_minus = t in (MINUS, MINUS2) and sign_prev(prev)+ op_tok = (t in INLINE_OPS or bs_op) and not sign_minus+ neutral = text and (dig or op_tok or sign_minus+ or t in (SPACE, LPAREN, SLPAREN, BSL, EQ, LPMINUS, SLPMINUS, *DOLLARS)+ or (t in (COMMA, RPAREN) and isd(prev))+ or (t == HEX[11] and prev == BSL))+ brk = not neutral+ after_eq = prev == EQ and not brk and not self.in_call # a fresh window after "="+ if t == TC_OPEN or (brk and not self.in_call) or after_eq:+ self.p = 0+ elif run_start:+ self.p += 1+ if self.p == 1:+ self.a_safe = safe(prev) or neg++ def kind(x):+ return 1 if x in (DOLLAR, SDOLLAR) else 2 if x in (DDOLLAR, SDDOLLAR) else 0++ after = prev in (SPACE, *SIGNS, SLPMINUS)+ self.opener = kind(prev) or (kind(prev2) if after else 0)+ if self.p in (1, 2):+ self.neg[self.p] = neg+ if run_start:+ self.d = 0+ elif cont:+ self.d += 1+ if brk or after_eq:+ self.nops = 0+ elif op_tok:+ self.nops += 1+ if dig and self.p in (1, 2):+ self.lens[self.p] = self.d + 1 if self.d < MAX_ARG else 0+ if self.d < MAX_ARG:+ self.digits[self.p, self.d] = HEX.index(t)+ fn = {cls: name for name, cls in FUNCS.items()}.get(t)+ if fn and self.in_call:+ self.op, self.op_ok = fn, self.p == 1+ if op_tok and not self.in_call:+ self.op = BSL_OPS[t] if bs_op else INLINE_OPS[t]+ self.op_ok = self.p == (0 if self.op in PREFIX else 1)+ if (t == EQ and not self.in_call and self.p == 2 and self.nops == 1 and self.op_ok+ and self.a_safe and self.lens[1] and self.lens[2] and self.op+ and self.lens[1] + self.lens[2] >= INLINE_MIN_DIGITS[self.op]+ and (self.op in SIGNED or not (self.neg[1] or self.neg[2]))):+ a = "".join(str(self.digits[1, w]) for w in range(self.lens[1]))+ b = "".join(str(self.digits[2, w]) for w in range(self.lens[2]))+ return self.op, a, b+ return None+++class ExprScan:+ """Which whole expressions the circuit evaluates: the reference for its tape capture.++ Outside tool calls, every token of an expression is stored on the tape (as its TAPE_CODE)+ since the last *reset*: a breaker (anything but stored tokens, spaces, "$", a backslash or a+ comma right after a digit) or "=". Stored: digits, a "." right after a digit, operators+ (also "\\times", "\\cdot", "\\div"), brackets and the fused EXPR classes. At "=" it fires if+ the window starts well (a number, a bracket or a minus, after a safe token or with a space+ of its own), has a bracket, power or "." or at least two operator tokens, has at least+ EXPR_MIN_DIGITS digits and at most TAPE_MAX tokens, and the single-op path does not fire.+ """++ def __init__(self):+ self.prev = self.prev2 = None+ self.in_call = self.calc = False+ self.reset()+ self.flags = None # (start, opener) of the window, as of its last stored token+ self.tape = []+ self.n = 0 # the tape's length (0 after <tool_call>)++ def reset(self):+ self.count = self.digits = self.ops = 0+ self.special = False++ def stored(self, t, prev) -> bool:+ if t in TAPE_CODE and t not in (TIMESW, FN_DIV, PERIOD):+ return True+ return (t in (TIMESW, FN_DIV) and prev == BSL) or (t == PERIOD and prev in HEX[:10])++ def feed(self, t: int, fast_fired: bool = False):+ """Returns (tape codes, mode) if the circuit evaluates an expression at token t."""+ prev, prev2 = self.prev, self.prev2+ self.prev, self.prev2 = t, prev+ # a tool call calc(expression): its tape is every stored token since <tool_call>+ tool = t == TC_CLOSE and self.calc and 1 <= self.n+ if t == TC_OPEN:+ self.calc, self.n = False, 0+ if t == FN_CALC:+ self.calc = True+ if t in (TC_OPEN, TC_CLOSE) or t in (spec.STEP, spec.BOOT):+ self.in_call = t == TC_OPEN+ st = is_text(t) and self.stored(t, prev)+ neutral = st or (is_text(t) and (t in (SPACE, BSL, *DOLLARS)+ or (t == COMMA and prev in HEX[:10])))+ brk = not neutral and t != EQ+ # a "." that ends a number ("3. $11"): a space or "$" after it breaks the window too+ if t in (SPACE, *DOLLARS) and prev == PERIOD and prev2 in HEX[:10]:+ brk = True+ if (brk and not self.in_call) or (t == EQ and not self.in_call) or t == TC_OPEN:+ self.reset()+ if brk:+ self.flags = None+ if st:+ self.count += 1+ if self.count == 1:+ ok = t in STARTS and (prev_safe(prev) or t in SPACED)+ self.start = (ok, 1 if t == DLPAREN else opener_kind(prev, prev2))+ self.digits += t in HEX[:10]+ ops = EXPANSION[TAPE_CODE[t]]+ self.ops += any(ch in "+-*/^" for ch in ops)+ self.special |= any(ch in "()^." for ch in ops)+ self.n = self.count if self.count <= TAPE_MAX else 0+ if self.count <= TAPE_MAX:+ del self.tape[self.count - 1:]+ self.tape.append(TAPE_CODE[t])+ self.flags = (self.start[0] and self.count <= TAPE_MAX+ and (self.special or self.ops >= 2)+ and self.digits >= EXPR_MIN_DIGITS, self.start[1], self.count)+ if tool:+ return self.tape[:self.n], 0+ if t == EQ and not self.in_call and not fast_fired and self.flags and self.flags[0]:+ opener, n = self.flags[1], self.flags[2]+ return self.tape[:n], 1 | (2 if opener else 0) | (4 if opener == 2 else 0)+ return None+++class BridgeMachine(Machine):+ """Reference trace for the v2 circuit: machine records, prints, and Qwen's text."""++ def __init__(self, rom: bytes, quirks=None):+ super().__init__(rom, quirks)+ self.halted = False+ self.scan = InlineScan()+ self.escan = ExprScan()+ for t in self.tokens:+ self.escan.feed(t, self.scan.feed(t) is not None)++ def _emit(self, tokens: list[int], generated: bool) -> None:+ self.fired = self.efired = None+ for t in tokens:+ self.fired = self.scan.feed(t & 0xFFFF) # (a token may carry a Qwen id above)+ self.efired = self.escan.feed(t & 0xFFFF, self.fired is not None)+ super()._emit(tokens, generated)++ def qwen(self, classes: list[int]) -> list[tuple]:+ """Qwen writes these text tokens one at a time; wherever the circuit answers an inline+ "A op B =" or an expression, it takes over (STEP, the CALL record, the routine, the+ print). Expressions are listed as ("eval", tape codes)."""+ calls = []+ for t in classes:+ self._emit([t], False)+ if self.fired:+ calls.append(self.fired)+ self._call(*self.fired, mode=self.scan.mode())+ elif self.efired:+ tape, mode = self.efired+ calls.append(("eval", tuple(tape)))+ self._ecall(tape, mode)+ else:+ continue+ while not self.halted:+ self.step()+ return calls++ def step(self):+ if self.halted:+ raise RuntimeError("machine halted")+ op = fetch(self.emu.cells)+ if op == PRINT_OP:+ self._emit(readout(self.emu.cells), True)+ self.halted = True+ return []+ from chipformer.q35.lib.macros import macro_writes++ w = macro_writes(self.emu.cells, op)+ if w is not None:+ self.emu.apply(w)+ self._emit(encode_writes(w) + [spec.STEP], True)+ return w+ return super().step()++ def text(self, n: int, cls: int = OTHER) -> None:+ """Qwen writes n text tokens (the circuit must hand over)."""+ self._emit([cls] * n, False)++ def tool_call(self, op: str, a: str, b: str) -> None:+ """Qwen calls a library routine with its native syntax; the circuit then writes the+ CALL record (the arguments into RAM, PC = the routine) and the machine runs it."""++ def lit(n):+ return [OTHER] * n++ def num(s): # a minus is its own token: Qwen writes "\n-1418261\n"+ return [MINUS2] * s.startswith("-") + [HEX[int(ch)] for ch in s.lstrip("-")]++ text = ([TC_OPEN, NL] + lit(2) + [FUNCS[op]] + lit(1) + [NL] + lit(3) + [NL] + num(a)+ + [NL] + lit(3) + [NL] + lit(3) + [NL] + num(b) + [NL] + lit(3) + [NL] + lit(3)+ + [NL, TC_CLOSE])+ self._emit(text, False)+ self._call(op, a.lstrip("-"), b.lstrip("-"), mode=8 * self.scan.negative())++ def _ecall(self, tape: list[int], mode: int) -> None:+ from chipformer.q35.lib import bcd++ w = bcd.eval_writes(tape, mode)+ self.emu.apply(w)+ self._emit([spec.STEP] * bool(mode) + encode_writes(w) + [spec.STEP], True)+ self.halted = False++ def calc_call(self, classes: list[int]) -> tuple:+ """Qwen calls calc(expression) with its native syntax (the expression's text classes);+ the circuit writes the CALL record and the machine runs eval."""+ text = ([TC_OPEN, NL, OTHER, OTHER, EQ, FN_CALC, OTHER, NL, OTHER, OTHER, EQ, OTHER,+ OTHER, NL] + list(classes) + [NL, OTHER, OTHER, OTHER, NL, OTHER, OTHER, OTHER,+ NL, TC_CLOSE])+ self._emit(text, False)+ fired = self.efired+ assert fired and fired[1] == 0, "calc did not fire"+ self._ecall(*fired)+ return ("eval", tuple(fired[0]))++ def _call(self, op: str, a: str, b: str, mode: int) -> None:+ from chipformer.q35.lib import bcd++ w = bcd.call_writes(op, a, b, mode)+ self.emu.apply(w)+ self._emit([spec.STEP] * (mode & 1) + encode_writes(w) + [spec.STEP], True)+ self.halted = False+++def owner_expected(m: BridgeMachine) -> list[int | None]:+ """1 where the circuit must predict the next token, 0 where Qwen does, None for host-only."""+ out = []+ toks, gen = m.tokens, m.generated+ for p, t in enumerate(toks):+ if not is_text(t):+ out.append(1)+ elif p + 1 < len(toks) and gen[p + 1]:+ out.append(1) # inside a print+ else:+ out.append(0)+ return outdiff --git a/chipformer/q35/check.py b/chipformer/q35/check.pynew file mode 100644index 0000000..050294d--- /dev/null+++ b/chipformer/q35/check.py@@ -0,0 +1,51 @@+"""Differential testing of the q35 circuit against the reference trace, token by token."""++import numpy as np++from chipformer import spec+from chipformer.check import Mismatch+from chipformer.q35 import ir+from chipformer.q35.chip8_q35 import Chip8Q35, decode_code+from chipformer.trace import Machine+++def predictions(c: Chip8Q35, tokens: list[int], check: bool = True) -> np.ndarray:+ """Next-token prediction at every position, from the circuit's output bits."""+ m = ir.evaluate(c, tokens, check)+ bits = np.stack([ir.ev(b, m) for b in c.next_bits], 1)+ assert np.isin(bits, [0.0, 1.0]).all(), "output bits must be 0/1"+ return np.array([decode_code(row) for row in bits.astype(int).tolist()]), m+++def check(c: Chip8Q35, m: Machine, limit: int = 10, exact: bool = True):+ """(number of checked predictions, first mismatches)."""+ pred, _ = predictions(c, m.tokens, exact)+ want = np.array(m.tokens[1:] + [-1])+ gen = np.array(m.generated[1:] + [False])+ bad = np.nonzero(gen & (pred != want))[0]+ starts = [p for p, _ in getattr(m, "ops", [])]+ out = []+ for p in bad[:limit]:+ i = np.searchsorted(starts, p + 1, side="right") - 1+ op = m.ops[i][1] if i >= 0 and starts else "?"+ out.append(Mismatch(int(p), int(want[p]), float(pred[p]), op))+ return int(gen.sum()), out+++def describe(mm: list[Mismatch]) -> str:+ return "\n".join(f" {x}" for x in mm) or " none"+++__all__ = ["check", "describe", "predictions", "spec"]+++def check_bridge(c: Chip8Q35, tokens: list[int], generated: list[bool], owner_want, limit=10):+ """Predictions at generated positions, and the owner bit everywhere it is defined."""+ pred, m = predictions(c, tokens)+ owner = ir.ev(c.owner, m)+ want = np.array(tokens[1:] + [-1])+ gen = np.array(generated[1:] + [False])+ bad = [(int(p), int(want[p]), int(pred[p])) for p in np.nonzero(gen & (pred != want))[0]]+ bad_owner = [(p, float(owner[p]), w) for p, w in enumerate(owner_want)+ if w is not None and owner[p] != w]+ return int(gen.sum()), bad[:limit], bad_owner[:limit]diff --git a/chipformer/q35/chip8_q35.py b/chipformer/q35/chip8_q35.pynew file mode 100644index 0000000..021e9a6--- /dev/null+++ b/chipformer/q35/chip8_q35.py@@ -0,0 +1,1354 @@+"""CHIP-8 as a q35 circuit: the same machine as chipformer/chip8.py, built for Qwen3.5 layers.++Same structure as the original: every position computes the instruction context from the state+it reads (meaningful at STEP/BOOT positions), a DeltaNet slot carries that context from the+latest record start to the rest of the record, and a per-position micro-step picks the output.+What changes is how state is read:++ memory the delta rule with one-hot keys overwrites a state row, so a cell's row always+ holds its latest write; a read is a one-hot query. RAM is two layers of 16 heads+ (head = address bits 3:0, so PC and PC+1 always land in different heads).+ registers one head, one row per register in its own column block, read all at once.+ positions none: the write index j comes from a head whose row halves at every VAL token+ and resets at STEP/BOOT (x = 2^-j), and "latest STEP/TICK" style lookups are+ rows overwritten at those tokens.++All arithmetic is on bits, one-hot nibbles and small integers, so it is exact in bf16.+"""++from fractions import Fraction++from chipformer import spec+from chipformer.dsl.graph import Expr+from chipformer.q35 import bridge as B+from chipformer.q35.ir import A_BIT, A_REF, REF_COLS, Chan, Target+from chipformer.q35.ir import K as ir_K+from chipformer.q35.lib import bcd as bios+from chipformer.q35.prims import Circ, cat, from_bits, onehot_bits++JMAX = 64 # write indices above this all read as JMAX + 1+NCODE = 15 # output code: [is_addr, is_ctrl, payload bits 0..12]+CODE_AMP = 128+KAPPA = 2**-20 # output code after the exit: tiny, so it never disturbs Qwen's norms+TAU_TEXT = 2.0**-40+V, VF = spec.V, spec.V + 0xF+TEXT_CODE = 16 # text classes are ctrl codes with payload 16 + class (STEP/TICK/BOOT: 0..2)+++def token_code(t: int) -> list[int]:+ """0/1 code of a token; the embedding stores CODE_AMP * (2 * bit - 1)."""+ if B.is_text(t):+ kind, payload = (0, 1), TEXT_CODE + t - B.TEXT_BASE+ elif spec.is_val(t):+ kind, payload = (0, 0), t - spec.VAL_BASE+ elif spec.is_addr(t):+ kind, payload = (1, 0), t - spec.ADDR_BASE+ else:+ kind, payload = (0, 1), t - spec.STEP+ return [*kind] + [payload >> i & 1 for i in range(13)]+++def decode_code(bits: list[int]) -> int:+ payload = sum(b << i for i, b in enumerate(bits[2:]))+ if bits[1]:+ return B.TEXT_BASE + payload - TEXT_CODE if payload >= TEXT_CODE else spec.STEP + payload+ return (spec.ADDR_BASE if bits[0] else spec.VAL_BASE) + payload+++class Chip8Q35(Circ):+ def __init__(self, target: Target | None = None) -> None:+ super().__init__(target or Target())+ f = self.token_feature+ isv, isa = spec.is_val, spec.is_addr+ self.is_val = f("is_val", lambda t: int(isv(t)))+ self.is_addr = f("is_addr", lambda t: int(isa(t)))+ self.is_step = f("is_step", lambda t: int(t == spec.STEP))+ self.is_tick = f("is_tick", lambda t: int(t == spec.TICK))+ self.is_boot = f("is_boot", lambda t: int(t == spec.BOOT))+ self.vb = [f(f"vb{i}", lambda t, i=i: (t - spec.VAL_BASE) >> i & 1 if isv(t) else 0)+ for i in range(12)]+ # an ADDR token's address as 13 bits: write keys read them at the next token through a+ # conv tap, so they must be visible on text tokens (overlay.py); bits keep that to 13 dims+ self.ab = [f(f"ab{i}", lambda t, i=i: int(isa(t) and (t - spec.ADDR_BASE) >> i & 1))+ for i in range(13)]+ self.is_cls_addr = f("is_cls_addr", lambda t: int(t == spec.addr_token(spec.CLS)))+ self.is_text = f("is_text", lambda t: int(B.is_text(t)))+ self.mu = f("mu", lambda t: int(not B.is_text(t))) # chip token: Qwen's layers mute on it+ self.text = {cls: f(f"text{cls - B.TEXT_BASE}", lambda t, cls=cls: int(t == cls))+ for cls in range(B.TEXT_BASE, B.TEXT_BASE + B.N_TEXT)}+ self.code = [f(f"code{i}", lambda t, i=i: 0 if t == B.OTHER else+ CODE_AMP * (2 * token_code(t)[i] - 1)) for i in range(NCODE)]+ # tau: -1 on chip tokens (cancelled by the last layer), a tiny value on text tokens.+ # The final norm amplifies it, pushing the chip tokens' logits far down wherever Qwen+ # speaks.+ self.tau = f("tau", lambda t: TAU_TEXT if B.is_text(t) else -1)+ self.rs = self.is_step + self.is_boot # record start++ def bit_lit(self, i: int, v: int):+ """1 at an ADDR token whose address bit i is v; 0 at every other token (so a sum of+ these, like one-hots, is 0 away from ADDR tokens)."""+ return self.ab[i] if v else self.is_addr - self.ab[i]++ def addr_lits(self, a: int, bits=range(13)) -> list:+ """0/1 literals that are all 1 exactly at an ADDR a token (and all 0 at other tokens);+ ``bits``: only these address bits."""+ return [self.bit_lit(i, a >> i & 1) for i in bits]++ def write_key(self, a: int) -> Chan:+ """Key pre-activation (AND units) for a write to cell a, seen at its VAL token."""+ lits = self.addr_lits(a)+ return Chan(sum(lits, Expr()) - (len(lits) - 1), (0, 1, 0, 0))++ def bank(self, name: str, cells: dict[int, int], query: dict, nbits: int, a=0) -> list:+ """One head of trace-written cells (cell -> slot); returns the read bits."""+ keys = {s: self.write_key(c) for c, s in cells.items()}+ data = {2 + i: self.vb[i] for i in range(nbits)}+ return self.slot_head(name, keys, query, data, self.one, a)++ def ram_head(self, name: str, h: int, query: dict, gates: tuple, acc: list | None,+ hi11=None, half: int | None = None) -> list:+ """Key head h of the 4 KB RAM (cells with low nibble h); its read bits are added into+ ``acc`` (None: new dims), so all RAM heads share 8 dims: a query matches at most one+ head, the others read exactly 0.++ With two value heads per key head (Qwen3.5-4B, ``hi11`` given) row s holds cell+ (s >> 4 << 8) | ((s & 15) << 4) | h of both halves of the RAM, one per value head: the+ write gate picks the value head by bit 11 of the address (``hi11``: 1 when the ADDR token+ before this VAL has it), and on a read the output gate gates[x] passes value head x+ only. Without (``half`` given) the head holds one half and the query selects it."""+ one, K_ = self.one, ir_K+ bits = [b for b in range(13) if b != 11 or half is not None] # both halves: not bit 11+ cell = [(half or 0) << 11 | (s >> 4 << 8) | ((s & 15) << 4) | h for s in range(128)]+ keys = {s: Chan(sum(self.addr_lits(cell[s], bits), Expr()) - (len(bits) - 1),+ (0, 1, 0, 0)) for s in range(128)}++ def sel(c):+ if isinstance(c, Chan):+ return Chan(c.expr * (2 * K_) - one * K_, c.taps)+ return Chan(c * (2 * K_) - one * K_)++ v = {2 + i: Chan(self.vb[i] * A_BIT) for i in range(8)}+ for col in REF_COLS:+ v[col] = Chan(one * A_REF)+ cols = [2 + i for i in range(8)]+ outs = [(col, 2.0**7) for col in cols]+ q = {s: sel(e) for s, e in query.items()}+ k = {s: sel(e) for s, e in keys.items()}+ z0 = {col: K_ * gates[0] for col in cols}+ if hi11 is None:+ return self.head(name, q, k, v, 0, z0, outs, into=acc)+ vx = [{"v": v, "a": 0, "z": {col: K_ * gates[1] for col in cols}, "outs": outs,+ "beta": hi11, "into": "main"}]+ return self.head(name, q, k, v, 0, z0, outs, beta=one - hi11, into=acc, vx=vx)[:8]+++def build(target: Target | None = None, store=None, schedule: bool = True) -> Chip8Q35:+ """The circuit; with ``store`` (a store.Vocab, or True for the default tokenizer) it also+ holds the object store (store.py, store_q35.py). ``schedule``: place the nodes again by+ list scheduling once built (ir.Graph.reschedule), which saves layers."""+ c = Chip8Q35(target)+ one, rs = c.one, c.rs+ if store is True:+ from chipformer.q35.store import Vocab++ store = Vocab()++ # ------------------------------------------------------------ registers (one head)+ regs = [("PC", spec.PC, 12), ("I", spec.I, 12), ("SP", spec.SP, 4), ("DT", spec.DT, 8),+ ("ST", spec.ST, 8), ("RNG", spec.RNG, 12), ("V0", V, 8)]+ regs += [(f"key{k:X}", spec.KEYS + k, 1) for k in range(16)]+ regs += [(f"q.{q}", spec.QUIRKS + i, 1) for i, q in enumerate(spec.QUIRK_NAMES)]+ regs += [("LA", bios.LEN, 5), ("LB", bios.LEN + 1, 5)] # bignum argument lengths+ regs += [("MODE", bios.MODE, 4)] # inline call, inside "$", inside "$$", negative+ regs += [("DS", bios.DSTOP, 4)] # where BDIV / BADB stop: t = LA + DS+ keys, data, col, blocks = {127: Chan(c.is_boot)}, {}, 2, {}+ for slot, (name, cell, nb) in enumerate(regs):+ keys[slot] = c.write_key(cell)+ lits = c.addr_lits(cell)+ blocks[name] = list(range(col, col + nb))+ for i in range(nb):+ # the VAL bit (this token) AND the address (previous token), over both conv taps+ data[col] = Chan(c.vb[i] + sum(lits, Expr()) - one * Fraction(len(lits), 2), (1, 1, 0, 0))+ col += 1+ out = c.slot_head("regs", keys, {s: one for s in keys}, data, Chan(c.is_boot))+ by_col = dict(zip(sorted(data), out))+ R = {name: [by_col[k] for k in cols] for name, cols in blocks.items()}+ pcb, ib, spb, dtb, stb, rngb, v0b = (R[k] for k in ("PC", "I", "SP", "DT", "ST", "RNG", "V0"))+ keyb = [R[f"key{k:X}"][0] for k in range(16)]+ lab, lbb = R["LA"], R["LB"]+ mode, mode_d, mode_dd, mode_neg = R["MODE"]+ q = {name: R[f"q.{name}"][0] for name in spec.QUIRK_NAMES}++ # ------------------------------------------------------------ micro-step counter+ # j = VAL tokens (inside a record) or text tokens (inside a print) since the record start;+ # a CALL record starts at the </tool_call> token+ tco, tcc = c.text[B.TC_OPEN], c.text[B.TC_CLOSE]+ rsc = rs + tcc+ (x,) = c.slot_head("counter", {0: rsc, 1: one - rsc}, {0: one - rsc, 1: one - rsc},+ {2: rsc}, one - rsc, 110 * (c.is_val + c.is_text - tcc) + 238 * rsc,+ tiny=2.0**-90)+ cnt = [c.node([(1, one - rsc, 7 * one - 8 * 2**m * x),+ (-1, one - rsc, 6 * one - 8 * 2**m * x)], f"j>{m}")+ for m in range(JMAX + 1)] # cnt[m] = [j > m]+ j = sum(cnt, Expr())++ def jeq(v): # [j == v], a 0/1 linear form+ return (one if v == 0 else cnt[v - 1]) - (cnt[v] if v <= JMAX else 0)++ (in_tick,) = c.slot_head("in_tick", {0: rs + c.is_tick}, {0: one}, {2: c.is_tick},+ rs + c.is_tick)++ # ------------------------------------------------------------ argument capture+ # Tool calls: inside <tool_call> ... </tool_call> the circuit keeps the routine's name and+ # the digits of each argument (runs of digit tokens, in order) for the CALL record.+ # Inline: outside tool calls it keeps the same for "A op B" since the last breaker token, and+ # answers at "=" (bridge.InlineScan is the reference).+ T = c.text+ is_dig = sum((T[B.HEX[v]] for v in range(10)), Expr())+ dvb = [sum((T[B.HEX[v]] for v in range(10) if v >> i & 1), Expr()) for i in range(4)]+ fns = {op: T[cls] for op, cls in B.FUNCS.items()}+ fn_any = sum(fns.values(), Expr())+ (in_call,) = c.slot_head("in_call", {0: tco + tcc + rs}, {0: one}, {2: tco}, tco + tcc + rs)+ st = None+ s_gate = [] # literals that hold everything else off during a store answer+ if store is not None:+ from chipformer.q35 import store_q35++ st = store_q35.capture(c, store, in_call, tco, tcc)+ s_gate = [one - st.S]+ c.store = st+ dollars = sum((T[d_] for d_ in B.DOLLARS), Expr())+ safe = (T[B.SPACE] + T[B.NL] + T[B.WS] + dollars + T[B.LPAREN] + T[B.SLPAREN]+ + c.mu) # may precede a first number+ open1, open2 = T[B.DOLLAR] + T[B.SDOLLAR], T[B.DDOLLAR] + T[B.SDDOLLAR] # "$", "$$"+ minus = T[B.MINUS] + T[B.MINUS2]+ signs = minus + T[B.LPMINUS] # signs after a token of SIGN_PREV (" (-" always is one)+ sign_prev = (safe + T[B.EQ] + T[B.COMMA]+ + sum((T[x] for x in B.OPISH), Expr())) # (one-hot classes: 0/1)+ prev_cols = {2: Chan(is_dig, (0, 1, 0, 0)), 3: Chan(is_dig, (0, 0, 1, 0)),+ 4: Chan(T[B.COMMA], (0, 1, 0, 0)), 5: Chan(T[B.BSL], (0, 1, 0, 0)),+ 6: Chan(safe, (0, 1, 0, 0)), 7: Chan(c.is_text, (0, 1, 0, 0)),+ 8: Chan(T[B.EQ], (0, 1, 0, 0)), 9: Chan(open1, (0, 1, 0, 0)),+ 10: Chan(open2, (0, 1, 0, 0)), 11: Chan(open1, (0, 0, 1, 0)),+ 12: Chan(open2, (0, 0, 1, 0)), 13: Chan(T[B.SPACE], (0, 1, 0, 0)),+ 14: Chan(signs, (0, 1, 0, 0)), 15: Chan(T[B.SLPMINUS], (0, 1, 0, 0)),+ 16: Chan(sign_prev, (0, 1, 0, 0)), 17: Chan(sign_prev, (0, 0, 1, 0)),+ 18: Chan(T[B.HEX[11]], (0, 1, 0, 0)), 19: Chan(T[B.BSL], (0, 0, 1, 0)),+ 20: Chan(T[B.PERIOD], (0, 1, 0, 0))}+ paired = c.target.value_heads >= 2 * c.target.key_heads # RAM halves share key heads+ if paired: # bit 11 of the ADDR token before a VAL: which RAM half the VAL writes+ prev_cols[21] = Chan(c.ab[11], (0, 1, 0, 0))+ (prev_dig, prev2_dig, prev_cm, prev_bs, prev_safe, prev_text, prev_eq, prev_o1, prev_o2,+ prev2_o1, prev2_o2, prev_sp, prev_sg, prev_slm, prev_spv, prev2_spv, prev_b,+ prev2_bs, prev_pt, *prev_hi11) = c.slot_head("prev", {0: one}, {0: one}, prev_cols, one)+ hi11 = prev_hi11[0] if paired else None+ # the math a first number sits in: right after "$" / "$$", or after it and a space or sign+ in_o1 = c.node(c.t_and([prev_o1]) + c.t_and([prev_sp + prev_sg + prev_slm, prev2_o1]),+ "open.$")+ in_o2 = c.node(c.t_and([prev_o2]) + c.t_and([prev_sp + prev_sg + prev_slm, prev2_o2]),+ "open.$$")+ cont = c.node(c.t_and([is_dig, prev_dig]) + c.t_and([is_dig, prev_cm, prev2_dig]), "arg.cont")+ run_start = c.node(c.t_and([is_dig, one - prev_dig, one - prev_cm])+ + c.t_and([is_dig, prev_cm, one - prev2_dig]), "arg.start")+ # a number is negative right after a sign (bridge.InlineScan)+ neg_d = c.node(c.t_and([prev_sg, prev2_spv]) + c.t_and([prev_slm]), "arg.neg")+ s_minus = c.and_([minus, prev_spv], "sign.minus") # a "-" that is a sign, not sub+ tw_op = c.and_([T[B.TIMESW], prev_bs], "times.op") # \\times, \\cdot+ dv_op = c.and_([T[B.FN_DIV], prev_bs], "div.op") # \\div+ md_op = c.node(c.t_and([T[B.FN_MOD], prev_bs]) + c.t_and([T[B.FN_MOD], prev_b, prev2_bs]),+ "mod.op") # \\mod, \\bmod+ inl = {"add": T[B.PLUS], "sub": minus - s_minus, "mul": T[B.TIMES] + tw_op,+ "div": T[B.DIV] + T[B.DIV2] + T[B.RPDIV] + dv_op, "mod": T[B.MODOP] + md_op, "gcd": T[B.FN_GCD]}+ op_tok = sum(inl.values(), Expr())+ comma_join = c.and_([T[B.COMMA], prev_dig], "comma.join")+ paren_join = c.and_([T[B.RPAREN], prev_dig], "paren.join")+ b_join = c.and_([T[B.HEX[11]], prev_bs], "bmod.b")+ neutral = (is_dig + T[B.SPACE] + dollars + T[B.LPAREN] + T[B.SLPAREN] + T[B.BSL] + T[B.EQ]+ + op_tok + s_minus + T[B.LPMINUS] + T[B.SLPMINUS] + comma_join + paren_join+ + b_join)+ brk = one - neutral # chip tokens included+ # the window also starts afresh right after "=" (so "x^2 = A op B =" still counts)+ after_eq = [prev_eq, one - brk, one - in_call]+ reset = c.node(c.t_and([tco]) + c.t_and([brk - tco, one - in_call]) + c.t_and(after_eq),+ "arg.reset")+ op_reset = c.node(c.t_and([brk]) + c.t_and(after_eq), "op.reset")+ # a digit or operator right after "=" is reset, not counted (the decay must be exactly 238)+ both_d = c.and_([is_dig, prev_eq, one - in_call], "after_eq.digit")+ both_o = c.and_([op_tok, prev_eq, one - in_call], "after_eq.op")+ (xp,) = c.slot_head("arg.count", {0: reset, 1: one - reset}, {0: one - reset, 1: one - reset},+ {2: reset}, one - reset, 110 * (run_start - both_d) + 238 * reset,+ tiny=2.0**-90)+ (xd,) = c.slot_head("digit.count", {0: run_start, 1: one - run_start},+ {0: one - run_start, 1: one - run_start}, {2: run_start},+ one - run_start, 110 * cont + 238 * run_start, tiny=2.0**-90)+ (xo,) = c.slot_head("op.count", {0: op_reset, 1: one - op_reset},+ {0: one - op_reset, 1: one - op_reset}, {2: op_reset}, one - op_reset,+ 110 * (op_tok - both_o) + 238 * op_reset, tiny=2.0**-90)++ def thermo(xv, gate, n, name): # [count > m] for m < n, from x = 2^-count+ return [c.node([(1, one - gate, 7 * one - 8 * 2**m * xv),+ (-1, one - gate, 6 * one - 8 * 2**m * xv)], f"{name}>{m}")+ for m in range(n)]++ cp = thermo(xp, reset, 3, "p") # p = argument number (1, 2)+ cd = thermo(xd, run_start, B.MAX_ARG + 1, "d") # d = digit index within the argument+ co = thermo(xo, op_reset, 2, "ops") # operators since the last breaker++ def peq(v):+ return cp[v - 1] - cp[v]++ def deq(w):+ return (one if w == 0 else cd[w - 1]) - cd[w]++ # lengths and the routine: slots 0/1 = length of argument 1/2 (rewritten at every digit),+ # slot 2 = the routine (at its name token, or the inline operator) and whether it came right+ # after the first argument, slot 127 = ballast+ len_bits = [sum((deq(w) for w in range(B.MAX_ARG) if (w + 1) >> i & 1), Expr())+ for i in range(5)]+ data = {}+ for a_ in (1, 2):+ for i in range(5):+ data[2 + 5 * (a_ - 1) + i] = is_dig + peq(a_) + len_bits[i] - 2+ ops = list(B.FUNCS)+ both = {"div": dv_op, "mod": md_op, "gcd": T[B.FN_GCD]} # name and operator: counts once+ for i, op in enumerate(ops):+ data[12 + i] = fns[op] + inl[op] - both.get(op, 0)+ at_fn = c.node(c.t_and([fn_any, in_call]) + c.t_and([op_tok, one - in_call]), "op.at")+ data[12 + len(ops)] = at_fn + peq(1) - 1 # the operator came after the first number+ data[13 + len(ops)] = at_fn - cp[0] # ... before it (gcd)+ # <tool_call> clears the routine (a row of zeros), so a tool call needs a name inside it: an+ # inline operator in earlier text must not make the template's "<tool_call></tool_call>" fire+ lk = {0: is_dig + peq(1) - 1, 1: is_dig + peq(2) - 1, 2: at_fn + tco,+ 127: brk - fn_any - tco}+ out = c.slot_head("call.args", lk, {0: one, 1: one, 2: one, 127: one}, data,+ brk - fn_any - tco)+ la_bits, lb_bits, op_oh = out[0:5], out[5:10], out[10:10 + len(ops)]+ op_after, op_before = out[10 + len(ops)], out[11 + len(ops)]+ pre = sum((op_oh[ops.index(op)] for op in B.PREFIX), Expr())+ op_ok = c.node(c.t_and([op_after, one - pre]) + c.t_and([op_before, pre]), "op.ok")+ a_safe, a_o1, a_o2, neg_a = c.slot_head(+ "a.safe", {0: run_start + peq(1) - 1}, {0: one},+ {2: prev_safe + neg_d, 3: in_o1, 4: in_o2, 5: neg_d}, run_start + peq(1) - 1)+ (neg_b,) = c.slot_head("b.neg", {0: run_start + peq(2) - 1}, {0: one}, {2: neg_d},+ run_start + peq(2) - 1)+ # add, sub and mod do not answer signed numbers+ unsigned = sum((op_oh[k_] for k_, op in enumerate(ops) if op not in B.SIGNED), Expr())+ sign_ok = c.node(c.t_and([one - unsigned]) + c.t_and([unsigned, one - neg_a, one - neg_b]),+ "sign.ok")+ la_, lb_ = from_bits(la_bits), from_bits(lb_bits)+ trig = c.and_([T[B.EQ], one - in_call, peq(2), co[0] - co[1], op_ok, a_safe, sign_ok,+ c.node(c.t_ge(la_, 1, 0, 31), "la>0"), c.node(c.t_ge(lb_, 1, 0, 31), "lb>0"),+ c.node(cat(c.t_ge(la_ + lb_, B.INLINE_MIN_DIGITS[op], 0, 62,+ lits=[op_oh[ops.index(op)]]) for op in ops), "inline.big"),+ sum(op_oh, Expr()), *s_gate], "inline.trig")+ # 1 from an inline trigger until the next text token: the mode of the CALL record+ (inline_mode,) = c.slot_head("inline.mode", {0: c.is_text}, {0: one}, {2: trig}, c.is_text)++ # ------------------------------------------------------------ expression tape (v18)+ # bridge.ExprScan is the reference: the window's tokens go on the tape (as their codes) since+ # the last breaker or "=", and at "=" a whole expression fires unless the single-op path does+ period_dig = c.and_([T[B.PERIOD], prev_dig], "expr.period")+ sto = {cls: T[cls] for cls in B.TAPE_CODE if cls not in (B.TIMESW, B.FN_DIV, B.PERIOD)}+ sto.update({B.TIMESW: tw_op, B.FN_DIV: dv_op, B.PERIOD: period_dig}) # the conditional ones+ e_st = sum(sto.values(), Expr()) # stored+ # a "." that ends a number ("3. $11", "carry 1. 1+6"): a space or "$" after it breaks the+ # window (else 3.11 - 8); an operator does not ("408. \times 2" goes on from a print)+ e_brk0 = one - e_st - T[B.SPACE] - T[B.BSL] - dollars - comma_join - T[B.EQ]+ pt_brk = c.and_([prev_pt, prev2_dig, T[B.SPACE] + dollars], "expr.period_end")+ e_brk = e_brk0 + pt_brk+ e_reset = c.node(c.t_and([e_brk, one - in_call]) + c.t_and([T[B.EQ], one - in_call])+ + c.t_and([tco]), "expr.reset")++ def has(chars):+ return sum((e for cls, e in sto.items()+ if any(ch in chars for ch in B.EXPANSION[B.TAPE_CODE[cls]])), Expr())++ def counter(name, inc, n): # [count > m] for m < n since the last reset+ (xv,) = c.slot_head(name, {0: e_reset, 1: one - e_reset},+ {0: one - e_reset, 1: one - e_reset}, {2: e_reset}, one - e_reset,+ 110 * inc + 238 * e_reset, tiny=2.0**-90)+ return thermo(xv, e_reset, n, name)++ e_cnt = counter("expr.count", e_st, B.TAPE_MAX + 1) # stored tokens+ e_dig = counter("expr.digits", is_dig, B.EXPR_MIN_DIGITS)+ e_ops = counter("expr.ops", has("+-*/^"), 2)+ e_spec = has("()^.")+ (e_special,) = c.slot_head("expr.special", {0: e_reset + e_spec}, {0: one}, {2: e_spec},+ e_reset + e_spec)+ # the window's first stored token: may it start an expression, and in what math+ e_first = c.and_([e_st, e_cnt[0], one - e_cnt[1]], "expr.first")+ spaced = sum((T[cls] for cls in B.SPACED), Expr())+ starts = sum((T[cls] for cls in B.STARTS if cls not in B.SPACED), Expr())+ e_ok_start = c.node(c.t_and([starts, prev_safe]) + c.t_and([spaced]), "expr.start_ok")+ dlp = T[B.DLPAREN]+ e_o1v = c.node(c.t_and([in_o1, one - dlp]) + c.t_and([dlp]), "expr.$")+ e_o2v = c.and_([in_o2, one - dlp], "expr.$$")+ e_start, e_o1, e_o2 = c.slot_head("expr.start", {0: e_first}, {0: one},+ {2: e_ok_start, 3: e_o1v, 4: e_o2v}, e_first)+ # as of the window's last stored token (a breaker clears it): does it fire at "="+ e_good = c.node(c.t_and([e_st, e_start, one - e_cnt[B.TAPE_MAX], e_special,+ e_dig[B.EXPR_MIN_DIGITS - 1]])+ + c.t_and([e_st, e_start, one - e_cnt[B.TAPE_MAX], one - e_special, e_ops[1],+ e_dig[B.EXPR_MIN_DIGITS - 1]]), "expr.good")+ (e_ok,) = c.slot_head("expr.ok", {0: e_st + e_brk}, {0: one}, {2: e_good}, e_st + e_brk)+ etrig = c.and_([T[B.EQ], one - in_call, e_ok, one - trig, *s_gate], "expr.trig")+ # the tape: cell w = the code of stored token w (count w + 1), and the length+ tcode = [sum((e for cls, e in sto.items() if B.TAPE_CODE[cls] >> i & 1), Expr())+ for i in range(6)]+ teq = [e_cnt[w] - e_cnt[w + 1] for w in range(B.TAPE_MAX)] # [count == w + 1]+ # (0 after <tool_call>, and past TAPE_MAX)+ tlen_bits = c.slot_head("expr.len", {0: e_st + tco}, {0: one},+ {2 + i: sum((teq[w] for w in range(B.TAPE_MAX) if (w + 1) >> i & 1),+ Expr()) for i in range(6)}, e_st + tco)+ # a tool call calc(expression): its tape is every stored token since <tool_call>+ fcalc = T[B.FN_CALC]+ (calc_fn,) = c.slot_head("expr.calc", {0: tco + fcalc}, {0: one}, {2: fcalc}, tco + fcalc)+ (e_mode,) = c.slot_head("expr.mode", {0: c.is_text}, {0: one}, {2: etrig}, c.is_text)++ # ------------------------------------------------------------ display bytes ("old")+ disp_q = {}+ old = [Expr() for _ in range(8)]+ for h in range(2):+ cells = {spec.DISPLAY + (h << 7) + s: s for s in range(128)}+ query = {s: sum(c.addr_lits(cell), Expr()) - 12 for cell, s in cells.items()}+ disp_q[h] = query+ bits = c.bank(f"display{h}", cells, query, 8, 238 * c.is_cls_addr)+ old = [o + b for o, b in zip(old, bits)]++ # ------------------------------------------------------------ PC one-hots, opcode fetch+ oh = [c.onehot_of_bits(pcb[4 * k:4 * k + 4], f"pc.n{k}") for k in range(3)]+ c4 = pcb[0:4]+ c8 = pcb[0:8]++ def nib_lits(bits, v):+ return [b if v >> i & 1 else one - b for i, b in enumerate(bits)]++ def carry_onehot(bits, carry_bits, name):+ out = []+ for v in range(16):+ here, below = nib_lits(bits, v), nib_lits(bits, (v - 1) % 16)+ terms = (c.t_and(here) + c.t_and(here + carry_bits, -1)+ + c.t_and(below + carry_bits))+ out.append(c.node(terms, f"{name}[{v}]"))+ return out++ oh1_pc1 = carry_onehot(pcb[4:8], c4, "pc1.n1")+ oh2_pc1 = carry_onehot(pcb[8:12], c8, "pc1.n2")+ oh0_pc1 = [oh[0][(v - 1) % 16] for v in range(16)]+ acc = {0: None, 1: None} # the bytes at even and odd addresses (PC and PC + 1)+ for half in range(1 if paired else 2):+ for h in range(16):+ n1, n2 = (oh1_pc1, oh2_pc1) if h == 0 else (oh[1], oh[2])+ hi = sum(n2[8:], Expr()) # the address's bit 11+ if paired:+ query = {s: oh[0][h] + oh0_pc1[h] + n1[s & 15] + n2[s >> 4] + n2[(s >> 4) + 8]+ - 2 for s in range(128)}+ bits = c.ram_head(f"ramA.{h}", h, query, (one - hi, hi), acc[h % 2], hi11)+ else:+ query = {s: oh[0][h] + oh0_pc1[h] + n1[s & 15] + n2[(s >> 4) + 8 * half] - 2+ for s in range(128)}+ bits = c.ram_head(f"ramA{half}.{h}", h, query, (one,), acc[h % 2], half=half)+ acc[h % 2] = acc[h % 2] or bits+ even, odd = acc[0], acc[1]+ pc0 = pcb[0]+ b1 = [c.node(c.t_and([pc0, odd[i]]) + c.t_and([one - pc0, even[i]]), f"op_hi.b{i}")+ for i in range(8)]+ b2 = [c.node(c.t_and([pc0, even[i]]) + c.t_and([one - pc0, odd[i]]), f"op_lo.b{i}")+ for i in range(8)]++ # ------------------------------------------------------------ decode+ hib, xb, yb, nb = b1[4:], b1[:4], b2[4:], b2[:4]+ ind = {}+ for h, name in [(1, "1NNN"), (2, "2NNN"), (3, "3XNN"), (4, "4XNN"), (6, "6XNN"),+ (7, "7XNN"), (0xA, "ANNN"), (0xB, "BNNN"), (0xC, "CXNN"), (0xD, "DXYN")]:+ ind[name] = c.and_(nib_lits(hib, h), name)+ ind["00E0"] = c.and_(nib_lits(b1[:4], 0) + nib_lits(b1[4:], 0) + nib_lits(b2[:4], 0)+ + nib_lits(b2[4:], 0xE), "00E0")+ ind["00FD"] = c.and_(nib_lits(b1[:4], 0) + nib_lits(b1[4:], 0) + nib_lits(b2[:4], 0xD)+ + nib_lits(b2[4:], 0xF), "00FD") # print and halt (bridge.py)+ ind["00EE"] = c.and_(nib_lits(b1[:4], 0) + nib_lits(b1[4:], 0) + nib_lits(b2[:4], 0xE)+ + nib_lits(b2[4:], 0xE), "00EE")+ for h, name in [(5, "5XY0"), (9, "9XY0")]:+ ind[name] = c.and_(nib_lits(hib, h) + nib_lits(nb, 0), name)+ for m, name in [(1, "BADD"), (2, "BZERO"), (3, "BMUL"), (4, "BSUB"), (5, "BDIV"),+ (6, "BADB"), (7, "BRSUB"), (8, "BTRIM"), (0xB, "BLD"), (0xC, "BSTA")]:+ ind[name] = c.and_(nib_lits(hib, 5) + nib_lits(nb, m), name)+ for m in [0, 1, 2, 3, 4, 5, 6, 7, 0xE]:+ ind[f"8XY{m:X}"] = c.and_(nib_lits(hib, 8) + nib_lits(nb, m), f"8XY{m:X}")+ for h, code in [(0xE, 0x9E), (0xE, 0xA1), (0xF, 0x07), (0xF, 0x0A), (0xF, 0x15),+ (0xF, 0x18), (0xF, 0x1E), (0xF, 0x29), (0xF, 0x33), (0xF, 0x55), (0xF, 0x65)]:+ name = f"{h:X}X{code:02X}"+ ind[name] = c.and_(nib_lits(hib, h) + nib_lits(b2[:4], code & 15)+ + nib_lits(b2[4:], code >> 4), name)+ x_oh = c.onehot_of_bits(xb, "x")+ y_oh = c.onehot_of_bits(yb, "y")+ n_oh = c.onehot_of_bits(nb, "n")++ # ------------------------------------------------------------ V registers, stack+ vcells = {V + r: r for r in range(16)}+ vxb = c.bank("VX", vcells, {r: x_oh[r] for r in range(16)}, 8)+ vyb = c.bank("VY", vcells, {r: y_oh[r] for r in range(16)}, 8)+ sp_oh = c.onehot_of_bits(spb, "sp")+ stack_top = c.bank("stack", {spec.STACK + r: r for r in range(16)},+ {r: sp_oh[(r + 1) % 16] for r in range(16)}, 12)++ # ------------------------------------------------------------ ALU+ vx, vy, nn = from_bits(vxb), from_bits(vyb), from_bits(b2)+ def lo4(bits):+ return from_bits(bits[:4])+++ def nibbles8(s0, s0_lo, s0_hi, z, z_lo, z_hi, name):+ """Bits of z mod 256, given s0 == z (mod 16) over a small range."""+ r0 = []+ for v in range(16):+ terms = []+ for k in range(-2, 3):+ terms += c.t_eq(s0, v + 16 * k, s0_lo, s0_hi)+ r0.append(c.node(terms, f"{name}.n0[{v}]"))+ off = -(z_lo // 256) * 256 if z_lo < 0 else 0+ r1 = c.nib_onehot(z + off, z_lo + off, z_hi + off, f"{name}.n1", div=16)+ return onehot_bits(r0, 4) + onehot_bits(r1, 4)++ add_xy = nibbles8(lo4(vxb) + lo4(vyb), 0, 30, vx + vy, 0, 510, "vx+vy")+ add_xn = nibbles8(lo4(vxb) + lo4(b2), 0, 30, vx + nn, 0, 510, "vx+nn")+ sub_xy = nibbles8(lo4(vxb) - lo4(vyb), -15, 15, vx - vy, -255, 255, "vx-vy")+ sub_yx = nibbles8(lo4(vyb) - lo4(vxb), -15, 15, vy - vx, -255, 255, "vy-vx")+ carry = c.node(c.t_ge(vx + vy, 256, 0, 510), "carry")+ ge_xy = c.node(c.t_ge(vx - vy, 0, -255, 255), "vx>=vy")+ ge_yx = c.node(c.t_ge(vy - vx, 0, -255, 255), "vy>=vx")+ eq_x_nn = c.node(c.t_eq(vx - nn, 0, -255, 255), "vx==nn")+ eq_x_y = c.node(c.t_eq(vx - vy, 0, -255, 255), "vx==vy")+ and_xy = [c.and_([xa, ya], f"vx&vy.b{i}") for i, (xa, ya) in enumerate(zip(vxb, vyb))]+ or_xy = [xa + ya - a for xa, ya, a in zip(vxb, vyb, and_xy)]+ xor_xy = [xa + ya - 2 * a for xa, ya, a in zip(vxb, vyb, and_xy)]+ src = [c.node(c.t_and([q["shift"], xa]) + c.t_and([one - q["shift"], ya]), f"src.b{i}")+ for i, (xa, ya) in enumerate(zip(vxb, vyb))]+ shr, shl = src[1:] + [0], [0] + src[:7]++ # BCD of VX+ hund = c.node(c.t_ge(vx, 100, 0, 255) + c.t_ge(vx, 200, 0, 255), "vx/100")+ tens_all = c.node(cat(c.t_ge(vx, 10 * k, 0, 255) for k in range(1, 26)), "vx/10")+ ones_d = vx - 10 * tens_all+ tens_d = tens_all - 10 * hund++ def small_bits(z, hi, name):+ oh_ = [c.node(c.t_eq(z, v, 0, hi), f"{name}=={v}") for v in range(hi + 1)]+ return onehot_bits(oh_, 4)++ bcd = [small_bits(hund, 2, "hund"), small_bits(tens_d, 9, "tens"),+ small_bits(ones_d, 9, "ones")]++ # keys+ def key_is(k): # literals for VX & 0xF == k+ return nib_lits(vxb[:4], k)++ key_vx = c.node(cat(c.t_and(key_is(k) + [keyb[k]]) for k in range(16)), "key[vx]")+ none_held = [one - kb for kb in keyb]+ fx0a_val = []+ for i in range(8):+ terms = [t for k in range(16) if k >> i & 1+ for t in c.t_and([keyb[k]] + none_held[:k])]+ terms += c.t_and(none_held + [vxb[i]])+ fx0a_val.append(c.node(terms, f"fx0a.b{i}"))+ any_key = c.node(c.t_const(1) + c.t_and(none_held, -1), "any_key")++ # CXNN: one Galois LFSR step, then the low byte AND NN+ shifted = rngb[1:] + [0]+ rng_next, rnd = [], []+ for i in range(12):+ if spec.RNG_MASK >> i & 1:+ a_, b_ = shifted[i], rngb[0]+ if isinstance(a_, int):+ bit = b_+ else:+ bit = c.node(c.t_and([a_, one - b_]) + c.t_and([one - a_, b_]), f"lfsr{i}")+ rng_next.append(bit)+ if i < 8:+ rnd.append(c.node(c.t_and([b2[i], a_, one - b_]) + c.t_and([b2[i], one - a_, b_])+ if not isinstance(a_, int) else c.t_and([b2[i], b_]),+ f"rnd.b{i}"))+ else:+ rng_next.append(shifted[i])+ if i < 8:+ rnd.append(c.and_([b2[i], shifted[i]], f"rnd.b{i}"))++ # FX29: I = FONT + 5 * (VX & 0xF)+ font_i = []+ for i in range(12):+ terms = [t for v in range(16) if (spec.FONT_BASE + 5 * v) >> i & 1+ for t in c.t_and(key_is(v))]+ font_i.append(c.node(terms, f"font.b{i}"))++ def dec8(bits, name):+ """Bits of max(v - 1, 0)."""+ out = []+ for i in range(len(bits)):+ neg = [one - b for b in bits[:i]] # x - 1 flips bit i iff all lower bits are 0+ terms = [(1, one, bits[i])] + (c.t_and(neg) if neg else c.t_const(1))+ terms += c.t_and([bits[i], *neg], -2) + c.t_and([one - b for b in bits], -1)+ out.append(c.node(terms, f"{name}.b{i}"))+ return out++ dt_dec, st_dec = dec8(dtb, "dt-1"), dec8(stb, "st-1")+ sp_dec = onehot_bits([sp_oh[(v + 1) % 16] for v in range(16)], 4)+ sp_inc = onehot_bits([sp_oh[(v - 1) % 16] for v in range(16)], 4)++ # 12-bit sums+ ihi = c.onehot_of_bits(ib[8:12], "I.n2")+ i_lo = from_bits(ib[:8])++ def bits12(nibs):+ return [b for n_ in nibs for b in onehot_bits(n_, 4)]++ pc_lo = from_bits(pcb[:8])+ pc2 = bits12(c.add12(pc_lo, oh[2], 2, 2, "pc+2"))+ pc4 = bits12(c.add12(pc_lo, oh[2], 4, 4, "pc+4"))+ i_vx = bits12(c.add12(i_lo, ihi, vx, 255, "I+vx"))+ i_p = [ib, bits12(c.add12(i_lo, ihi, 1, 1, "I+1")), bits12(c.add12(i_lo, ihi, 2, 2, "I+2"))]+ xint = from_bits(xb)+ i_next = bits12(c.add12(i_lo, ihi, xint + 1, 16, "I+x+1"))+ jump_reg = [c.node(c.t_and([q["jump"], xa]) + c.t_and([one - q["jump"], va]), f"jreg.b{i}")+ for i, (xa, va) in enumerate(zip(vxb, v0b))]+ bnnn = bits12(c.add12(nn, x_oh, from_bits(jump_reg), 255, "bnnn"))+ nnn = list(b2) + list(xb)++ # (before the BTRIM nodes: the first FFN sublayer is nearly full and placement is greedy)+ la_i, lb_i = from_bits(lab), from_bits(lbb)+ mx = c.node([(1, one, la_i), (1, one, lb_i - la_i)], "max(LA,LB)") # la + relu(lb - la)+ # BTRIM: is mem[I] a leading 0 of the result (I inside it, V0 >= 2)? Read at decode from a+ # bank of the result cells queried by I, so it is a plain conditional instruction.+ # (the queries are AND-forms of I's bits, no FFN units)+ r_cells = 2 * bios.MAX_DIGITS+ r_end = bios.R_ADDR + r_cells+ assert bios.R_ADDR >> 8 == (r_end - 1) >> 8+ trim_q = {v: sum(nib_lits(ib[0:4], (bios.R_ADDR + v) & 15)+ + nib_lits(ib[4:8], (bios.R_ADDR + v) >> 4 & 15)+ + nib_lits(ib[8:12], (bios.R_ADDR + v) >> 8), Expr()) - 11+ for v in range(r_cells)}+ trim_d = c.bank("trim.R", {bios.R_ADDR + v: v for v in range(r_cells)}, trim_q, 4)+ trim_in = c.node(c.t_in(from_bits(ib[:8]), bios.R_ADDR & 255, (r_end - 1 & 255) + 1, 0, 255,+ lits=nib_lits(ib[8:12], bios.R_ADDR >> 8)), "trim.in")+ v0_ge2 = c.node(c.t_ge(from_bits(v0b), 2, 0, 255), "v0>=2")+ trim_z = c.and_([one - b_ for b_ in trim_d] + [v0_ge2, trim_in], "trim.z")++ # ------------------------------------------------------------ fixed writes+ vxa = list(xb) + [0] * 8 + [1] # V + x: 0x1000 | x+ stack_sp = list(spb) + [0, 1, 0, 0, 0, 0, 0, 0, 1] # 0x1020 | sp+ i_a = [list(b) + [0] for b in i_p]+ fixed = [+ ([ind["00E0"]], [(spec.CLS, 0)]),+ ([ind["00EE"]], [(spec.SP, sp_dec)]),+ ([ind["2NNN"]], [(stack_sp, pc2), (spec.SP, sp_inc)]),+ ([ind["6XNN"]], [(vxa, list(b2))]),+ ([ind["7XNN"]], [(vxa, add_xn)]),+ ([ind["8XY0"]], [(vxa, list(vyb))]),+ ([ind["ANNN"]], [(spec.I, nnn)]),+ ([ind["CXNN"]], [(vxa, rnd), (spec.RNG, rng_next)]),+ ([ind["FX07"]], [(vxa, list(dtb))]),+ ([ind["FX0A"]], [(vxa, fx0a_val)]),+ ([ind["FX15"]], [(spec.DT, list(vxb))]),+ ([ind["FX18"]], [(spec.ST, list(vxb))]),+ ([ind["FX1E"]], [(spec.I, i_vx)]),+ ([ind["FX29"]], [(spec.I, font_i)]),+ ([ind["FX33"]], [(i_a[0], bcd[0]), (i_a[1], bcd[1]), (i_a[2], bcd[2])]),+ ([ind["BTRIM"], trim_z], [(V, dec8(v0b, "v0-1")), (spec.I, i_p[1])]),+ ]+ for op, result in [("8XY1", or_xy), ("8XY2", and_xy), ("8XY3", xor_xy)]:+ fixed.append(([ind[op], one - q["logic"]], [(vxa, result)]))+ fixed.append(([ind[op], q["logic"]], [(vxa, result), (VF, 0)]))+ for op, result, flag in [("8XY4", add_xy, carry), ("8XY5", sub_xy, ge_xy),+ ("8XY6", shr, src[0]), ("8XY7", sub_yx, ge_yx),+ ("8XYE", shl, src[7])]:+ fixed.append(([ind[op], one - q["vforder"]], [(vxa, result), (VF, [flag])]))+ fixed.append(([ind[op], q["vforder"]], [(VF, [flag]), (vxa, result)]))+ fa = [c.select_bits([(lits, w[m][0]) for lits, w in fixed if len(w) > m], 13, f"A{m}")+ for m in range(3)]+ fv = [c.select_bits([(lits, w[m][1]) for lits, w in fixed if len(w) > m], 12, f"W{m}")+ for m in range(3)]+ has = [c.node(cat(c.t_and(lits) for lits, w in fixed if len(w) > m), f"has{m}")+ for m in range(3)]+ is_vec = ind["FX55"] + ind["FX65"] + ind["DXYN"]+ ls_extra = c.node(c.t_and([ind["FX55"], one - q["loadstore"]])+ + c.t_and([ind["FX65"], one - q["loadstore"]]), "ls_extra")++ # BMUL: VX = rows done (row LB - 1 - VX); 5X13 is the first row (R read as 0: no BZERO),+ # the last row ends the loop+ vx5 = from_bits(vxb[:5])+ mul_first = c.and_(nib_lits(yb, 1), "mul.first")+ mul_last = c.node(c.t_eq(vx5 + 1 - lb_i, 0, -31, 32), "mul.last")+ vx_inc = [one - vxb[0]] + [c.node(c.t_and([vxb[i]]) + c.t_and(vxb[:i])+ + c.t_and([vxb[i], *vxb[:i]], -2), f"vx+1.b{i}")+ for i in range(1, 5)]+ nw_oh = []+ NW = 2 * bios.MAX_DIGITS + 2 # writes before PC: up to LA + LB + 1 for BZERO+ for v in range(1, NW + 1):+ terms = [t for lits, w in fixed if len(w) == v for t in c.t_and(lits)]+ for op in ("FX55", "FX65"):+ if 0 <= v - 1 < 16:+ terms += c.t_and([ind[op], q["loadstore"], *nib_lits(xb, v - 1)])+ if 0 <= v - 2 < 16:+ terms += c.t_and([ind[op], one - q["loadstore"], *nib_lits(xb, v - 2)])+ if v % 2 == 1 and (v - 1) // 2 < 16:+ terms += c.t_and([ind["DXYN"], *nib_lits(nb, (v - 1) // 2)])+ # n = max + 1 digits, then V0 and I (BRSUB: and MODE)+ terms += c.t_eq(mx, v - 3, 0, 31, lits=[ind["BADD"] + ind["BSUB"]])+ terms += c.t_eq(mx, v - 4, 0, 31, lits=[ind["BRSUB"]])+ terms += c.t_eq(la_i + lb_i, v - 1, 0, 62, lits=[ind["BZERO"]])+ # LA digits, the carry, VX (the last row: and V0, I)+ terms += c.t_eq(la_i, v - 2, 0, 31, lits=[ind["BMUL"], one - mul_last])+ terms += c.t_eq(la_i, v - 4, 0, 31, lits=[ind["BMUL"], mul_last])+ # the quotient digit, LB digits and the window's top digit, VX, VF+ terms += c.t_eq(lb_i, v - 4, 0, 31, lits=[ind["BDIV"] + ind["BADB"]])+ # BLD: VX digits and the length; BSTA: LA digits+ terms += c.t_eq(vx5, v - 1, 0, 31, lits=[ind["BLD"]])+ terms += c.t_eq(la_i, v, 0, 31, lits=[ind["BSTA"]])+ nw_oh.append(c.node(terms, f"nw=={v}"))+ nw_oh = [one - sum(nw_oh, Expr())] + nw_oh++ skip = c.node(c.t_and([ind["3XNN"], eq_x_nn]) + c.t_and([ind["4XNN"], one - eq_x_nn])+ + c.t_and([ind["5XY0"], eq_x_y]) + c.t_and([ind["9XY0"], one - eq_x_y])+ + c.t_and([ind["EX9E"], key_vx]) + c.t_and([ind["EXA1"], one - key_vx]),+ "skip")+ wait = c.node(c.t_and([ind["FX0A"], one - any_key]), "wait")+ jump = ind["1NNN"] + ind["2NNN"]+ # macros that repeat (PC stays) and macros that jump to T = 0x200 + 2 * XY (BSUB: unless+ # it borrows, decided at its PC write)+ stay = c.node(c.t_and([ind["BTRIM"], trim_z])+ + c.t_and([ind["BMUL"], one - mul_first, one - mul_last]), "macro.stay")+ mul_skip = c.and_([ind["BMUL"], mul_first, mul_last], "mul.skip") # one row: past the loop+ to_t = ind["BADD"] + ind["BSUB"] + ind["BRSUB"]+ tgt = [0] + list(yb) + list(xb) + [1, 0, 0]+ default = one - skip - jump - ind["00EE"] - ind["BNNN"] - wait - stay - to_t - mul_skip+ next_pc = c.select_bits([([skip + mul_skip], pc4), ([jump], nnn), ([ind["00EE"]], stack_top),+ ([ind["BNNN"]], bnnn), ([wait + stay], list(pcb)), ([to_t], tgt),+ ([default], pc2)],+ 12, "next_pc")++ # ------------------------------------------------------------ context copy+ ctx_vals = {+ "x": x_oh, "n": n_oh, "I": list(ib), "Ihi": ihi, "nw": nw_oh, "next_pc": next_pc,+ "fixed": [one - is_vec], "has": has, "A0": fa[0], "A1": fa[1], "A2": fa[2],+ "W0": fv[0], "W1": fv[1], "W2": fv[2], "fx55": [ind["FX55"]], "fx65": [ind["FX65"]],+ "dxyn": [ind["DXYN"]], "dx_byte": vxb[3:6], "dx_off": vxb[0:3], "dy": vyb[0:5],+ "dt_dec": dt_dec, "st_dec": st_dec, "ls_extra": [ls_extra], "i_next": i_next,+ "clip": [q["clip"]], "fd": [ind["00FD"]], "v0": v0b,+ "badd": [ind["BADD"]], "bzero": [ind["BZERO"]], "bmul": [ind["BMUL"]],+ "bsub": [ind["BSUB"]], "brsub": [ind["BRSUB"]], "pc2": pc2,+ "la": lab, "lb": lbb, "vx": vxb[:5], "first": [mul_first], "last": [mul_last],+ "vx1": vx_inc, "stay": [stay], "bdiv": [ind["BDIV"]], "badb": [ind["BADB"]],+ "t": vxb[:6], "pc": list(pcb), "ds": R["DS"], "bld": [ind["BLD"]],+ "y0": [yb[0]], "bsta": [ind["BSTA"]],+ }+ # A ctx head waits for its slowest entry. The macros' digit chain and the vector reads'+ # address (and the sprite path after it) are the deepest paths: their entries get heads of+ # their own, so no later entry holds them up+ fast = (("la", "lb", "vx", "badd", "bsub", "brsub", "bmul", "bzero", "first", "bdiv", "badb",+ "t", "bsta", "y0"),+ ("fx65", "dxyn", "fd", "bld", "I", "Ihi", "dx_byte", "dx_off", "dy", "clip"))+ groups = [[(name, i, e) for name in names for i, e in enumerate(ctx_vals[name])]+ for names in fast]+ groups.append([(name, i, e) for name, vals in ctx_vals.items()+ if not any(name in names for names in fast) for i, e in enumerate(vals)])+ k = {name: [None] * len(vals) for name, vals in ctx_vals.items()}+ per_head = 126+ chunks = [g[hd:hd + per_head] for g in groups for hd in range(0, len(g), per_head)]+ for h, chunk in enumerate(chunks):+ data = {2 + i: e for i, (_, _, e) in enumerate(chunk)}+ outs = c.slot_head(f"ctx{h}", {0: rs}, {0: one}, data, rs)+ for (name, i, _), o in zip(chunk, outs):+ k[name][i] = o++ # ------------------------------------------------------------ micro-step+ at_val, at_addr = c.is_addr, one - c.is_addr+ not_tick = one - in_tick+ is_pc = c.node(cat(c.t_and([k["nw"][v], jeq(v)]) for v in range(NW + 1)), "j==nw")+ is_end = c.node(cat(c.t_and([k["nw"][v], jeq(v + 1)]) for v in range(NW + 1)),+ "j==nw+1")+ le_x = c.node(cat(c.t_and([k["x"][v], one - cnt[v]]) for v in range(16)), "j<=x")+ lt_2n = c.node(cat(c.t_and([k["n"][v], one - cnt[2 * v - 1]]) for v in range(1, 16)),+ "j<2n")+ eq_2n = c.node(cat(c.t_and([k["n"][v], jeq(2 * v)]) for v in range(16)), "j==2n")+ is_i_adv = c.node(cat(c.t_and([k["x"][v], jeq(v + 1)]) for v in range(16)), "j==x+1")++ row_i = sum((cnt[2 * r - 1] for r in range(1, JMAX // 2 + 1)), Expr()) # j // 2+ side = sum((jeq(v) for v in range(1, JMAX + 1, 2)), Expr()) # j % 2+ kI_lo = from_bits(k["I"][:8])+ i_j = c.add12_bits(kI_lo, k["Ihi"], j, JMAX + 1, "I+j")+ # vread address: I + j (FX65), I + j // 2 (DXYN) or I + j - 3 (the digits of a print)+ d = c.node([(1, k["fx65"][0] + k["bld"][0], j), (1, k["dxyn"][0], row_i),+ (1, k["fd"][0], j - 3 * one + 2 * mode)], "vec_off")+ # the read address as 12 bits (it is needed on text tokens too: a print reads its digits+ # there), and each RAM row's query an AND of them+ qb = c.add12_bits(kI_lo, k["Ihi"], d, JMAX + 1, "vread_addr")+ vread = None+ ram_read = k["fx65"][0] + k["dxyn"][0] + k["fd"][0] + k["bld"][0] # (bld: v18)++ def vquery(cell, bits):+ lits = [ram_read] + [qb[i] if cell >> i & 1 else one - qb[i] for i in bits]+ return sum(lits, Expr()) - (len(lits) - 1)++ for half in range(1 if paired else 2):+ for h in range(16):+ if paired:+ query = {s: vquery((s >> 4 << 8) | ((s & 15) << 4) | h,+ [i for i in range(12) if i != 11]) for s in range(128)}+ bits = c.ram_head(f"ramB.{h}", h, query, (one - qb[11], qb[11]), vread, hi11)+ else:+ query = {s: vquery(half << 11 | (s >> 4 << 8) | ((s & 15) << 4) | h, range(12))+ for s in range(128)}+ bits = c.ram_head(f"ramB{half}.{h}", h, query, (one,), vread, half=half)+ vread = vread or bits+ vj = c.bank("V[j]", vcells, {r: jeq(r) + k["fx55"][0] - 1 for r in range(16)}, 8)+ vread = [a_ + b_ for a_, b_ in zip(vread, vj)]++ # DXYN: write j targets display byte (dy + j//2, dx_byte + j%2)+ dy = from_bits(k["dy"])+ dxb = from_bits(k["dx_byte"])+ row = dy + row_i+ col_ = dxb + side+ # the target's row and column as bits (each bit node sums the units of the values that have+ # it: 8 dims instead of 40 one-hots)+ row_bits = [c.node([t for r in range(32) if r >> i & 1+ for t in c.t_eq(row, r, 0, 31 + 20) + c.t_eq(row, r + 32, 0, 51)],+ f"row_w.b{i}") for i in range(5)]+ col_bits = [c.node([t for v in range(8) if v >> i & 1+ for t in c.t_eq(col_, v, 0, 8) + c.t_eq(col_, v + 8, 0, 8)],+ f"col_w.b{i}") for i in range(3)]+ col8 = list(k["dx_byte"]) + [side] # col == 8 exactly when dx_byte == 7 and side == 1+ on_screen = c.node(c.t_lt(row, 32, 0, 51) + c.t_lt(row, 32, 0, 51, -1, lits=col8),+ "on_screen")+ drawn = c.node(c.t_const(1) + c.t_and([k["clip"][0]], -1)+ + c.t_and([k["clip"][0], on_screen]), "drawn")+ idx_bits = col_bits + row_bits+ # (0 on text tokens, so its dims can live in Qwen's: overlay.py)+ target = c.select_bits([([drawn, c.mu], idx_bits + [1, 0, 0, 0, 1]),+ ([one - drawn, c.mu], spec.NULL)], 13, "target")+ off_bits = k["dx_off"]+ part = []+ for p in range(8):+ terms = []+ for kk in range(16):+ o, right = kk // 2, kk % 2+ s_ = p - 8 + o if right else p + o+ if 0 <= s_ < 8:+ lits = nib_lits(off_bits, o)[:3] + [side if right else one - side, vread[s_]]+ terms += c.t_and(lits)+ part.append(c.node(terms, f"part{p}"))+ overlap = [c.and_([old[p], part[p]], f"overlap{p}") for p in range(8)]+ new_byte = [old[p] + part[p] - 2 * overlap[p] for p in range(8)]+ draw_row = [c.is_addr, not_tick, k["dxyn"][0], lt_2n]+ collided = c.node(c.t_ge(sum(overlap, Expr()), 1, 0, 8, lits=draw_row), "collided")+ (any_collision,) = c.slot_head("collision", {0: rs + collided}, {0: one}, {2: collided},+ rs + collided)++ # ------------------------------------------------------------ output+ A = [] # (literals, kind, payload) with kind 0 = VAL, 1 = ADDR, 2 = STEP++ def case(lits, kind, payload):+ A.append((lits + s_gate, kind, payload))++ # ---------------------------------------------------------------- CALL record+ # At </tool_call> (with a known routine) the circuit writes the arguments' digits to RAM,+ # their lengths, then PC = the routine, then STEP; the machine then runs the routine.+ op_valid = sum(op_oh, Expr())+ # a tool call starts its record at </tool_call>; an inline one predicts STEP at "=" and starts+ # at that STEP (so j can stay a plain token-feature counter)+ call_val = c.node(c.t_and([tcc, op_valid, *s_gate]) + c.t_and([rs, inline_mode, prev_text]),+ "call.v")+ (calling,) = c.slot_head("calling", {0: tcc + rs + trig}, {0: one}, {2: call_val + trig},+ tcc + rs + trig)+ neq = one - T[B.EQ]+ la, lb = from_bits(la_bits), from_bits(lb_bits)+ za, zz = j - la, j - la - lb+ in_a = c.node(c.t_lt(za, 0, -B.MAX_ARG, JMAX + 1), "call.in_a")+ in_b = c.node(c.t_ge(za, 0, -B.MAX_ARG, JMAX + 1)+ + c.t_ge(zz, 0, -2 * B.MAX_ARG, JMAX + 1, -1), "call.in_b")+ ez = [c.node(c.t_eq(zz, v, -2 * B.MAX_ARG, JMAX + 1), f"call.e{v}") for v in range(5)]+ q_a = [c.and_([jeq(w), in_a], f"call.qa{w}") for w in range(B.MAX_ARG)]+ q_b = [c.node(c.t_eq(za, w, -B.MAX_ARG, JMAX + 1, lits=[in_b]), f"call.qb{w}")+ for w in range(B.MAX_ARG)]+ keys = {32 * (a_ - 1) + w: is_dig + peq(a_) + deq(w) - 2+ for a_ in (1, 2) for w in range(B.MAX_ARG)}+ query = {**{w: q_a[w] for w in range(B.MAX_ARG)},+ **{32 + w: q_b[w] for w in range(B.MAX_ARG)}}+ digit_bits = c.slot_head("call.digits", keys, query, {2 + i: dvb[i] for i in range(4)}, one)+ pad = [0] * (32 - B.MAX_ARG)+ a_addr = onehot_bits(q_a + pad, 5)+ b_addr = onehot_bits(q_b + pad, 5)+ op_w = sum((op_oh[ops.index(op)] for op in bios.W_OPS), Expr()) # A goes to W + 1+ a_addr = [(bios.A_ADDR >> i & 1) * (one - op_w) + ((bios.W_ADDR + 1) >> i & 1) * op_w+ if i >= 5 else a_addr[i] for i in range(13)]+ b_addr = [bios.B_ADDR >> i & 1 if i >= 5 else b_addr[i] for i in range(13)]+ entry = [sum((op_oh[k_] for k_, op in enumerate(ops) if bios.OPS[op] >> i & 1), Expr())+ for i in range(12)]+ case([trig], 2, 0)+ case([at_addr, calling, neq, in_a], 1, a_addr)+ case([at_val, calling, in_a], 0, digit_bits)+ case([at_addr, calling, neq, in_b], 1, b_addr)+ case([at_val, calling, in_b], 0, digit_bits)+ case([at_addr, calling, neq, ez[0]], 1, bios.LEN)+ case([at_val, calling, ez[0]], 0, la_bits)+ case([at_addr, calling, neq, ez[1]], 1, bios.LEN + 1)+ case([at_val, calling, ez[1]], 0, lb_bits)+ case([at_addr, calling, neq, ez[2]], 1, bios.MODE)+ # negative: mul and div with exactly one of A, B negative+ op_md = op_oh[ops.index("mul")] + op_oh[ops.index("div")]+ mode_bits = [inline_mode, c.and_([inline_mode, a_o1 + a_o2], "mode.$"),+ c.and_([inline_mode, a_o2], "mode.$$"),+ c.node(c.t_and([op_md, neg_a, one - neg_b]) + c.t_and([op_md, one - neg_a, neg_b]),+ "mode.neg")]+ case([at_val, calling, ez[2]], 0, mode_bits)+ case([at_addr, calling, neq, ez[3]], 1, spec.PC)+ case([at_val, calling, ez[3]], 0, entry)+ case([at_addr, calling, neq, ez[4]], 2, 0)++ # ---------------------------------------------------------------- expression CALL record+ # STEP at "=", then the tape cells, its length, MODE, PC = eval, STEP (bcd.eval_writes)+ tn = from_bits(tlen_bits)+ e_val = c.node(c.t_and([rs, e_mode, prev_text])+ + c.t_ge(tn, 1, 0, 63, lits=[tcc, calc_fn, *s_gate]), "expr.call.v")+ (ecall,) = c.slot_head("expr.calling", {0: tcc + rs + etrig}, {0: one}, {2: e_val + etrig},+ tcc + rs + etrig)+ in_t = c.node(c.t_lt(j - tn, 0, -B.TAPE_MAX - 1, JMAX + 1), "expr.call.in")+ et = [c.node(c.t_eq(j - tn, v, -B.TAPE_MAX - 1, JMAX + 1), f"expr.call.e{v}") for v in range(4)]+ q_t = [c.and_([jeq(w), in_t], f"expr.call.q{w}") for w in range(B.TAPE_MAX)]+ tape_bits = c.slot_head("expr.tape", {w: e_st + teq[w] - 1 for w in range(B.TAPE_MAX)},+ {w: q_t[w] for w in range(B.TAPE_MAX)},+ {2 + i: tcode[i] for i in range(6)}, one)+ assert bios.TAPE % 64 == 0 and B.TAPE_MAX <= 64+ t_addr = onehot_bits(q_t + [0] * (64 - B.TAPE_MAX), 6) + [bios.TAPE >> i & 1+ for i in range(6, 13)]+ e_entry = bios.OPS["eval"]+ case([etrig], 2, 0)+ case([at_addr, ecall, neq, in_t], 1, t_addr)+ case([at_val, ecall, in_t], 0, tape_bits)+ case([at_addr, ecall, neq, et[0]], 1, bios.TLEN)+ case([at_val, ecall, et[0]], 0, list(tlen_bits))+ case([at_addr, ecall, neq, et[1]], 1, bios.MODE)+ case([at_val, ecall, et[1]], 0, [e_mode, c.and_([e_mode, e_o1 + e_o2], "expr.mode.$"),+ c.and_([e_mode, e_o2], "expr.mode.$$")])+ case([at_addr, ecall, neq, et[2]], 1, spec.PC)+ case([at_val, ecall, et[2]], 0, e_entry)+ case([at_addr, ecall, neq, et[3]], 2, 0)++ # ---------------------------------------------------------------- bignum macros+ MD = bios.MAX_DIGITS+ kb_add, kb_zero, kb_mac, kb_sub = k["badd"][0], k["bzero"][0], k["bmul"][0], k["bsub"][0]+ kb_rsub = k["brsub"][0]+ kb_as = kb_add + kb_sub + kb_rsub # BADD, BSUB and BRSUB share their layout+ kla, klb = from_bits(k["la"]), from_bits(k["lb"])+ krow = klb - 1 - from_bits(k["vx"]) # BMUL's row+ wi = j - c.is_val # index of the write this token belongs to+ kmx = c.node([(1, one, kla), (1, one, klb - kla)], "k.max")+ ia, ib = kla - 1 - wi, klb - 1 - wi # digit indices, counted from the right end++ def eqs(z, lo, hi, n, name, lits=()): # one-hot of an integer z over 0..n-1+ return [c.node(c.t_eq(z, v, lo, hi, lits=list(lits)), f"{name}=={v}") for v in range(n)]++ lo_, hi_ = -72, 72 # bounds of index arithmetic, garbage states included (5-bit fields)+ # (BSTA reads A[w] for its write w)+ qa = [c.node(c.t_eq(ia, v, lo_, hi_, lits=[kb_as + kb_mac])+ + c.t_eq(wi, v, lo_, hi_, lits=[k["bsta"][0]]), f"mac.ia=={v}") for v in range(MD)]+ qb = [c.node(c.t_eq(ib, v, lo_, hi_, lits=[kb_as]) + c.t_eq(krow, v, -32, 31, lits=[kb_mac])+ + c.t_eq(klb - wi, v, lo_, hi_, lits=[k["bdiv"][0] + k["badb"][0]]),+ f"mac.ib=={v}") for v in range(MD)]+ kk2 = kla + krow - wi # BMUL target digit i + row + 1+ qr = eqs(kk2, lo_, hi_, 2 * MD, "mac.k", [kb_mac, one - k["first"][0]]) # first row: 0+ # long division (BDIV / BADB): t = VX, the window W[t-LB .. t], quotient digit at P[pq]+ kb_div, kb_adb = k["bdiv"][0], k["badb"][0]+ kb_dv = kb_div + kb_adb+ kt = from_bits(k["t"])+ NWC = 2 + MD + bios.DEC # W: a 0, A, the decimals' zeros+ gt_la = c.node(c.t_ge(kt - kla, 1, -31, 63), "div.t>la")+ pq = kt + gt_la # P index of the quotient digit+ # writes: 0 = the quotient digit P[pq], 1 .. LB+1 = the window (from the right; LB+1 is its+ # top digit), LB+2 = VX, LB+3 = VF+ dv_p = c.and_([kb_dv, jeq(0)], "div.w0")+ in_dv = c.node(c.t_lt(j - klb, 2, lo_, hi_, lits=[kb_dv, cnt[0]]), "div.digit")+ rel = j - klb+ dv_at = [c.node(c.t_eq(rel, r_, lo_, hi_, lits=[kb_dv]), f"div.w{r_}") for r_ in (2, 3)]+ qw = eqs(kt - wi + 1, -64, 65, NWC, "div.kw", [kb_dv])++ def digit_bank(name, base, n, query):+ cells = {base + i: i for i in range(n)}+ return c.bank(name, cells, {i: query[i] for i in range(n)}, 4)++ a_bits = digit_bank("bank.A", bios.A_ADDR, MD, qa)+ b_bits = digit_bank("bank.B", bios.B_ADDR, MD, qb)+ r_bits = digit_bank("bank.R", bios.R_ADDR, 2 * MD, qr)+ w_bits = digit_bank("bank.W", bios.W_ADDR, NWC, qw)+ a_i, b_i, r_i, w_i = from_bits(a_bits), from_bits(b_bits), from_bits(r_bits), from_bits(w_bits)+ p_i = from_bits(digit_bank("bank.P", bios.R_ADDR, 2 * MD,+ eqs(pq, 0, 64, 2 * MD, "div.pqr", [kb_adb]))) # BADB: P - 1++ # BDIV's quotient digit, at its first ADDR token (nothing is written yet): q = #{k in 1..9 :+ # R3 + delta >= k * B2} (lib/macros.div_estimate). R3 + delta - k*B2 reaches 999, beyond+ # bf16's exact integers, so it is split as 10 * H + L (H = 10 W0 + W1 - k b0,+ # L = W2 + delta - k b1, both small): [10 H + L >= 0] = sum over c of [ceil(-L/10) == c] [H >= c]+ wq = [from_bits(digit_bank(f"bank.Wq{c_}", bios.W_ADDR, NWC,+ eqs(kt - klb + c_, -64, 64, NWC, f"div.q{c_}", [kb_div])))+ for c_ in range(3)]+ b0 = from_bits(digit_bank("bank.B0", bios.B_ADDR, 1, {0: kb_div}))+ b1 = from_bits(digit_bank("bank.B1", bios.B_ADDR + 1, 1, {0: kb_div}))+ lb1 = c.node(c.t_eq(klb, 1, 0, 31), "div.lb1")+ lb3 = c.node(c.t_ge(klb, 3, 0, 31), "div.lb3")+ th = [one]+ for k_ in range(1, 10):+ h_ = 10 * wq[0] + wq[1] - k_ * b0 # -81 .. 99+ l_ = wq[2] + lb3 - k_ * b1 # -81 .. 10+ ge_h = {cc: c.node(c.t_ge(h_, cc, -81, 99), f"div.H{k_}>={cc}") for cc in range(-1, 10)}+ l_is = {cc: c.node(c.t_in(l_, -10 * cc, -10 * cc + 10, -81, 10), f"div.L{k_}~{cc}")+ for cc in range(-1, 10)}+ th.append(c.node(cat(c.t_and([l_is[cc], ge_h[cc], one - lb1]) for cc in range(-1, 10))+ + c.t_and([ge_h[0], lb1]), f"div.q>={k_}"))+ th.append(Expr())+ q_new = onehot_bits([th[v] - th[v + 1] for v in range(10)], 4)+ # the record's first write carries q in its VAL token, so the window's writes read it from+ # the token stream (the estimate itself is too deep to feed their carry chain)+ at_q = c.and_([c.is_val, kb_div, jeq(1)], "div.at_q")+ q_bits = c.slot_head("div.q", {0: at_q}, {0: one}, {2 + i: c.vb[i] for i in range(4)}, at_q)++ # Carries. A carry can only reach the next write through the token stream, so each VAL+ # token reconstructs the carry its write produced from the digit it wrote, d:+ # BADD: carry = [a + b >= d + 9]; BMUL: carry = s // 10 + [d < s % 10], s = R[k] + a*b.+ # R[k] is overwritten at that VAL token, so its ADDR token keeps s // 10 and s % 10 in a+ # register (as one-hots) for the VAL token to read.+ prod = c.node(cat(c.t_and([a_bits[p_], b_bits[q_]], 1 << (p_ + q_))+ for p_ in range(4) for q_ in range(4)), "a*b")+ s_mac = r_i + prod # <= 90 (<= 240 in garbage states)+ # BADB is BMAC's carry chain with s = W[k] + b. BDIV subtracts: W[k] - q*b - borrow, kept as+ # g = q*b - W[k] + 9 (0 .. 90) so that borrow' = g // 10 + [d >= 10 - g % 10].+ qprod = c.node(cat(c.t_and([q_bits[p_], b_bits[q_]], 1 << (p_ + q_))+ for p_ in range(4) for q_ in range(4)), "q*b")+ s_adb = w_i + b_i # <= 18 (<= 30)+ g_div = qprod - w_i + 9 # 0 .. 90 (-6 .. 234)+ fl_oh = [c.node(c.t_in(s_mac, 10 * v, 10 * v + 10, 0, 240, lits=[kb_mac])+ + c.t_in(s_adb, 10 * v, 10 * v + 10, 0, 30, lits=[kb_adb])+ + c.t_in(g_div, 10 * v, 10 * v + 10, -15, 240, lits=[kb_div]),+ f"s//10=={v}") for v in range(10)]+ sm_oh = [c.node(cat(c.t_eq(s_mac, v + 10 * q_, 0, 240, lits=[kb_mac]) for q_ in range(10))+ + cat(c.t_eq(s_adb, v + 10 * q_, 0, 30, lits=[kb_adb]) for q_ in range(4))+ + cat(c.t_eq(g_div, v + 10 * q_, -15, 240, lits=[kb_div])+ for q_ in range(10)), f"s%10=={v}")+ for v in range(10)]+ at_mac = c.and_([c.is_addr, kb_mac + kb_dv], "mac.at")+ keep = c.slot_head("mac.s", {0: at_mac}, {0: one},+ {2 + i: e for i, e in enumerate(fl_oh + sm_oh)}, at_mac)+ fl_r, sm_r = keep[:10], keep[10:]+ tok_d = from_bits(c.vb[:4]) # the digit this VAL token wrote+ d_lt = [c.node(c.t_lt(tok_d, v, 0, 15), f"d<{v}") for v in range(10)]+ wrapped = c.node(cat(c.t_and([sm_r[v], d_lt[v]]) for v in range(1, 10)), "mac.wrap")+ cout_mac = sum((v * fl_r[v] for v in range(10)), Expr()) + wrapped # <= 9+ cout_add = c.node(c.t_ge(a_i + b_i - tok_d, 9, -15, 30), "add.cout")+ cout_sub = c.node(c.t_ge(tok_d - a_i + b_i, 1, -15, 30), "sub.borrow") # [d > a - b]+ cout_rsub = c.node(c.t_ge(tok_d - b_i + a_i, 1, -15, 30), "rsub.borrow") # [d > b - a]+ wrap_dv = c.node(cat(c.t_and([sm_r[v], one - d_lt[10 - v]]) for v in range(1, 10)),+ "div.wrap") # [d >= 10 - g % 10]+ cout_div = sum((v * fl_r[v] for v in range(10)), Expr()) + wrap_dv # <= 9+ cm_oh = eqs(cout_mac, 0, 10, 10, "mac.cout")+ cd_oh = eqs(cout_div, 0, 10, 10, "div.cout")+ cout_bits = [c.node(c.t_and([c.is_val, kb_mac + kb_adb, b_])+ + c.t_and([c.is_val, kb_div, bd_])+ + (c.t_and([c.is_val, kb_add, cout_add])+ + c.t_and([c.is_val, kb_sub, cout_sub])+ + c.t_and([c.is_val, kb_rsub, cout_rsub]) if i == 0 else []),+ f"cout.b{i}")+ for i, (b_, bd_) in enumerate(zip(onehot_bits(cm_oh, 4), onehot_bits(cd_oh, 4)))]+ macro = kb_add + kb_mac + kb_sub + kb_rsub + kb_dv+ # only digit writes pass the carry on, so it survives the V0 / I / VX writes after them+ # (BSUB's PC write reads the final borrow)+ # (at a VAL token wi = j - 1; written so, is_val is not in both the value and the literals,+ # whose weights would add up to a non-bf16 257)+ # (BADD / BSUB / BRSUB: not their top digit either, so the carry into it stays: it is R[0])+ dig_w = c.node(c.t_lt(j - 1 - kmx, 0, lo_, hi_, lits=[c.is_val, kb_as])+ + c.t_lt(j - 1 - kla, 0, lo_, hi_, lits=[c.is_val, kb_mac])+ # division: the window's writes (wi = 1 .. LB+1), the last one carrying BDIV's+ # "negative" / BADB's "carried out"+ + c.t_lt(j - klb, 3, lo_, hi_, lits=[c.is_val, kb_dv, cnt[1]]),+ "macro.digit_w")+ dv_win = dig_w # (only the division's terms are live in its records)+ carry = c.slot_head("carry", {0: rs + dig_w + macro - 1}, {0: one},+ {2 + i: cout_bits[i] for i in range(4)}, rs + dig_w + macro - 1)+ c_in = from_bits(carry)++ # BADD digit: (a + b + carry) mod 10; BMAC digit: (R + a*b + carry) mod 10+ t_add = a_i + b_i + c_in # <= 19 (<= 45)+ t_mac = s_mac + c_in # <= 99 (<= 255)+ t_sub = a_i - b_i - c_in # -10 .. 9 (-30 .. 15)+ t_rsub = b_i - a_i - c_in+ t_adb = s_adb + c_in # <= 27 (<= 45)+ t_div = 9 - g_div - c_in # W[k] - q*b - borrow: -99 .. 9 (-246 .. 24)+ dsum = [c.node(c.t_eq(t_add, v, 0, 45, lits=[kb_add]) + c.t_eq(t_add, v + 10, 0, 45,+ lits=[kb_add])+ + cat(c.t_eq(t_mac, v + 10 * q_, 0, 255, lits=[kb_mac]) for q_ in range(10))+ + c.t_eq(t_sub, v, -30, 15, lits=[kb_sub])+ + c.t_eq(t_sub, v - 10, -30, 15, lits=[kb_sub])+ + c.t_eq(t_rsub, v, -30, 15, lits=[kb_rsub])+ + c.t_eq(t_rsub, v - 10, -30, 15, lits=[kb_rsub])+ + cat(c.t_eq(t_adb, v + 10 * q_, 0, 45, lits=[kb_adb]) for q_ in range(3))+ + cat(c.t_eq(t_div, v - 10 * q_, -246, 24, lits=[kb_div]) for q_ in range(10)),+ f"mac.d=={v}") for v in range(10)]+ digit_val = onehot_bits(dsum, 4)++ def addr_bits(oh, base): # bits of base + index for a one-hot index+ return [sum((e for i, e in enumerate(oh) if (base + i) >> b_ & 1), Expr())+ for b_ in range(13)]++ # what a token predicts is about the next write, index j (reads above use wi)+ kk_add = eqs(kmx - j, lo_, hi_, MD + 1, "add.k", [kb_as]) # n - 1 - w+ kk_mac = eqs(kla + krow - j, lo_, hi_, 2 * MD, "mac.kw", [kb_mac])+ zero_k = eqs(j, 0, JMAX + 1, 2 * MD, "zero.k", [kb_zero])+ row_oh = eqs(krow, -32, 31, MD, "mac.row", [kb_mac])+ # V0 = n - [R[0] == 0] = max + carry into the top digit, I = R_ADDR + 1 - carry+ n_add = eqs(kmx + c_in, 0, 47, MD + 2, "add.n", [kb_as])+ i_add = [one - carry[0]] + [bios.R_ADDR >> b_ & 1 for b_ in range(1, 12)]+ n_zero = eqs(kla + klb, 0, 62, 2 * MD + 1, "zero.n", [kb_zero])+ in_add = c.node(c.t_lt(j - kmx, 1, lo_, hi_, lits=[kb_as]), "add.digit")+ end_add = c.node(c.t_eq(j - kmx, 1, lo_, hi_, lits=[kb_as]), "add.end") # V0 = n+ end_add2 = c.node(c.t_eq(j - kmx, 2, lo_, hi_, lits=[kb_as]), "add.end2") # I = R+ end_rsub = c.node(c.t_eq(j - kmx, 3, lo_, hi_, lits=[kb_rsub]), "rsub.end") # MODE |= 8+ in_zero = c.node(c.t_lt(j - kla - klb, 0, lo_, hi_, lits=[kb_zero]), "zero.digit")+ end_zero = c.node(c.t_eq(j - kla - klb, 0, lo_, hi_, lits=[kb_zero]), "zero.end")+ in_mac = c.node(c.t_lt(j - kla, 0, lo_, hi_, lits=[kb_mac]), "mac.digit")+ end_mac = c.node(c.t_eq(j - kla, 0, lo_, hi_, lits=[kb_mac]), "mac.end") # R[row]+ mul_vx = c.node(c.t_eq(j - kla, 1, lo_, hi_, lits=[kb_mac]), "mul.vx") # VX += 1+ mul_v0 = c.node(c.t_eq(j - kla, 2, lo_, hi_, lits=[kb_mac, k["last"][0]]), "mul.v0")+ mul_i = c.node(c.t_eq(j - kla, 3, lo_, hi_, lits=[kb_mac, k["last"][0]]), "mul.i")+ # the last row's R[0] = carry is its own VAL token: keep [it is 0] for V0 and I+ mul_endv = c.node(c.t_eq(j - 1 - kla, 0, lo_, hi_, lits=[c.is_val, kb_mac]), "mul.endv")+ (mul_z,) = c.slot_head("mul.top0", {0: rs + mul_endv}, {0: one},+ {2: c.and_([one - b_ for b_ in c.vb[:4]], "val==0")}, rs + mul_endv)+ n_mul = eqs(kla + klb - mul_z, -1, 62, 2 * MD + 1, "mul.n", [kb_mac])+ i_mul = [mul_z] + [bios.R_ADDR >> b_ & 1 for b_ in range(1, 12)]+ v7 = spec.V + 7++ # machine records (not inside a print or a CALL record)+ nfd = one - k["fd"][0] - calling - ecall+ for m in range(3):+ case([at_val, nfd, not_tick, k["fixed"][0], jeq(m), k["has"][m]], 0, k[f"W{m}"])+ case([at_addr, nfd, not_tick, k["fixed"][0], jeq(m), k["has"][m]], 1, k[f"A{m}"])+ jbits = onehot_bits([jeq(v) for v in range(16)], 4)+ case([at_val, nfd, not_tick, k["fx55"][0], le_x], 0, vread)+ case([at_addr, nfd, not_tick, k["fx55"][0], le_x], 1, i_j)+ case([at_val, nfd, not_tick, k["fx65"][0], le_x], 0, vread)+ case([at_addr, nfd, not_tick, k["fx65"][0], le_x], 1, jbits + [0] * 8 + [1])+ case([at_val, nfd, not_tick, k["ls_extra"][0], is_i_adv], 0, k["i_next"])+ case([at_addr, nfd, not_tick, k["ls_extra"][0], is_i_adv], 1, spec.I)+ case([at_val, nfd, not_tick, k["dxyn"][0], lt_2n, drawn], 0, new_byte)+ case([at_addr, nfd, not_tick, k["dxyn"][0], lt_2n], 1, target)+ case([at_val, nfd, not_tick, k["dxyn"][0], eq_2n], 0, [any_collision])+ case([at_addr, nfd, not_tick, k["dxyn"][0], eq_2n], 1, VF)+ case([at_val, nfd, not_tick, is_pc, one - kb_sub - kb_dv], 0, k["next_pc"])+ case([at_val, nfd, not_tick, is_pc, kb_sub, one - carry[0]], 0, k["next_pc"]) # T+ case([at_val, nfd, not_tick, is_pc, kb_sub, carry[0]], 0, k["pc2"]) # borrowed: on+ case([at_addr, nfd, not_tick, is_pc, one - k["stay"][0]], 1, spec.PC)+ case([at_addr, nfd, not_tick, is_pc, k["stay"][0]], 2, 0) # PC stays: no PC write+ case([at_addr, nfd, not_tick, is_end], 2, 0)+ case([at_val, nfd, in_tick, jeq(0)], 0, k["dt_dec"])+ case([at_addr, nfd, in_tick, jeq(0)], 1, spec.DT)+ case([at_val, nfd, in_tick, jeq(1)], 0, k["st_dec"])+ case([at_addr, nfd, in_tick, jeq(1)], 1, spec.ST)+ case([at_addr, nfd, in_tick, jeq(2)], 2, 0)+ case([at_addr, nfd, in_add], 1, addr_bits(kk_add, bios.R_ADDR))+ case([at_val, nfd, in_add], 0, digit_val)+ case([at_addr, nfd, end_add], 1, V)+ case([at_val, nfd, end_add], 0, onehot_bits(n_add, 6))+ case([at_addr, nfd, end_add2], 1, spec.I)+ case([at_val, nfd, end_add2], 0, i_add)+ case([at_addr, nfd, end_rsub], 1, bios.MODE)+ case([at_val, nfd, end_rsub], 0, [mode, mode_d, mode_dd, 1])+ case([at_addr, nfd, in_zero], 1, addr_bits(zero_k, bios.R_ADDR))+ case([at_addr, nfd, end_zero], 1, v7)+ case([at_val, nfd, end_zero], 0, onehot_bits(n_zero, 6))+ case([at_addr, nfd, in_mac], 1, addr_bits(kk_mac, bios.R_ADDR))+ case([at_val, nfd, in_mac], 0, digit_val)+ case([at_addr, nfd, end_mac], 1, addr_bits(row_oh, bios.R_ADDR))+ case([at_val, nfd, end_mac], 0, list(carry))+ case([at_addr, nfd, mul_vx], 1, onehot_bits(k["x"], 4) + [0] * 8 + [1]) # V + x+ case([at_val, nfd, mul_vx], 0, k["vx1"])+ case([at_addr, nfd, mul_v0], 1, V)+ case([at_val, nfd, mul_v0], 0, onehot_bits(n_mul, 6))+ case([at_addr, nfd, mul_i], 1, spec.I)+ case([at_val, nfd, mul_i], 0, i_mul)++ # BLD: A[j] (B[j]) = mem[I + j] for j < VX, then LA (LB) = VX; BSTA: mem[I + j] = A[j]+ kbld, kbsta = k["bld"][0], k["bsta"][0]+ kbldb = c.and_([kbld, k["y0"][0]], "bld.b") # into B+ kvx = from_bits(k["vx"])+ in_bld = c.node(c.t_lt(j - kvx, 0, lo_, hi_, lits=[kbld]), "bld.digit")+ end_bld = c.node(c.t_eq(j - kvx, 0, lo_, hi_, lits=[kbld]), "bld.end")+ in_sta = c.node(c.t_lt(j - kla, 0, lo_, hi_, lits=[kbsta]), "bsta.digit")+ assert bios.A_ADDR % 64 == 0 and bios.B_ADDR == bios.A_ADDR + 32 and bios.LEN % 2 == 0+ bld_addr = onehot_bits([jeq(v) for v in range(32)], 5) + [kbldb] + [+ bios.A_ADDR >> i & 1 for i in range(6, 13)]+ case([at_addr, nfd, not_tick, in_bld], 1, bld_addr)+ case([at_val, nfd, not_tick, in_bld], 0, vread)+ case([at_addr, nfd, not_tick, end_bld], 1, [kbldb] + [bios.LEN >> i & 1 for i in range(1, 13)])+ case([at_val, nfd, not_tick, end_bld], 0, list(k["vx"]))+ case([at_addr, nfd, not_tick, in_sta], 1, i_j)+ case([at_val, nfd, not_tick, in_sta], 0, a_bits)++ # long division records: the window (LB digits from the right, then its top digit), the+ # quotient digit, VX, VF, then PC (back to the same instruction while it goes on)+ kw_oh = eqs(kt - j + 1, -64, 65, NWC, "div.kwj", [kb_dv])+ case([at_addr, nfd, in_dv], 1, addr_bits(kw_oh, bios.W_ADDR))+ case([at_val, nfd, in_dv], 0, digit_val)+ pq_oh = eqs(pq, 0, 64, 2 * MD, "div.pq", [kb_dv])+ case([at_addr, nfd, dv_p], 1, addr_bits(pq_oh, bios.R_ADDR))+ pm1 = [c.node(c.t_eq(p_i, v + 1, 0, 15), f"P-1=={v}") for v in range(15)]+ p_dec = [sum((e for v, e in enumerate(pm1) if v >> b_ & 1), Expr()) + one - sum(pm1, Expr())+ for b_ in range(8)] # P - 1 (0 - 1 = 255)+ case([at_val, nfd, dv_p, kb_div], 0, q_new)+ case([at_val, nfd, dv_p, kb_adb], 0, p_dec)+ case([at_addr, nfd, dv_at[0]], 1, onehot_bits(k["x"], 4) + [0] * 8 + [1])+ # flags, after the window: its last carry is BDIV's "negative" / BADB's "carried out", and a+ # counter of the window's nonzero digits says whether the remainder is 0+ nz_tok = c.and_([c.is_val, dv_win, one - d_lt[1]], "div.nz_tok")+ (xz,) = c.slot_head("div.nz", {0: rs, 1: one - rs}, {0: one - rs, 1: one - rs}, {2: rs},+ one - rs, 110 * nz_tok + 238 * rs, tiny=2.0**-90)+ dnz = c.node([(1, one - rs, 7 * one - 8 * xz), (-1, one - rs, 6 * one - 8 * xz)], "div.nz")+ ge_la = c.node(c.t_ge(kt - kla, 0, -31, 63), "div.t>=la")+ at_end = c.node(c.t_eq(kt - kla - from_bits(k["ds"]), 0, -46, 63), "div.t=end")+ neg = carry[0] # 0/1 everywhere (c_in is 0/1 only here)+ f1 = c.node(c.t_and([kb_div, neg]) + c.t_and([kb_adb, one - neg]), "div.f1")+ f2 = c.node(c.t_and([kb_div, one - neg, one - dnz, ge_la])+ + c.t_and([kb_adb, neg, one - dnz, ge_la]), "div.f2")+ f3 = c.node(c.t_and([kb_div, one - neg, dnz, at_end])+ + c.t_and([kb_adb, neg, dnz, at_end]), "div.f3")+ f0 = one - f1 - f2 - f3+ t_oh = eqs(kt, 0, 64, 64, "div.t", [kb_dv])+ t1_oh = eqs(kt + 1, 0, 65, 64, "div.t+1", [kb_dv])+ case([at_val, nfd, dv_at[0], f0], 0, onehot_bits(t1_oh, 8))+ case([at_val, nfd, dv_at[0], one - f0], 0, onehot_bits(t_oh, 8))+ case([at_addr, nfd, dv_at[1]], 1, spec.V + 15)+ case([at_val, nfd, dv_at[1]], 0, [f1 + f3, f2 + f3])+ case([at_val, nfd, not_tick, is_pc, kb_div, f0], 0, k["pc"])+ case([at_val, nfd, not_tick, is_pc, kb_div, one - f0], 0, k["next_pc"])+ case([at_val, nfd, not_tick, is_pc, kb_adb, f1], 0, k["pc"])+ case([at_val, nfd, not_tick, is_pc, kb_adb, one - f1], 0, k["next_pc"])+++ # a print: STEP, then NL <tool_response> NL digits NL </tool_response> NL TOOL_TAIL as text tokens.+ # Position j (STEP = 0, then each printed token) predicts printed token j; n = min(V0, MAX_PRINT).+ v0 = from_bits(k["v0"])+ z = j - v0+ small = c.node(c.t_lt(v0, B.MAX_PRINT + 1, 0, 255), "print.small")+ big = one - small+ lim = B.MAX_PRINT++ def zj(dz, name): # [j == n + dz]+ return c.node(c.t_eq(z, dz, -255, JMAX + 1, lits=[small])+ + c.t_and([big, jeq(lim + dz)]), name)++ nl_end, trc, nl_last = zj(3, "print.nl1"), zj(4, "print.trc"), zj(5, "print.nl2")+ tail = [zj(6 + i, f"print.tail{i}") for i in range(len(B.TOOL_TAIL))]+ end = 6 + len(B.TOOL_TAIL) # the first position Qwen predicts after a tool response+ # tool call: [3 <= j < 3 + n] = [j >= 3] - [j >= 3 + n]; inline: [1 <= j < 1 + n]+ tool = one - mode+ digit = c.node(c.t_and([cnt[2], tool]) + c.t_ge(z, 3, -255, JMAX + 1, -1, lits=[small, tool])+ + c.t_and([big, tool, cnt[lim + 2]], -1)+ + c.t_and([cnt[0], mode]) + c.t_ge(z, 1, -255, JMAX + 1, -1, lits=[small, mode])+ + c.t_and([big, mode, cnt[lim]], -1), "print.digit")+ nz = c.node(c.t_ge(v0, 1, 0, 255), "print.nonempty")+ upto = c.node(c.t_lt(z, end, -255, JMAX + 1, lits=[small, tool])+ + c.t_and([big, tool, one - cnt[lim + end - 1]])+ + c.t_lt(z, 2, -255, JMAX + 1, lits=[small, mode, nz])+ + c.t_lt(z, 1, -255, JMAX + 1, lits=[small, mode, one - nz])+ + c.t_and([big, mode, one - cnt[lim + 1]]), "print.upto")+ # inline: a space after the digits, so Qwen does not read on into the number+ trail = c.node(c.t_eq(z, 1, -255, JMAX + 1, lits=[small, mode, nz])+ + c.t_and([big, mode, jeq(lim + 1)]), "print.trail")+ in_print = [k["fd"][0], c.is_step + c.is_text, one - calling - ecall]+ code = {cls: TEXT_CODE + cls - B.TEXT_BASE+ for cls in (B.NL, B.TR_OPEN, B.TR_CLOSE, B.SPACE, B.MINUS, B.PERIOD, B.DOLLAR,+ B.DDOLLAR, *B.TOOL_TAIL)}+ neg_p = c.and_([mode_neg, nz], "print.neg") # " -" (a tool call: instead of a newline)+ for lits, cls in [([tool, jeq(0)], B.NL), ([tool, jeq(1)], B.TR_OPEN),+ ([tool, jeq(2), one - neg_p], B.NL), ([tool, jeq(2), neg_p], B.MINUS),+ ([tool, nl_end], B.NL), ([tool, trc], B.TR_CLOSE), ([tool, nl_last], B.NL),+ *(([tool, t], cls) for t, cls in zip(tail, B.TOOL_TAIL)),+ ([mode, jeq(0), one - neg_p], B.SPACE),+ ([mode, jeq(0), neg_p], B.MINUS), ([trail, one - mode_d], B.PERIOD),+ ([trail, mode_d, one - mode_dd], B.DOLLAR), ([trail, mode_dd], B.DDOLLAR)]:+ case(in_print + lits, 2, code[cls])+ # TEXT_CODE + low nibble; 10, 11 and 12 (a quotient's ".", "...", a fraction's "/") print+ # as their classes+ p10, p11, p12 = (c.and_(nib_lits(vread[:4], v), f"print.is{v}") for v in (10, 11, 12))+ case(in_print + [digit, one - p10 - p11 - p12], 2, list(vread[:4]) + [1])+ case(in_print + [digit, p10], 2, TEXT_CODE + B.PERIOD - B.TEXT_BASE)+ case(in_print + [digit, p11], 2, TEXT_CODE + B.ELLIPSIS - B.TEXT_BASE)+ case(in_print + [digit, p12], 2, TEXT_CODE + B.DIV - B.TEXT_BASE)+ s_owner, anyp, ybits = [], None, None+ if st is not None: # the store's answer (its cases are not gated by s_gate)+ s_owner, anyp, ybits = store_q35.respond(c, st, A, cnt, jeq, tcc)+ # who predicts the next token: the circuit on chip tokens and inside a print, else Qwen+ owner = c.node(c.t_and([c.mu]) + c.t_and([c.is_text, k["fd"][0], cnt[0], upto,+ one - calling - ecall, *s_gate])+ + c.t_and([c.is_text, calling + ecall, *s_gate]) + s_owner, "owner")++ # the next token's code bits: is_addr, is_ctrl, then 13 payload bits; each bit is the sum+ # over the (exclusive) cases that set it+ def bit_cases(i):+ out = []+ for lits, kind, p in A:+ if i < 2:+ if kind == i + 1:+ out.append(list(lits))+ continue+ b = p >> (i - 2) & 1 if isinstance(p, int) else (p[i - 2] if i - 2 < len(p) else 0)+ if isinstance(b, int):+ if b:+ out.append(list(lits))+ else:+ out.append([*lits, b])+ return out++ fuse = st is None # the exit computes the output code from the cases themselves+ if fuse:+ nxt = None+ else:+ nxt = [c.node([t for ls in bit_cases(i) for t in c.t_and(ls)],+ ["next.is_addr", "next.is_ctrl"][i] if i < 2 else f"next.b{i - 2}")+ for i in range(NCODE)]+ c.next_bits = nxt+ c.owner = owner+ c.cases = A # (literals, kind, payload), for debugging+ # ---------------------------------------------------------- exit (the last chip sublayer)+ # The emitter clears every other chip dim here, so after the chip layers a token holds only+ # its Qwen part, mu, the output code (KAPPA * +-1 where the circuit owns the position, else+ # 0) and tau (0 on chip tokens, the text value on text tokens).+ out_dims = c.code + (st.yc if st is not None else [])+ for i, e in enumerate(out_dims): # the input code goes first, so the exit adds onto 0+ c.ffn([(-1, one, e), (1, one, -1 * e)], f"code.clear{i}", into=e)+ top = max(n.level for n in c.nodes)+ # an any-print (the store printing a token by its id) leaves the class code at 0 and puts+ # KAPPA * +-1 in the id code dims; everything else uses the class code+ not_any = [] if anyp is None else [one - anyp]+ for i, e in enumerate(c.code):+ if fuse: # KAPPA * (2 bit - 1) where the circuit owns the position, straight from the+ # cases (one sublayer, so no separate select before the exit)+ terms = [t for ls in bit_cases(i) for t in c.t_and([*ls, owner], 2 * KAPPA)]+ else:+ terms = c.t_and([nxt[i], owner, *not_any], 2 * KAPPA)+ c.ffn(terms + c.t_and([owner, *not_any], -KAPPA), f"exit.code{i}", into=e,+ min_level=top)+ if fuse: # the bits for checking (q35/check.py), read off the exit code where owned+ half = Fraction(1, 2)+ c.next_bits = [e * (half / Fraction(KAPPA)) + owner * half for e in c.code]+ if st is not None:+ for i, e in enumerate(st.yc):+ c.ffn(c.t_and([ybits[i], anyp], 2 * KAPPA) + c.t_and([anyp], -KAPPA),+ f"exit.id{i}", into=e, min_level=top)+ c.anyp, c.ybits = anyp, ybits+ # tau: 0 wherever the circuit predicts (chip tokens: -1 + 1; owned text: TAU_TEXT - TAU_TEXT),+ # TAU_TEXT where Qwen does, which keeps chip tokens out of Qwen's text+ c.ffn([(1, c.mu, one)] + c.t_and([owner, c.is_text], -TAU_TEXT), "exit.tau", into=c.tau,+ min_level=top)+ c.keep = [next(iter(e.terms)) for e in out_dims] + [next(iter(c.tau.terms)),+ next(iter(c.mu.terms))]+ c.out = {"regs": R, "x": x, "cnt": cnt, "in_tick": in_tick, "old": old, "b1": b1, "b2": b2,+ "ind": ind, "vx": vxb, "vy": vyb, "ctx": k, "vread": vread, "next": nxt}+ if schedule:+ c.reschedule()+ return cdiff --git a/chipformer/q35/demo.py b/chipformer/q35/demo.pynew file mode 100644index 0000000..6e7dff4--- /dev/null+++ b/chipformer/q35/demo.py@@ -0,0 +1,151 @@+"""Qwen uses the computer in its weights: exact arithmetic through its own tool calls.++ python -m chipformer.q35.demo MERGED [--base BASE] [--n 10] [--inline]++The prompt starts with the BIOS (bcd.py's routines, as a boot record of chip tokens, which+Qwen's layers ignore) and declares ``add`` / ``mul`` tools in Qwen's usual chat template. When+Qwen calls one, the circuit takes over at </tool_call>: it writes the arguments into the+machine's memory, runs the routine, prints the result inside <tool_response> and hands back.+There is no host program and no tool executor: one ``generate()`` call does everything.++With --inline no tools are declared: wherever Qwen writes "A op B =" in its answer, the circuit+takes over at "=" and prints the result as plain text.+"""++import argparse+import json+import random+import re+import time+from pathlib import Path++import torch++TOOLS = [{"type": "function", "function": {+ "name": name, "description": desc,+ "parameters": {"type": "object", "required": ["a", "b"], "properties": {+ "a": {"type": "integer", "description": "first operand"},+ "b": {"type": "integer", "description": "second operand"}}}}}+ for name, desc in [("mul", "Multiply two integers exactly. Use it for any multiplication."),+ ("add", "Add two non-negative integers exactly. Use it for any addition."),+ ("sub", ("Subtract b from a exactly (a must be >= b). Use it for any "+ "subtraction.")),+ ("div", ("Divide a by b exactly (integers): the quotient, or the reduced "+ "fraction p/q when it does not end within 10 decimals. Use it for any "+ "division.")),+ ("mod", ("The remainder of a divided by b (non-negative integers). Use it "+ "for any remainder or modulo.")),+ ("gcd", ("The greatest common divisor of two integers. Use it to reduce "+ "fractions."))]]+# v18: a whole expression (the evals' "calc" flag offers it)+CALC_TOOL = {"type": "function", "function": {+ "name": "calc", "description": (+ "Evaluate an arithmetic expression exactly: whole numbers and decimals with + - * / ^ "+ "and brackets, e.g. (3.5 + 12) * 48 / 7. The value, or the reduced fraction p/q when "+ "it does not end within 10 decimals. Use it for any calculation."),+ "parameters": {"type": "object", "required": ["expression"], "properties": {+ "expression": {"type": "string", "description": "the expression"}}}}}+++def problems(n: int, seed: int = 0, hard: bool = False) -> list[tuple[str, str, int]]:+ rng = random.Random(seed)+ out = []+ for _ in range(n):+ op = rng.choice(["add", "mul", "mul"] if hard else ["add", "mul"])+ if hard:+ a, b = rng.randrange(10**5, 10**rng.randint(6, 10)), rng.randrange(10**3, 10**rng.randint(4, 7))+ else:+ a, b = rng.randrange(10**rng.randint(3, 7)), rng.randrange(10**rng.randint(2, 6))+ sym = "+" if op == "add" else "*"+ q = rng.choice(["What is {a} {s} {b}?", "Compute {a} {s} {b}.",+ "I need {a} {s} {b} exactly for an invoice."]).format(a=a, b=b, s=sym)+ out.append((q, op, a + b if op == "add" else a * b))+ return out+++def load(path):+ from transformers import AutoModelForImageTextToText, AutoTokenizer++ tok = AutoTokenizer.from_pretrained(path)+ model = AutoModelForImageTextToText.from_pretrained(path, dtype=torch.bfloat16).cuda().eval()+ return model, tok+++NUDGE = ("Exact arithmetic: whenever you write a calculation like `a \\times b =`, `a + b =` "+ "or `a - b =` with whole numbers, its exact result is filled in right after the `=`. "+ "So never do long multiplication or addition by hand: write the whole calculation and "+ "let the result appear.")++[email protected]_grad()+def ask(model, tok, question: str, prefix: list[int], tools, max_new: int, system=None) -> str:+ msgs = [{"role": "system", "content": system}] if system else []+ ids = tok.apply_chat_template(msgs + [{"role": "user", "content": question}], tools=tools,+ add_generation_prompt=True, return_tensors="pt",+ enable_thinking=False)["input_ids"][0].tolist()+ x = torch.tensor([prefix + ids], device="cuda")+ out = model.generate(x, max_new_tokens=max_new, do_sample=False,+ cache_implementation="static")+ return out[0, x.shape[1]:].tolist()+++def answer_number(text: str):+ nums = re.findall(r"\d[\d,]*", text.replace("**", ""))+ return int(nums[-1].replace(",", "")) if nums else None+++def main() -> None:+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ ap = argparse.ArgumentParser()+ ap.add_argument("merged")+ ap.add_argument("--base")+ ap.add_argument("--n", type=int, default=10)+ ap.add_argument("--max-new", type=int, default=4000)+ ap.add_argument("--out")+ ap.add_argument("--inline", action="store_true", help="no tools: inline arithmetic only")+ ap.add_argument("--nudge", action="store_true", help="system prompt: write whole calculations")+ ap.add_argument("--hard", action="store_true", help="6-10 digit x 4-7 digit problems")+ args = ap.parse_args()+ meta = json.loads((Path(args.merged) / "chipformer.json").read_text())+ off = meta["chip_token_offset"]+ bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ probs = problems(args.n, hard=args.hard)+ rep = {"merged": [], "base": []}+ model, tok = load(args.merged)+ for q, op, want in probs:+ t = time.time()+ out = ask(model, tok, q, bios, None if args.inline else TOOLS, args.max_new,+ NUDGE if args.nudge else None)+ text = tok.decode(out, skip_special_tokens=False)+ n_chip = sum(1 for i in out if off <= i < off + meta["chip_vocab"])+ shown = re.sub(r"(<\|chip[^|]*\|>)+", " [machine] ", text)+ resp = re.search(r"<tool_response>\s*(\d+)\s*</tool_response>", text)+ tail = text.split("</tool_response>")[-1]+ got = answer_number(tail) if resp else answer_number(text)+ inline = re.findall(r"(?:<\|chip[^|]*\|>)+ (\d+)", text)+ rep["merged"].append({"q": q, "want": want, "tool_result": int(resp[1]) if resp else None,+ "inline_results": [int(v) for v in inline],+ "answer": got, "ok": got == want, "chip_tokens": n_chip,+ "seconds": round(time.time() - t, 1), "text": shown[-400:]})+ print(json.dumps(rep["merged"][-1], ensure_ascii=False), flush=True)+ del model+ if args.base:+ model, tok = load(args.base)+ for q, op, want in probs:+ out = ask(model, tok, q, [], None, args.max_new, NUDGE if args.nudge else None)+ text = tok.decode(out, skip_special_tokens=True)+ got = answer_number(text)+ rep["base"].append({"q": q, "want": want, "answer": got, "ok": got == want})+ print("base", json.dumps(rep["base"][-1]), flush=True)+ for k in ("merged", "base"):+ if rep[k]:+ print(k, "accuracy", sum(r["ok"] for r in rep[k]), "/", len(rep[k]))+ if args.out:+ Path(args.out).write_text(json.dumps(rep, indent=2, ensure_ascii=False))+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/emit.py b/chipformer/q35/emit.pynew file mode 100644index 0000000..dc7d67b--- /dev/null+++ b/chipformer/q35/emit.py@@ -0,0 +1,330 @@+"""Turn a scheduled q35 ``Graph`` into Qwen3.5 weights.++Residual layout of the chip region, starting at ``Layout.off`` (Qwen's own dims come first in a+merged model):++ [ballast x m] [tau] [IR dims 1..n-1] [unused]++With an overlay (overlay.py), circuit dims that live only on chip tokens use Qwen's dims instead,+from ``off - 1`` down, next to the chip region.++Chip tokens carry ``B`` in every ballast dim. With m * B^2 / hidden a power of 4, every RMSNorm+on a chip token divides by exactly sqrt(m / hidden) * B, so a residual value x reaches the+projections as x * c with c = sqrt(hidden / m) / B, a power of 2; weights carry 1 / c. The IR+constant ONE is the normed ballast, B * c.+"""++import math+from dataclasses import dataclass, field++import torch++from chipformer.dsl.graph import ONE, Expr+from chipformer.q35.ir import FFN, HD, Graph, Head, K+++@dataclass+class Layout:+ hidden: int+ off: int = 0 # first chip dim+ m: int = 1 # ballast dims+ log2_ballast: int = 40+ slots: dict[int, int] | None = None # IR dim -> residual slot, when slots are reused+ over: dict[int, int] | None = None # IR dim -> overlay slot (the Qwen dim off - 1 - slot)++ @property+ def ballast(self) -> float:+ return 2.0**self.log2_ballast++ @property+ def scale(self) -> float:+ """c: normed value of a residual value 1 on a chip token."""+ mean = self.m * self.ballast**2 / self.hidden+ log4 = math.log(mean, 4)+ assert abs(log4 - round(log4)) < 1e-12, "m * B^2 / hidden must be a power of 4"+ return 2.0 ** -round(log4)++ @property+ def tau(self) -> int:+ return self.off + self.m++ def dim(self, d: int) -> int:+ """Residual index of IR dim d >= 1."""+ if self.over and d in self.over:+ return self.off - 1 - self.over[d]+ if self.slots is not None:+ return self.off + self.m + 1 + self.slots[d]+ return self.off + self.m + d+++@dataclass+class Alloc:+ """Residual slots shared by IR dims whose lifetimes do not overlap.++ A residual write adds, so a slot is handed over by subtracting its old value: an MLP node+ defining the new dim does it with one extra unit; a head's dim is cleared by a unit in the+ MLP sublayer just before it.+ """++ slots: dict[int, int]+ n_slots: int+ subtract: dict[int, int] = field(default_factory=dict) # FFN-defined dim -> old dim+ clears: dict[int, list[int]] = field(default_factory=dict) # MLP level -> old dims+ final: list[int] = field(default_factory=list) # each slot's last owner+ over: dict[int, int] = field(default_factory=dict) # overlay dims -> overlay slot+ final_over: list[int] = field(default_factory=list) # each overlay slot's last owner+++def reads(n) -> list[int]:+ if isinstance(n, FFN):+ return [d for _, a, b in n.terms for e in (a, b) for d in e.terms]+ exprs = [c.expr for ch in (n.q, n.k) for c in ch.values()]+ for vh in n.vheads():+ exprs += [c.expr for c in vh.v.values()] + [vh.a, *vh.z.values()]+ exprs += [] if vh.beta is None else [vh.beta]+ return [d for e in exprs for d in e.terms]+++def allocate(g: Graph, clearable: set[int] | None, keep: set[int], unit_cap: int,+ overlay: set[int] | None = None, overlay_cap: int = 0) -> Alloc:+ """Assign slots; dims in ``keep`` are never handed over (outputs read by the lm_head).++ A handover clears the old value with one "a-form" unit, -x * relu(ONE), exact for any bf16+ x, so every dim is clearable unless ``clearable`` restricts it. Dims in ``overlay`` go to+ a second pool of at most ``overlay_cap`` slots (Qwen's dims, see overlay.py), and only share+ slots with each other; once it is full they fall back to the chip region.+ """+ top = max(n.level for n in g.nodes) + 1+ last = {d: g.dims[d].level for d in range(1, len(g.dims))}+ for n in g.nodes:+ for d in reads(n):+ if d != ONE:+ last[d] = max(last[d], n.level)+ for d in keep:+ last[d] = top+ units = dict(g.units)+ # a head whose outputs may be tiny (the counter's 2^-j) keeps its slots: the a-form unit+ # that hands a slot over cannot subtract a value that underflows after the norm+ sticky = {d for n in g.nodes if isinstance(n, Head) and n.tiny+ for vh in n.vheads() for _, d, _ in vh.out}+ slots: dict[int, int] = {}+ pools: tuple[list[int], list[int]] = ([], []) # slot -> current dim: chip region, overlay+ a = Alloc(slots, 0)++ def take(d, level, by_ffn):+ for pool in (1, 0) if overlay is not None and d in overlay else (0,):+ owner = pools[pool]+ for s, o in enumerate(owner):+ if o in keep or o in sticky or (clearable is not None and o not in clearable):+ continue+ born = g.dims[o].level # the old value must already be in the sublayer's input+ # (a large old value is not subtracted in the MLP that writes the new one: the+ # sum would round; a clear in the MLP before a head is alone in its slot, so exact)+ if (by_ffn and o not in getattr(g, "big", ()) and last[o] <= level and born < level+ and units.get(level, 0) < unit_cap):+ units[level] = units.get(level, 0) + 1+ a.subtract[d] = o+ elif (not by_ffn and last[o] <= level - 1 and born < level - 1 and level >= 3+ and units.get(level - 1, 0) < unit_cap):+ units[level - 1] = units.get(level - 1, 0) + 1+ a.clears.setdefault(level - 1, []).append(o)+ else:+ continue+ owner[s] = d+ (a.over if pool else slots)[d] = s+ return+ if pool == 0 or len(owner) < overlay_cap:+ (a.over if pool else slots)[d] = len(owner)+ owner.append(d)+ return++ for d in sorted(g.token_fns):+ take(d, 0, False)+ defined = {}+ for n in g.nodes:+ if isinstance(n, FFN):+ if not n.into:+ defined.setdefault(n.level, []).append((n.out, True))+ else: # heads: their own output dims (not the ones they add onto)+ for vh in n.vheads():+ if not vh.into:+ defined.setdefault(n.level, []).extend((d, False) for _, d, _ in vh.out)+ for level in sorted(defined):+ for d, by_ffn in defined[level]:+ take(d, level, by_ffn)+ a.n_slots = len(pools[0])+ a.final = list(pools[0])+ a.final_over = list(pools[1])+ return a+++def bf16_parts(x: float, n: int) -> list[float]:+ """Split x into at most n bf16 values that sum to it exactly."""+ parts = []+ for _ in range(n):+ if x == 0:+ break+ p = torch.tensor(x, dtype=torch.float64).to(torch.bfloat16).double().item()+ parts.append(p)+ x -= p+ if x != 0:+ raise ValueError(f"constant needs more than {n} bf16 parts")+ return parts+++def row(e: Expr, lay: Layout, out: torch.Tensor, w: float = 1.0) -> None:+ """Add w * e as a projection row (input side: normed residual).++ A constant rides on the ballast dims; it is split over them so that each weight is a bf16+ value (the matmul sums the parts exactly in fp32).+ """+ c = lay.scale+ for d, coef in e.terms.items():+ if d == ONE:+ for i, p in enumerate(bf16_parts(w * float(coef) / (lay.ballast * c), lay.m)):+ out[lay.off + i] += p+ else:+ out[lay.dim(d)] += w * float(coef) / c+++def check_bf16(name: str, t: torch.Tensor) -> None:+ b = t.to(torch.bfloat16).to(t.dtype)+ bad = ((b != t) | ~torch.isfinite(b)).nonzero() # (a weight beyond bf16 rounds to inf)+ if len(bad):+ i = tuple(bad[0].tolist())+ raise ValueError(f"{name}: weight {t[i].item()!r} at {i} is not a bf16 value")+++def clearable_dims(g: Graph, traces: list[list[int]]) -> set[int]:+ """Dims a single unit can clear: every value seen is 0 or >= 0.15 (silu is exact there)."""+ import numpy as np++ from chipformer.q35 import ir++ ok = np.ones(len(g.dims), dtype=bool)+ for toks in traces:+ m = ir.evaluate(g, toks, check=False)+ ok &= ((m == 0) | (m >= 0.15)).all(0)+ return {d for d in range(1, len(g.dims)) if ok[d]}+++def emit_layers(g: Graph, lay: Layout, tc: dict, first: int, prefix: str,+ alloc: Alloc | None = None, keep: set[int] | None = None,+ chip: Expr | None = None) -> dict:+ """Weights of the chip layers, named ``{prefix}layers.{first + i}.*``.++ With ``keep``, the MLP sublayer of the nodes named ``exit.*`` also clears every chip dim+ except ``keep`` (and the ballast), so nothing of the circuit is left for later layers.+ Overlay slots hold Qwen's embedding on text tokens: their handovers and exit clears are+ gated by ``chip`` (1 on chip tokens, 0 on text) instead of ONE, so text keeps it.+ """+ h, nk, nv = tc["hidden_size"], tc["linear_num_key_heads"], tc["linear_num_value_heads"]+ inter = tc["intermediate_size"]+ # With more value heads than key heads (4B: 32 vs 16), value head i reads key head+ # i // r; a circuit head uses the first value head of its group and leaves the rest at zero.+ r = nv // nk+ assert r * nk == nv+ f32 = torch.float32+ out = {}+ exit_level = None+ if keep is not None:+ exit_level = max(n.level for n in g.nodes if n.name.startswith("exit."))+ assert all(n.level < exit_level for n in g.nodes if not n.name.startswith("exit."))+ live = (alloc.final + alloc.final_over) if alloc else list(range(1, len(g.dims)))+ exit_clear = [d for d in live if d not in keep]+ over = alloc.over if alloc else {}+ assert chip is not None or not over, "an overlay needs the chip-token gate"+ one = Expr({ONE: 1})+ for L in range(g.n_layers()):+ p = f"{prefix}layers.{first + L}."+ qkv = torch.zeros(2 * nk * HD + nv * HD, h, dtype=f32)+ convw = torch.zeros(2 * nk * HD + nv * HD, 1, tc["linear_conv_kernel_dim"], dtype=f32)+ zw = torch.zeros(nv * HD, h, dtype=f32)+ bw = torch.zeros(nv, h, dtype=f32)+ aw = torch.zeros(nv, h, dtype=f32)+ ow = torch.zeros(h, nv * HD, dtype=f32)+ gate = torch.zeros(inter, h, dtype=f32)+ up = torch.zeros(inter, h, dtype=f32)+ down = torch.zeros(h, inter, dtype=f32)+ heads = [n for n in g.nodes if isinstance(n, Head) and n.level == 2 * L + 1]+ for j, n in enumerate(heads):+ vhs = n.vheads()+ assert len(vhs) <= r, (n.name, len(vhs), r)+ for base, chans in ((j * HD, n.q), (nk * HD + j * HD, n.k)):+ for i, ch in chans.items():+ row(ch.expr, lay, qkv[base + i])+ for lag, w in enumerate(ch.taps):+ convw[base + i, 0, -1 - lag] = w+ for x, vhd in enumerate(vhs):+ vh = j * r + x # value head x of this key head+ base = 2 * nk * HD + vh * HD+ for i, ch in vhd.v.items():+ row(ch.expr, lay, qkv[base + i])+ for lag, w in enumerate(ch.taps):+ convw[base + i, 0, -1 - lag] = w+ # beta = sigmoid(128) = 1, or a write gate: sigmoid(256 g - 128), 1 or exactly 0+ beta = Expr({ONE: 128}) if vhd.beta is None else 256 * vhd.beta - 128 * one+ row(beta, lay, bw[vh])+ row(vhd.a, lay, aw[vh])+ for col, e in vhd.z.items():+ row(e, lay, zw[vh * HD + col])+ for col, d, w in vhd.out:+ ow[lay.dim(d), vh * HD + col] += w+ one = Expr({ONE: 1})++ def clear_gate(d):+ return chip if d in over else one++ units = [] # (coef, a, b, output dim)+ for n in g.nodes:+ if not (isinstance(n, FFN) and n.level == 2 * L + 2):+ continue+ units += [(coef, a, b, n.out) for coef, a, b in n.terms]+ if alloc and n.out in alloc.subtract: # hand the slot over: remove the old value+ old = alloc.subtract[n.out]+ units.append((-1, Expr({old: 1}), clear_gate(old), n.out))+ units += [(-1, Expr({o: 1}), clear_gate(o), o)+ for o in (alloc.clears.get(2 * L + 2, []) if alloc else [])]+ if exit_level == 2 * L + 2:+ units += [(-1, Expr({o: 1}), clear_gate(o), o) for o in exit_clear]+ assert len(units) <= inter, (L, len(units), inter)+ for u, (coef, a, b, d) in enumerate(units):+ row(b, lay, gate[u], K)+ row(a, lay, up[u], 1 / K)+ down[lay.dim(d), u] = float(coef)+ if exit_level == 2 * L + 2: # the ballast goes too: -B per ballast dim+ for i in range(lay.m):+ u = len(units) + i+ row(Expr({ONE: K}), lay, gate[u])+ up[u, lay.off + i] = (lay.ballast / K) / (lay.ballast * lay.scale)+ down[lay.off + i, u] = -1.0+ layer = {+ "input_layernorm.weight": torch.zeros(h, dtype=f32),+ "post_attention_layernorm.weight": torch.zeros(h, dtype=f32),+ "linear_attn.in_proj_qkv.weight": qkv,+ "linear_attn.in_proj_z.weight": zw,+ "linear_attn.in_proj_a.weight": aw,+ "linear_attn.in_proj_b.weight": bw,+ "linear_attn.conv1d.weight": convw,+ "linear_attn.dt_bias": torch.full((nv,), -110.0, dtype=f32),+ "linear_attn.A_log": torch.zeros(nv, dtype=f32),+ "linear_attn.norm.weight": torch.ones(HD, dtype=f32),+ "linear_attn.out_proj.weight": ow,+ "mlp.gate_proj.weight": gate,+ "mlp.up_proj.weight": up,+ "mlp.down_proj.weight": down,+ }+ for k, t in layer.items():+ check_bf16(p + k, t)+ out[p + k] = t+ return out+++def embedding_rows(g: Graph, lay: Layout, vocab: list[int], h: int) -> torch.Tensor:+ """Rows for the chip tokens ``vocab`` (chip token ids, in order)."""+ e = torch.zeros(len(vocab), h, dtype=torch.float32)+ e[:, lay.off : lay.off + lay.m] = lay.ballast+ for d in g.token_fns:+ e[:, lay.dim(d)] = torch.tensor([float(g.feature_value(d, t)) for t in vocab])+ check_bf16("embedding", e)+ return ediff --git a/chipformer/q35/evals.py b/chipformer/q35/evals.pynew file mode 100644index 0000000..e80dac0--- /dev/null+++ b/chipformer/q35/evals.py@@ -0,0 +1,1353 @@+"""Math evals: does the computer in the weights beat the base model?++ python -m chipformer.q35.evals run COND DATA --n 100 [--model PATH] [--out dist/evals]+ python -m chipformer.q35.evals summary [--out dist/evals]++Conditions (same user prompt everywhere; only the model, the tools and the thinking flag change):++ base base Qwen3.5-4B, no tools, non-thinking+ base_think base, no tools, thinking mode (bigger budget)+ merged merged 4B: BIOS boot record + tools; the circuit runs every call (one decode)+ merged_plain merged 4B with no BIOS and no tools (should match ``base``: a fidelity check)+ pytool_inline base + tools; a Python executor writes exactly what the circuit would print+ (same token stream as ``merged``, minus the chip tokens)+ pytool base + tools in the standard agent loop (tool turn in Qwen's chat template):+ the "external calculator" upper bound++Datasets: ``gsm8k`` (openai/gsm8k test), ``gsmhard`` (reasoning-machines/gsm-hard, the same 1319+problems with larger numbers; the same seeded indices are used for both), ``arith`` (synthetic+multi-digit add / mul and mixed expressions). Problem ids are stable, so conditions are paired.++The tool set is ``demo.TOOLS``; ``PY_TOOLS`` gives each tool's exact meaning for checking.+Results go to OUT/DATA.COND.jsonl (one line per problem, resumable) and ``summary`` prints+accuracy (Wilson 95% CI), paired differences (McNemar), tool-call rate, call correctness,+machine exactness, budget cutoffs and timings.+"""++import argparse+import itertools+import json+import math+import os+import random+import re+import sys+import time+from collections import deque+from decimal import Decimal, InvalidOperation+from fractions import Fraction+from pathlib import Path++try: # chip exactness: no bf16 split-K reductions in GEMMs (they silently break the circuit)+ import torch++ torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False+except ImportError: # summary / rescore need no torch+ pass++ROOT = Path(__file__).resolve().parents[2]+sys.set_int_max_str_digits(0) # transcripts can hold runaway digit strings+SUFFIX = "\n\nGive the final answer as a single number in \\boxed{} at the end."+KINDS = ("base", "merged", "merged_inline", "merged_plain", "pytool_inline", "pytool", "oracle",+ "oraclec")+# oracle / oraclec: base Qwen; wherever its text matches the circuit's inline grammar+# (bridge.InlineScan) at "=", the exact result is injected as text " R " (oraclec: " R,RRR ")+BIOS_KINDS = ("merged", "merged_inline") # the prompt starts with the BIOS boot record+TOOL_KINDS = ("merged", "pytool_inline", "pytool")+FLAGS = ("think", "nb", "ne", "nudge", "nudge2", "nudge3", "nudge4", "fp", "tc", "mix", "calc") # suffixes: thinking mode, <|im_start|> banned, nudge,+# ne = <|endoftext|> banned too (Qwen ends its turn with <|im_end|>; after a printed+# </tool_response> it sometimes writes <|endoftext|> instead of the answer)+# nudge3 = the inline nudge with division (or, with mix, products and quotients inline)+# nudge4 = the inline nudge for whole expressions (v18); calc = the calc(expression) tool too+# fp = RoPE positions frozen on chip tokens (BatchDecoder freeze_chip_positions)+# tc = chip tokens invisible to Qwen's layers (BatchDecoder transparent_chip)+CONDS = ("base", "base_think", "merged", "merged_nb", "merged_nudge", "merged_nb_nudge",+ "merged_inline", "merged_inline_nudge", "merged_inline_fp", "merged_inline_nudge_fp", "merged_inline_fp_tc", "merged_inline_nudge_fp_tc",+ "base_nudge", "oracle_nudge", "oraclec_nudge", "oracle_nudge-close", "oracle_nudge2",+ "oracle_nudge2-close", "merged_plain", "pytool_inline", "pytool_inline_nudge", "pytool", "pytool_nudge")+IM_START = 248045+ENDOFTEXT = 248044+NUDGE2 = ("Exact arithmetic: whenever you write a calculation like `a \\times b =`, `a + b =` or "+ "`a - b =` with whole numbers, its exact result is filled in right after the `=`. That "+ "number is final and correct: never write another number after it or recompute it; just "+ "continue your sentence after it. So never do long multiplication or addition by hand: "+ "write the whole calculation and let the result appear.")+NUDGE = ("You have tools for exact arithmetic ({names}). Use them for every arithmetic step "+ "they cover instead of calculating by hand.")+NUDGE3 = ("Exact arithmetic: whenever you write a calculation like `a \\times b =`, `a / b =`, "+ "`a + b =` or `a - b =` with whole numbers, its exact result is filled in right after the "+ "`=` (a quotient that does not end within 10 decimals shows as the reduced fraction `p/q`). So never do long "+ "multiplication or long division by hand: write the whole calculation, e.g. "+ "`918274 / 37 =`, and let the result appear.")+NUDGE4 = ("Exact arithmetic: whenever you write a whole calculation followed by `=`, its exact "+ "value is filled in right after the `=`. It can be one line of whole numbers and "+ "decimals with +, -, \\times, /, ^ and brackets, e.g. `(3.5 + 12) \\times 48 / 7 =` or "+ "`20 \\times (-8) / (-60) =` (a quotient that does not end within 10 decimals shows as "+ "the reduced fraction `p/q`). So never calculate by hand or step by step: write the whole calculation at "+ "once and let the result appear.")+# mix (a tool kind): products inline, sums and differences of big numbers through the tools+NUDGE_MIX = ("Exact arithmetic: whenever you write a multiplication of whole numbers like "+ "`a \\times b =`, its exact result is filled in right after the `=`, so never "+ "multiply long numbers by hand: write the whole product and let the result appear. "+ "For additions and subtractions of large numbers, call the add and sub tools.")+NUDGE_MIX3 = ("Exact arithmetic: whenever you write a multiplication or division of whole numbers "+ "like `a \\times b =` or `a / b =`, its exact result is filled in right after the "+ "`=` (a quotient that does not end within 10 decimals shows as the reduced fraction `p/q`), so never multiply "+ "or divide long numbers by hand: write the whole calculation and let the result "+ "appear. For additions and subtractions of large numbers, call the add and sub tools.")+++def parse_cond(cond: str) -> tuple[str, set]:+ """'merged_nb_nudge' -> ('merged', {'nb', 'nudge'}); a '-tag' (model version) is ignored."""+ parts, flags = cond.split("-")[0].split("_"), set()+ while parts and parts[-1] in FLAGS:+ flags.add(parts.pop())+ kind = "_".join(parts)+ if kind not in KINDS:+ raise argparse.ArgumentTypeError(f"unknown condition {cond}")+ return kind, flags+++def cond_arg(cond: str) -> str:+ parse_cond(cond)+ return cond+++BUDGET = {False: 2048, True: 8192} # Qwen-written tokens (thinking or not); machine tokens extra+THINK_EXTRA = 512 # answer tokens after a forced </think>+THINK_FORCE = ("\n\nConsidering the limited time by the user, I have to give the solution based on "+ "the thinking directly now.\n</think>\n\n")+++def _div(a, b):+ if b == 0:+ raise ZeroDivisionError+ q = Fraction(a) / Fraction(b)+ return q.numerator if q.denominator == 1 else q+++def div_text(a: int, b: int) -> str:+ """How the computer prints a / b (lib/bcd.py div): the exact quotient if it ends within DEC+ decimals, else the reduced fraction "p/q"; "-" first when exactly one of a, b is negative."""+ from chipformer.q35.lib import bcd++ if (a < 0) != (b < 0):+ return "-" + div_text(abs(a), abs(b))+ a, b = abs(a), abs(b)+ if a == 0 and len(str(b)) > 1 + bcd.DEC: # (B is too long: div goes straight to frac)+ return "0/1"+ q, r = divmod(a, b)+ if not r:+ return str(q)+ digs = []+ for _ in range(bcd.DEC):+ r *= 10+ digs.append(str(r // b))+ r %= b+ if not r:+ return f"{q}." + "".join(digs)+ g = math.gcd(a, b)+ return f"{a // g}/{b // g}"+++def _mod(a, b):+ if b == 0:+ raise ZeroDivisionError+ return a % b+++def _gcd(a, b):+ if a.denominator != 1 or b.denominator != 1:+ raise ValueError("gcd of a fraction")+ return math.gcd(int(a), int(b))+++# exact meaning of each tool (by name); demo.TOOLS decides which ones the model sees+PY_TOOLS = {"add": lambda a, b: a + b, "mul": lambda a, b: a * b,+ "sub": lambda a, b: a - b, "div": _div, "mod": _mod, "gcd": _gcd}+SIGNED = ("mul", "div", "gcd") # the computer takes signed a, b for these (bridge.SIGNED)+++def computer_text(op: str, a: int, b: int) -> str | None:+ """Exactly what the computer prints for op(a, b) (inline or as a tool response), or None+ if it does not answer it (signed a or b for add / sub / mod); by 0 it prints "..."."""+ if op not in SIGNED and (a < 0 or b < 0):+ return None+ if op in ("div", "mod") and b == 0: # "..." (div's sign still prints)+ return ("-" if op == "div" and (a < 0) != (b < 0) else "") + "..."+ if op == "div":+ return div_text(a, b)+ if op == "mul": # sign and magnitude: -0 stays "-0"+ return ("-" if (a < 0) != (b < 0) else "") + str(abs(a * b))+ if op == "gcd":+ return str(math.gcd(a, b))+ return str(PY_TOOLS[op](a, b))+++def tools(flags=()):+ from chipformer.q35 import demo++ return demo.TOOLS + ([demo.CALC_TOOL] if "calc" in flags else [])+++def tool_names():+ return [t["function"]["name"] for t in tools(("calc",))]+++def expr_value(s: str):+ """The exact value of an expression as Qwen writes it (LaTeX operators, ^{}, commas between+ digits, "$"), or None."""+ s = re.sub(r"(?<=\d),(?=\d)", "", s.strip(" \t\n,")) # (a sentence's comma before it)+ s = s.replace("\\times", "*").replace("\\cdot", "*").replace("\\div", "/").replace("×", "*")+ s = s.replace("÷", "/").replace("−", "-").replace("{", "(").replace("}", ")").replace("$", "")+ s = s.replace("**", "^").replace("^", "**")+ s = re.sub(r"([\d)])\s*\(", r"\1*(", s)+ if not re.fullmatch(r"[\d.\s+\-*/()]+", s):+ return None+ s = re.sub(r"\d+(?:\.\d*)?", lambda m: f"Fraction('{m[0]}')", s).strip()+ try:+ return eval(s, {"Fraction": Fraction}) # noqa: S307 (digits and operators only)+ except (ZeroDivisionError, SyntaxError, TypeError, ValueError, OverflowError):+ return None+++def value_text(v: Fraction) -> str:+ """How the computer prints a value (eval / div): the digits, "-" first if negative."""+ t = div_text(abs(v.numerator), v.denominator)+ return "-" + t if v < 0 else t+++def calc_text(v: Fraction) -> str:+ """The calc tool's response: the value as eval prints it ("-" first if negative)."""+ return value_text(v)+++# ---------------------------------------------------------------------------------- datasets++def _num(s):+ s = str(s).replace(",", "").strip()+ try:+ v = Fraction(s)+ except (ValueError, ZeroDivisionError):+ return None+ return v+++def load_data(name: str, n: int, seed: int = 0, idx=None) -> list[dict]:+ """``n`` seeded problems, or exactly the problems ``idx`` (dataset indices)."""+ if name == "arith":+ probs = arith(max(idx) + 1 if idx else n, seed)+ return [probs[i] for i in idx] if idx else probs+ from datasets import load_dataset++ if idx is None:+ idx = sorted(random.Random(seed).sample(range(1319), n))+ if name == "gsm8k":+ ds = load_dataset("openai/gsm8k", "main", split="test")+ out = []+ for i in idx:+ r = ds[i]+ gold = _num(r["answer"].split("####")[-1])+ steps = [_num(v) for v in re.findall(r"<<[^=<>]*=([^<>]*)>>", r["answer"])]+ out.append({"id": f"gsm8k-{i}", "question": r["question"], "gold": str(gold),+ "steps": [str(v) for v in steps if v is not None]})+ return out+ if name == "gsmhard":+ ds = load_dataset("reasoning-machines/gsm-hard", split="train")+ out = []+ for i in idx:+ r = ds[i]+ out.append({"id": f"gsmhard-{i}", "question": r["input"],+ "gold": repr(float(r["target"]))})+ return out+ raise ValueError(name)+++ARITH_KINDS = ["add", "mul", "sub", "add3", "muladd", "mulmul", "sumprod", "mulsub"]+++def arith(n: int, seed: int = 0) -> list[dict]:+ """Seeded synthetic arithmetic with +, - (non-negative results) and *.++ Each item lists the calls a perfect solver makes (``calls``: [op, a, b], operands in any+ order) for checking tool-call arguments."""+ rng = random.Random(seed)++ def r(lo, hi):+ d = rng.randint(lo, hi)+ return rng.randrange(10 ** (d - 1), 10**d)++ out = []+ for k in range(n):+ kind = ARITH_KINDS[k % len(ARITH_KINDS)]+ if kind == "add":+ a, b = r(6, 12), r(5, 12)+ q, gold, calls = f"{a} + {b}", a + b, [("add", a, b)]+ elif kind == "mul":+ a, b = r(4, 6), r(3, 5)+ q, gold, calls = f"{a} * {b}", a * b, [("mul", a, b)]+ elif kind == "sub":+ a, b = r(8, 12), r(6, 11)+ a, b = max(a, b), min(a, b)+ q, gold, calls = f"{a} - {b}", a - b, [("sub", a, b)]+ elif kind == "mulsub":+ a, b, c = r(3, 5), r(2, 4), r(3, 6)+ if c > a * b:+ c = c % (a * b)+ q, gold, calls = f"{a} * {b} - {c}", a * b - c, [("mul", a, b), ("sub", a * b, c)]+ elif kind == "add3":+ a, b, c = r(5, 9), r(5, 9), r(4, 9)+ q, gold, calls = f"{a} + {b} + {c}", a + b + c, [("add", a, b), ("add", a + b, c)]+ elif kind == "muladd":+ a, b, c = r(3, 5), r(2, 4), r(4, 8)+ q, gold, calls = f"{a} * {b} + {c}", a * b + c, [("mul", a, b), ("add", a * b, c)]+ elif kind == "mulmul":+ a, b, c = r(2, 4), r(2, 3), r(2, 3)+ q, gold, calls = f"{a} * {b} * {c}", a * b * c, [("mul", a, b), ("mul", a * b, c)]+ elif kind == "sumprod":+ a, b, c, d = r(2, 4), r(2, 3), r(2, 4), r(2, 3)+ q, gold = f"{a} * {b} + {c} * {d}", a * b + c * d+ calls = [("mul", a, b), ("mul", c, d), ("add", a * b, c * d)]+ tmpl = rng.choice(["What is {q}?", "Compute {q}.", "Calculate {q} exactly."])+ out.append({"id": f"arith-{k}", "question": tmpl.format(q=q), "gold": str(gold),+ "kind": kind, "calls": [[o, str(x), str(y)] for o, x, y in calls]})+ return out+++# ---------------------------------------------------------------------------------- parsing++CALL_RE = re.compile(r"<tool_call>\s*<function=([^>\s]*)>(.*?)</function>\s*</tool_call>", re.DOTALL)+PARAM_RE = re.compile(r"<parameter=([^>\s]*)>\s*(.*?)\s*</parameter>", re.DOTALL)+RESP_RE = re.compile(r"\s*(?:<\|im_end\|>\s*<\|im_start\|>user\s*)?<tool_response>\s*(.*?)\s*"+ r"</tool_response>", re.DOTALL)+MARK = re.compile(r"\[machine:\d+\]")+++def parse_calls(text: str) -> list[dict]:+ """Tool calls in order, each with its arguments and the response that follows it."""+ text = MARK.sub("", text)+ calls = []+ for m in CALL_RE.finditer(text):+ args = dict(PARAM_RE.findall(m[2]))+ r = RESP_RE.match(text, m.end())+ calls.append({"op": m[1], "args": args, "response": r[1] if r else None})+ return calls+++def int_arg(s):+ s = (s or "").strip()+ return int(s) if re.fullmatch(r"-?\d+", s) else None+++def num_arg(s):+ try:+ return Fraction(Decimal((s or "").strip().replace(",", "")))+ except (InvalidOperation, ValueError):+ return None+++def fmt(v) -> str:+ if isinstance(v, Fraction):+ if v.denominator == 1:+ return str(v.numerator)+ return str(round(Decimal(v.numerator) / Decimal(v.denominator), 10).normalize())+ return str(v)+++def true_result(op: str, args: dict):+ """Exact value of a call by the tool's definition, or None if it is not a valid call."""+ if op == "calc":+ return expr_value(args.get("expression") or "")+ a, b = args.get("a"), args.get("b")+ if op not in PY_TOOLS or a is None or b is None:+ return None+ x, y = num_arg(a), num_arg(b)+ if x is None or y is None:+ return None+ try:+ return PY_TOOLS[op](x, y)+ except (ZeroDivisionError, ValueError):+ return None+++def _balanced(s: str, i: int) -> str:+ depth, j = 1, i+ while j < len(s) and depth:+ depth += {"{": 1, "}": -1}.get(s[j], 0)+ j += 1+ return s[i:j - 1]+++NUM = re.compile(r"-?\d+(?:\.\d+)?")+++def to_number(s: str):+ s = re.sub(r"\\(?:text|mathrm|textbf|mbox)\{([^{}]*)\}", r" \1 ", s)+ s = re.sub(r"\\d?frac\{([^{}]*)\}\{([^{}]*)\}", r"(\1)/(\2)", s)+ s = s.replace("\\$", " ").replace("$", " ").replace("\\,", "").replace("\\!", "")+ s = s.replace("**", " ").replace("\\%", " ").replace("%", " ")+ s = re.sub(r"(?<=\d),(?=\d{3}\b)", "", s)+ s = re.sub(r"(?<=\d)\s(?=\d{3}\b)", "", s) # 1 234 567+ f = re.fullmatch(r"\s*\(?(-?\d+(?:\.\d+)?)\)?\s*/\s*\(?(-?\d+(?:\.\d+)?)\)?\s*", s)+ if f and float(f[2]) != 0:+ return Fraction(Decimal(f[1])) / Fraction(Decimal(f[2]))+ nums = NUM.findall(s)+ return nums+++def extract_answer(text: str, think: bool):+ """The final answer: the last \\boxed{} after the thinking and outside tool calls, else the+ last number. Returns (value as Fraction or None, where it came from)."""+ if think:+ text = "<think>" + text+ if "<think>" in text and "</think>" not in text.split("<think>")[-1]:+ return None, "unclosed_think"+ text = re.sub(r"<think>.*?</think>", " ", text, flags=re.DOTALL)+ text = re.sub(r"<tool_call>.*?(</tool_call>|$)", " ", text, flags=re.DOTALL)+ text = re.sub(r"<tool_response>.*?(</tool_response>|$)", " ", text, flags=re.DOTALL)+ text = re.sub(r"<\|[^|]*\|>(assistant|user)?", " ", MARK.sub(" ", text))+ boxes = [_balanced(text, m.end()) for m in re.finditer(r"\\boxed\s*\{", text)]+ for src, cand in [("boxed", boxes[-1:]), ("last_number", [text])]:+ for c in cand:+ v = to_number(c)+ if isinstance(v, Fraction):+ return v, src+ if v:+ return Fraction(Decimal(v[0] if src == "boxed" else v[-1])), src+ return None, "none"+++def is_correct(pred, gold: str) -> bool:+ if pred is None:+ return False+ g = Fraction(Decimal(gold)) if "/" not in gold else Fraction(gold)+ if abs(pred - g) <= Fraction(1, 10**6) * max(1, abs(g)):+ return True+ return g.denominator != 1 and abs(pred - g) < Fraction(1, 200) # 2-decimal answers+++# ---------------------------------------------------------------------------------- decoding++class Ctx:+ """What the decode loop needs to know about token ids."""++ def __init__(self, tok, chip=None):+ self.tok = tok+ ids = tok.convert_tokens_to_ids+ self.stop = {ids("<|im_end|>"), ids("<|endoftext|>")}+ self.tc_close = ids("</tool_call>")+ self.tr_close = ids("</tool_response>")+ self.nl = tok("\n", add_special_tokens=False).input_ids[0]+ self.space = tok(" ", add_special_tokens=False).input_ids[0]+ self.digits = {tok(str(d), add_special_tokens=False).input_ids[0] for d in range(10)}+ # v11 closes an inline result with "$", "$$" or "." (one token each)+ self.closers = {i[0] for i in (tok(c, add_special_tokens=False).input_ids+ for c in ("$", "$$", ".")) if len(i) == 1}+ self.chip = chip # (offset, n) or None+ self.think_close = tok.convert_tokens_to_ids("</think>")+ self.think_force = None # tokens that end an over-budget thinking block (run sets it)++ def is_chip(self, t: int) -> bool:+ return self.chip is not None and self.chip[0] <= t < self.chip[0] + self.chip[1]++ def ids(self, s: str) -> list[int]:+ return self.tok(s, add_special_tokens=False).input_ids++ def show(self, out: list[int]) -> str:+ """Decoded text with each run of chip tokens shown as [machine:N]."""+ parts, run, text = [], 0, []+ for t in out + [None]:+ if t is not None and self.is_chip(t):+ if text:+ parts.append(self.tok.decode(text, skip_special_tokens=False))+ text = []+ run += 1+ continue+ if run:+ parts.append(f"[machine:{run}]")+ run = 0+ if t is not None:+ text.append(t)+ if text:+ parts.append(self.tok.decode(text, skip_special_tokens=False))+ return "".join(parts)+++def oracle(ctx: Ctx, cmap: dict, fmt_: str):+ """Hook for the oracle conditions: feeds every token's class to bridge.InlineScan (the+ circuit's capture rule) and, when it fires at "=", returns " R " as text tokens (nothing for+ A - B < 0), so base Qwen sees what the circuit would print, without chip tokens."""+ from chipformer.q35 import bridge as B++ states: dict = {}+ digit = {str(d): ctx.tok(str(d), add_special_tokens=False).input_ids[0] for d in range(10)}+ comma = ctx.tok(",", add_special_tokens=False).input_ids[0]++ def run(out: list[int]) -> list[int]:+ st = states.get(id(out))+ if st is None or st[2] is not out:+ st = states[id(out)] = [B.InlineScan(), 0, out]+ fired = None+ for t in out[st[1]:]:+ fired = st[0].feed(cmap.get(t, B.OTHER))+ st[1] = len(out)+ if not fired:+ return []+ op, a, b = fired+ v = PY_TOOLS[op](int(a), int(b))+ if v < 0:+ return []+ if fmt_ == "{close}": # close the math or the sentence after the number+ text = ctx.tok.decode(out, skip_special_tokens=True)+ if text.count("$$") % 2:+ r = f" {v}$$"+ elif text.replace("$$", "").count("$") % 2:+ r = f" {v}$"+ else:+ r = f" {v}."+ else:+ r = fmt_.format(r=v, rc=f"{v:,}")+ return [ctx.space if c == " " else comma if c == "," else digit[c] if c in digit+ else ctx.tok(c, add_special_tokens=False).input_ids[0] for c in r]++ run.every = True+ return run+++def executor(style: str, ctx: Ctx):+ """Python tool executor for the pytool conditions: called at each </tool_call>, returns the+ tokens to inject."""+ def run(out: list[int]) -> list[int]:+ calls = parse_calls(ctx.show(out))+ if not calls:+ return []+ c = calls[-1]+ v = true_result(c["op"], c["args"])+ res = fmt(v) if v is not None else "error: invalid call"+ if style == "inline": # what the circuit prints: \n<tool_response>\n R \n</tool_response>\n+ # Q35_RESP_STYLE (an experiment): what follows the in-turn response+ st = os.environ.get("Q35_RESP_STYLE", "a")+ if st == "d": # the native user turn, without the <|im_end|>s (EOS)+ return ctx.ids("\n<|im_start|>user\n<tool_response>\n" + res+ + "\n</tool_response>\n<|im_start|>assistant\n<think>\n\n"+ "</think>\n\n")+ if st == "f": # b with "\n\n" spelled as two "\n" tokens (what NL-only can print)+ return ctx.ids("\n<tool_response>\n" + res + "\n</tool_response>") + [+ 198, 248068, 198, 198, 248069, 198, 198]+ tail = {"a": "\n", "b": "\n<think>\n\n</think>\n\n", "c": "\n\nThe result is ",+ "e": "\n\n"}[st]+ return ctx.ids("\n<tool_response>\n" + res + "\n</tool_response>" + tail)+ return ctx.ids("<|im_end|>\n<|im_start|>user\n<tool_response>\n" + res+ + "\n</tool_response><|im_end|>\n<|im_start|>assistant\n<think>\n\n"+ "</think>\n\n")+ return run+++class _Row:+ """One problem in flight: its tokens, the tokens still to force (a Python tool response), and+ the count of Qwen-written tokens (machine tokens, the circuit's print and injected tokens are+ not counted against the budget)."""++ def __init__(self, i: int, n_prompt: int):+ self.i, self.n_prompt, self.t0 = i, n_prompt, time.time()+ self.out: list[int] = []+ self.forced: deque = deque()+ self.n_qwen = self.n_inj = 0+ self.printing = False+ self.stop = None+ self.extra, self.think_forced = 0, False+ self.inj_at: list[int] = [] # where each injection starts in out++ def push(self, t: int, forced: bool, ctx, budget: int, max_len: int, hook) -> bool:+ """Record token t; returns True when the row is finished."""+ self.out.append(t)+ if self.n_prompt + len(self.out) >= max_len - 1:+ self.stop = "context"+ return True+ if forced:+ self.n_inj += 1+ return False+ if t in ctx.stop:+ self.stop = "eos"+ return True+ # the circuit prints "\n<tool_response>\n R \n</tool_response>\n" after a tool call and+ # " R" after an inline "A op B =" (or nothing for A - B < 0); those are not Qwen's tokens+ if ctx.is_chip(t):+ self.printing = True+ elif self.printing is True: # the first text token after a machine run+ self.printing = ("tool" if t == ctx.nl else "inline" if t == ctx.space else False)+ self.n_qwen += not self.printing+ elif self.printing == "tool":+ if t == ctx.tr_close:+ self.printing = "end"+ elif self.printing == "end":+ self.printing = False # the newline after </tool_response> is the circuit's+ elif self.printing == "inline" and t in ctx.digits:+ pass+ elif self.printing == "inline" and (t == ctx.space or t in ctx.closers):+ self.printing = False # v9 closes the number with a space, v11 with $ / $$ / .+ else:+ self.printing = False+ self.n_qwen += 1+ if self.n_qwen >= budget + self.extra:+ if ctx.think_force and not self.think_forced and ctx.think_close not in self.out:+ # thinking hit the budget: close it (Qwen's recommended phrase) and let it answer+ self.forced.extend(ctx.think_force)+ self.extra += THINK_EXTRA+ self.think_forced = True+ return False+ self.stop = "budget"+ return True+ if hook is not None and (getattr(hook, "every", False) or t == ctx.tc_close):+ inj = hook(self.out)+ if inj:+ self.inj_at.append(len(self.out))+ self.forced.extend(inj)+ return False++ def result(self) -> dict:+ return {"out": self.out, "n_qwen": self.n_qwen, "n_injected": self.n_inj,+ "stop": self.stop, "think_forced": self.think_forced, "inj_at": self.inj_at,+ "seconds": time.time() - self.t0}+++def decode_all(bd, ctx, prompts: list[list[int]], budget: int, hook, on_done) -> None:+ """Greedy decoding of every prompt through ``batch.BatchDecoder``'s rows (per-row positions,+ batch-1 prefill, continuous refill), with per-row stop rules and token forcing: after a+ </tool_call>, a ``hook`` (the Python executor) may queue tokens that are fed instead of the+ model's predictions. Calls ``on_done(i, result)`` as each problem finishes."""+ import torch++ queue = deque(range(len(prompts)))+ n = bd.batch+ bd.prefill(prompts[0]) # the batched cache copies a prefilled cache's shapes+ level = bd._prepare(n) if hasattr(bd, "_prepare") else bd._level(n)+ rows, tok, pos, step = level[:4]+ chip = level[4] if len(level) > 4 else None # step for all-chip-token inputs (skip_qwen)+ # fp: RoPE positions frozen on chip tokens (a separate per-row buffer in BatchDecoder)+ rpos = bd.rpos[n] if getattr(bd, "freeze", False) else None+ first = bd.prefill(prompts[queue[0]])+ last = [0] * n # each row's input token+ live: list[_Row | None] = [None] * n++ def admit(b, first=None):+ while queue:+ i = queue.popleft()+ t = bd.prefill(prompts[i]) if first is None else first+ first = None+ r = _Row(i, len(prompts[i]))+ if r.push(t, False, ctx, budget, bd.max_len, hook):+ on_done(i, r.result())+ continue+ rows.load_row(b, bd.small, len(prompts[i]))+ bd.mark_prompt(n, b, prompts[i]) # _tc: its chip slots+ tok[b, 0] = t+ pos[b] = len(prompts[i])+ if rpos is not None:+ rpos[b] = sum(not ctx.is_chip(x) for x in prompts[i])+ live[b], last[b] = r, t+ return+ live[b] = None+ tok[b, 0] = 0 # idle row: decodes garbage at slot 0+ pos[b] = 0++ admit(0, first)+ for b in range(1, n):+ admit(b)+ with torch.no_grad():+ while any(r is not None for r in live):+ busy = [b for b in range(n) if live[b] is not None]+ use_chip = chip is not None and all(ctx.is_chip(last[b]) for b in busy)+ pred = (chip if use_chip else step)(tok, pos).tolist()+ pos.add_(1)+ if rpos is not None: # the input was text: the next RoPE position moves on+ rpos.add_(bd._is_text(tok[:, 0]))+ nt, done = [], []+ for b, p in enumerate(pred):+ r = live[b]+ if r is None:+ nt.append(0)+ continue+ forced = bool(r.forced)+ t = r.forced.popleft() if forced else p+ nt.append(t)+ if r.push(t, forced, ctx, budget, bd.max_len, hook):+ done.append(b)+ tok.copy_(torch.tensor(nt, device=tok.device)[:, None])+ last = nt+ idle = [b for b in range(n) if live[b] is None]+ if idle:+ pos[idle] = 0+ if rpos is not None:+ rpos[idle] = 0+ for b in done:+ on_done(live[b].i, live[b].result())+ admit(b)+++# ---------------------------------------------------------------------------------- scoring++def _key(op, a, b):+ return (a, b) if op in ("sub", "div", "mod") else frozenset((a, b))+++USER_TURN = re.compile(r"<\|im_start\|>\s*user(?!\s*<tool_response>)")+++# "A op B =" with optional signs ("-5", "(-5)"), or "gcd(A, B) ="+INLINE_RE = re.compile(r"(?P<sa>(?<![\w)\]])[-−]|\(-)?(?P<a>\d[\d,]*)\)?[\s$(]*"+ r"(?P<op>\+|-|−|×|\*|\\times|\\cdot|/|÷|\\div|\\bmod|\\mod|mod|%)[\s$]*"+ r"\(?(?P<sb>[-−])?(?P<b>\d[\d,]*)\)?[\s$)]*=\s*$")+GCD_RE = re.compile(r"gcd\(\s*(?P<sa>[-−])?(?P<a>\d[\d,]*)\s*,\s*(?P<sb>[-−])?(?P<b>\d[\d,]*)"+ r"\s*\)\s*=\s*$")+# v18: the expression the circuit took (the text since a breaker; bridge.ExprScan decides)+EXPR_RE = re.compile(r"((?:[\d.,+\-−×*/÷()^{}$\s]|\\times|\\cdot|\\div)+?)\s*=\s*$")+INLINE_OPS = {"+": "add", "-": "sub", "−": "sub", "×": "mul", "*": "mul",+ "\\times": "mul", "\\cdot": "mul", "/": "div", "÷": "div", "\\div": "div",+ "\\bmod": "mod", "\\mod": "mod", "mod": "mod", "%": "mod"}+++def inline_expr(before: str):+ """The calculation the text before a machine run ends with: (match, op, a, b) or None."""+ tail = before[-200:]+ e, op = GCD_RE.search(tail), "gcd"+ if not e:+ e = INLINE_RE.search(tail)+ op = INLINE_OPS[e["op"]] if e else None+ if not e:+ return None++ def num(sign, digits):+ return -int(digits.replace(",", "")) if sign else int(digits.replace(",", ""))++ return e, op, num(e["sa"], e["a"]), num(e["sb"], e["b"])+++def inline_fires(text: str, pred) -> list[dict]:+ """Every machine run outside a tool call: the "A op B =" before it (as written), the digits+ right after it, the true value, and whether the result reappears later / is the answer.++ The circuit prints exactly the digits of the result (nothing for A - B < 0) and hands back;+ Qwen may append digits of its own (v8 prints no closing space). So the machine's part is the+ first len(true) digits: ``exact`` = the digits start with the true value, ``appended`` = the+ digits Qwen added after it (for A - B < 0, everything shown is Qwen's)."""+ out = []+ # v14 quotients: digits, then "." and decimals, then "..." if inexact+ for m in re.finditer(r"\[machine:\d+\]( ?-?)((?:\d|,(?=\d))*(?:\.\d+(?:\.\.\.)?|/\d+)?)", text):+ before = MARK.sub("", text[:m.start()])+ if before.rstrip().endswith("</tool_call>") or m.end() == len(text):+ continue # a tool call, or a run cut off by the budget / context before its print+ x = inline_expr(before)+ # v13+ prints A - B < 0 as " -digits" (before: nothing); v16 also signed mul / div;+ # v18 prints "..." when it can't+ shown = ("-" if "-" in m[1] else "") + m[2].replace(",", "")+ if not shown and text[m.end():].startswith("..."):+ shown = "..."+ machine = shown+ f = {"expr": x[0][0].strip() if x else before[-40:], "printed": shown[:60]}+ t = computer_text(*x[1:]) if x else None+ ex = EXPR_RE.search(before[-300:]) # v18: a whole expression (more than one operator)+ # The single-op path wins when its "A op B =" is the whole window: nothing of an+ # expression (a digit, ".", an operator, ")") right before it+ lead = before[-200:][:x[0].start()].rstrip(" ($") if x else "" # (also "*(7 - 4)")+ single = x and not re.search(r"([\d.)+\-*/×÷−^]|\\times|\\cdot|\\div)$", lead)+ if ex and not single and (+ re.search(r"[()^{}]|\d\.\d", ex[1]) or len(re.findall(+ r"[+\-*/×÷−]|\\times|\\cdot|\\div", ex[1])) >= 2) and (+ v := expr_value(ex[1])) is not None:+ t = value_text(v)+ exact = shown.startswith(t)+ machine = t if exact else shown+ f |= {"expr": ex[0].strip(), "op": "expr", "true": t, "exact": exact,+ "gave_up": shown == "...",+ "appended": shown[len(t):][:60] if exact else ""}+ elif t is not None:+ exact = shown.startswith(t)+ machine = t if exact else shown+ f |= {"op": x[1], "a": x[2], "b": x[3], "true": t, "exact": exact,+ "appended": shown[len(t):][:60] if exact else ""}+ later = MARK.sub("", text[m.end():]).replace(",", "")+ # a number Qwen writes right after the print (v9 " R ", v11 " R$" / " R$$" / " R.")+ ech = re.match(r"(?:\$\$|\$|\.)?[ \t]*\$?[ \t]*(\d+)", later)+ f["echo"] = (None if not ech or not machine or f.get("appended") else+ "same" if ech[1] == machine else "diff")+ f["reused"] = bool(machine) and machine in later+ f["is_answer"] = (bool(machine) and machine.lstrip("-") != "..." and len(machine) < 60+ and pred is not None+ and (Fraction(machine) if "/" in machine else+ Fraction(Decimal(machine.removesuffix("...")))) == pred)+ out.append(f)+ return out+++def judge(p: dict, text: str, think: bool) -> dict:+ """Score a transcript: the answer, its correctness, and every tool call. A user turn that+ Qwen writes itself (after a tool response, instead of an assistant turn) ends the answer:+ a chat runtime would hand the turn back there."""+ full = text+ m = USER_TURN.search(text)+ lenient = None+ if m: # lenient: a boxed answer Qwen wrote inside the user turn it made up+ rest = text[m.start():]+ boxes = [_balanced(rest, b.end()) for b in re.finditer(r"\\boxed\s*\{", rest)]+ v = to_number(boxes[-1]) if boxes else None+ lenient = v if isinstance(v, Fraction) else (Fraction(Decimal(v[0])) if v else None)+ text = text[:m.start()]+ pred, src = extract_answer(text, think)+ fake_resp = False+ if pred is None and lenient is None:+ # lenient: Qwen's last number, counting <tool_response> blocks it wrote itself (with+ # <|im_start|> banned it sometimes answers inside one instead of continuing)+ own = re.sub(r"<tool_call>.*?(</tool_call>|$)", " ", MARK.sub(" ", text), flags=re.DOTALL)+ own = re.sub(r"<\|[^|]*\|>", " ", own)+ nums = NUM.findall(re.sub(r"(?<=\d),(?=\d{3}\b)", "", own))+ if nums:+ lenient, fake_resp = Fraction(Decimal(nums[-1])), True+ gold_steps = {Fraction(s) for s in p.get("steps", [])} | {Fraction(Decimal(p["gold"]))}+ want_calls = [(o, _key(o, Fraction(x), Fraction(y))) for o, x, y in p.get("calls", [])]+ rec_calls = []+ for c in parse_calls(text):+ v = true_result(c["op"], c["args"])+ ia, ib = int_arg(c["args"].get("a")), int_arg(c["args"].get("b"))+ # valid: integer arguments the computer answers (v16: signed for mul, div and gcd)+ want = (computer_text(c["op"], ia, ib) if c["op"] in tool_names() and ia is not None+ and ib is not None else None)+ valid = want is not None+ if c["op"] == "calc":+ valid = v is not None+ want = calc_text(v) if valid else None+ resp = (c["response"] or "").strip()+ rv = num_arg(resp) if resp else None+ e = {"op": c["op"], "a": c["args"].get("a"), "b": c["args"].get("b"), "valid": valid,+ "response": c["response"], "true": fmt(v) if v is not None else None,+ "exact": (resp == want if valid+ else rv is not None and v is not None and rv == v)}+ if c["op"] == "calc":+ e["a"] = c["args"].get("expression")+ if "steps" in p and v is not None:+ e["in_gold_steps"] = v in gold_steps+ if want_calls and v is not None:+ key = _key(c["op"], num_arg(c["args"]["a"]), num_arg(c["args"]["b"]))+ e["planned"] = (c["op"], key) in want_calls+ rec_calls.append(e)+ fires = inline_fires(full, pred)+ return {"pred": fmt(pred) if pred is not None else None, "source": src, "inline": fires,+ "correct": is_correct(pred, p["gold"]), "user_turn": bool(m), "fake_resp": fake_resp,+ "correct_lenient": is_correct(pred, p["gold"]) or is_correct(lenient, p["gold"]),+ "calls": rec_calls}+++def score(p: dict, cond: str, ctx: Ctx, r: dict, seconds: float, think: bool) -> dict:+ out = r["out"]+ if parse_cond(cond)[0] in ("oracle", "oraclec") and r.get("inj_at"):+ # mark each injection like a machine run, so inline_fires reads it the same way+ cuts = [0, *r["inj_at"], len(out)]+ text = "[machine:0]".join(ctx.show(out[a:b]) for a, b in itertools.pairwise(cuts))+ else:+ text = ctx.show(out)+ return {"id": p["id"], "cond": cond, "question": p["question"], "gold": p["gold"],+ **judge(p, text, think), "stop": r["stop"],+ "n_qwen": r["n_qwen"], "n_chip": sum(1 for t in r["out"] if ctx.is_chip(t)),+ "n_out": len(r["out"]), "n_injected": r.get("n_injected", 0),+ "think_forced": r.get("think_forced", False),+ "seconds": round(seconds, 2), "text": text,+ "ids": r["out"] if ctx.chip else None} # generated ids (merged: for circuit replays)+++def rescore(args) -> None:+ """Re-judge every stored transcript with the current scoring code (no model needed)."""+ for f in sorted(Path(args.out).glob("*.jsonl")):+ data, cond = f.name.split(".")[:2]+ recs = [json.loads(line) for line in f.read_text(encoding="utf-8").splitlines() if line]+ idx = sorted({int(r["id"].rsplit("-", 1)[1]) for r in recs})+ probs = {p["id"]: p for p in load_data(data, 0, idx=idx)}+ changed = 0+ for r in recs:+ new = judge(probs[r["id"]], r["text"], "think" in parse_cond(cond)[1])+ changed += new["correct"] != r["correct"]+ r.update(new)+ f.write_text("".join(json.dumps(r, ensure_ascii=False) + "\n" for r in recs),+ encoding="utf-8")+ print(f"{f.name}: {len(recs)} records, {changed} changed verdicts")+++# ---------------------------------------------------------------------------------- running++def default_model(cond: str) -> str:+ if cond.startswith("merged"):+ return str(ROOT / "dist" / "q4-v5")+ snaps = sorted(Path.home().glob(".cache/huggingface/hub/models--Qwen--Qwen3.5-4B/snapshots/*"))+ return str(snaps[-1])+++def load_model(mpath):+ """The checkpoint in bf16: Qwen3.5 as shipped (with its vision tower), or a text-only export+ (textonly.py, Qwen3_5ForCausalLM)."""+ import torch+ from transformers import AutoModelForCausalLM, AutoModelForImageTextToText++ arch = json.loads((Path(mpath) / "config.json").read_text()).get("architectures", [""])[0]+ cls = AutoModelForCausalLM if arch.endswith("ForCausalLM") else AutoModelForImageTextToText+ return cls.from_pretrained(mpath, dtype=torch.bfloat16)+++def build_prompt(tok, question: str, cond: str, bios: list[int]) -> list[int]:+ kind, flags = parse_cond(cond)+ msgs = [{"role": "user", "content": question + SUFFIX}]+ if flags & {"nudge", "nudge2", "nudge3", "nudge4", "mix"}: # tools: use them; others: inline+ if "nudge4" in flags: # whole expressions (v18)+ nudge = NUDGE4+ elif "nudge3" in flags: # with division+ nudge = NUDGE_MIX3 if "mix" in flags else NUDGE3+ elif "mix" in flags:+ nudge = NUDGE_MIX+ elif "nudge2" in flags: # the inline nudge, plus: the filled-in number is final+ nudge = NUDGE2+ elif kind in TOOL_KINDS:+ nudge = NUDGE.format(names=", ".join(t["function"]["name"] for t in tools(flags)))+ else:+ from chipformer.q35 import demo++ nudge = demo.NUDGE+ msgs.insert(0, {"role": "system", "content": nudge})+ ids = tok.apply_chat_template(msgs, tools=tools(flags) if kind in TOOL_KINDS else None,+ add_generation_prompt=True, return_tensors="pt",+ enable_thinking="think" in flags)["input_ids"][0].tolist()+ return (bios if kind in BIOS_KINDS else []) + ids+++class BannedHead(torch.nn.Module):+ """The output head with -inf added at banned token ids (generation_config bad_words_ids of+ length 1), so the argmax inside the compiled step never picks them."""++ def __init__(self, head, banned: list[int], n: int):+ super().__init__()+ self.inner, self.weight = head, head.weight+ mask = torch.zeros(n, dtype=head.weight.dtype, device=head.weight.device)+ mask[banned] = float("-inf")+ self.register_buffer("mask", mask)++ def forward(self, h):+ return self.inner(h) + self.mask+++SELFTEST = [("mul", 4, 4), ("mul", 12345, 678), ("add", 987654321, 123456789),+ ("sub", 1000000, 1), ("sub", 5000, 4321), ("div", 1000000, 7), ("div", 8352, 48),+ ("div", -1418261, -408), ("mul", -63075, 8808), ("sub", 5, 675),+ ("mod", 13201915, 1785), ("gcd", 1785, 13201915),+ ("calc", "4^3 - (12 + 7) * 3", None), ("calc", "20 * (-8) / (-60) * 329", None),+ ("calc", "1.25 / (3 - 0.5)", None),+ ("calc", "25 - 130 - 7", None)] # negative, a chain: short (general ones run ~6k)+++def selftest(bd, ctx, tok, bios) -> None:+ """Before a merged run: Qwen calls each tool on known operands and every call it makes+ must print the exact result. Catches a BIOS that does not match the circuit (the routine+ entry points are compiled into the weights) or inexact kernels."""+ from chipformer.q35.lib import bcd++ sym = {"add": "+", "mul": "*", "sub": "-", "div": "/", "mod": "mod"}+ calc = "eval" in bcd.OPS # v18+ cases = [c for c in SELFTEST if c[0] in tool_names() and (c[0] != "calc" or calc)]+ prompts = [build_prompt(tok, f"Use the calc tool to compute {a}." if op == "calc" else+ f"Use the {op} tool to compute "+ + (f"gcd({a}, {b})." if op == "gcd" else f"{a} {sym[op]} {b}."),+ "merged_calc" if calc else "merged", bios) for op, a, b in cases]+ got = {}+ decode_all(bd, ctx, prompts, 400, None, lambda i, r: got.__setitem__(i, r))+ bad, n = [], 0+ for i, (op, a, b) in enumerate(cases):+ for c in parse_calls(ctx.show(got[i]["out"])):+ v = true_result(c["op"], c["args"])+ if v is None:+ continue+ if c["op"] == "calc":+ want = calc_text(v)+ else:+ ia, ib = int_arg(c["args"].get("a")), int_arg(c["args"].get("b"))+ want = (computer_text(c["op"], ia, ib) if ia is not None and ib is not None+ else None)+ if want is None:+ continue+ n += 1+ if (c["response"] or "").strip() != want:+ bad.append((c["op"], c["args"], c["response"], want))+ print(f"selftest: {n} calls, {len(bad)} wrong {bad}", flush=True)+ if bad or n == 0:+ raise SystemExit("selftest failed: the BIOS does not match this checkpoint's circuit?")+++def run(args) -> None:+ import torch+ from transformers import AutoTokenizer++ cond = args.cond + (f"-{args.tag}" if args.tag else "")+ kind, flags = parse_cond(cond)+ think = "think" in flags+ probs = load_data(args.data, args.n, args.seed)+ out_dir = Path(args.out)+ out_dir.mkdir(parents=True, exist_ok=True)+ path = out_dir / f"{args.data}.{cond}.jsonl"+ done = set()+ if path.exists():+ done = {json.loads(line)["id"] for line in path.read_text(encoding="utf-8").splitlines()+ if line.strip()}+ todo = [p for p in probs if p["id"] not in done]+ if args.shard:+ k, m = map(int, args.shard.split("/"))+ todo = todo[k::m]+ print(f"{path}: {len(done)} done, {len(todo)} to go", flush=True)+ if not todo:+ return+ mpath = args.model or default_model(cond)+ tok = AutoTokenizer.from_pretrained(mpath)+ model = load_model(mpath).cuda().eval()+ chip, bios, bios_id = None, [], None+ meta_path = Path(mpath) / "chipformer.json"+ if meta_path.exists():+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ meta = json.loads(meta_path.read_text())+ chip = (meta["chip_token_offset"], meta["chip_vocab"])+ if kind in BIOS_KINDS:+ import hashlib++ bios = [t + chip[0] for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ bios_id = f"{hashlib.sha1(bcd.ROM).hexdigest()[:10]} ({len(bcd.ROM)} B, OPS {bcd.OPS})"+ print(f"BIOS {bios_id}, boot record {len(bios)} tokens", flush=True)+ ctx = Ctx(tok, chip)+ budget = args.budget or BUDGET[think]+ if think:+ ctx.think_force = ctx.ids(THINK_FORCE)+ hook = {"pytool_inline": executor("inline", ctx),+ "pytool": executor("template", ctx)}.get(kind)+ if kind in ("oracle", "oraclec"):+ from chipformer.q35.spikes_replay import class_map++ cmeta = json.loads((Path(args.class_model) / "chipformer.json").read_text())+ fmt_ = args.oracle_fmt or (" {rc} " if kind == "oraclec" else " {r} ")+ print(f"oracle format {fmt_!r}", flush=True)+ hook = oracle(ctx, class_map(tok, cmeta), fmt_)+ banned = []+ if "nb" in flags or kind == "merged_inline":+ gc = Path(mpath) / "generation_config.json"+ bad = json.loads(gc.read_text()).get("bad_words_ids", []) if gc.exists() else []+ banned = sorted({w[0] for w in bad if len(w) == 1} | {IM_START}+ | ({ENDOFTEXT} if "ne" in flags else set()))+ model.lm_head = BannedHead(model.lm_head, banned, model.lm_head.weight.shape[0])+ print(f"banned token ids {banned}", flush=True)++ def emit(rec):+ with path.open("a", encoding="utf-8") as f:+ f.write(json.dumps(rec, ensure_ascii=False) + "\n")+ print(json.dumps({k: rec[k] for k in ("id", "gold", "pred", "correct", "stop", "n_qwen",+ "n_chip", "seconds")}+ | {"calls": [(c["op"], c["a"], c["b"], c["response"], c["exact"])+ for c in rec["calls"]]}), flush=True)++ prompts = [build_prompt(tok, p["question"], cond, bios) for p in todo]+ import torch._inductor.config++ from chipformer.q35.batch import BatchDecoder++ torch._inductor.config.shape_padding = False # its padded weight copies don't fit 24 GB++ max_len = args.max_len or (max(map(len, prompts)) + budget + (3072 if chip else 1024 if think else 512))+ max_len = -(-max_len // 1024) * 1024+ if args.fast and chip: # exact: chip layers read only their nonzero blocks, and steps on+ from chipformer.q35 import fast # chip-token inputs skip Qwen's (no-op) layers++ print("compact", fast.compact(model, qwen=False), flush=True)+ bd = BatchDecoder(model, batch=min(args.batch, len(todo)), max_len=max_len,+ eos=sorted(ctx.stop), sizes=[min(args.batch, len(todo))],+ compile=not args.no_compile,+ **({"skip_qwen": True} if args.fast and chip else {}),+ **({"freeze_chip_positions": True} if "fp" in flags else {}),+ **({"transparent_chip": True} if "tc" in flags else {}))+ if kind == "merged" and not args.no_selftest: # (tool mode)+ selftest(bd, ctx, tok, bios)+ t0 = time.time()+ print(f"batch {bd.batch}, max_len {max_len}, budget {budget}", flush=True)++ def done(i, r):+ emit(score(todo[i], cond, ctx, r, r["seconds"], think)+ | {"batch": bd.batch, "model": mpath, "bios": bios_id, "banned": banned})++ if bd.skip_qwen: # --fast: chip steps for the rows on chip tokens, prefill ahead (same bits)+ from chipformer.q35.arith_bench import decode_rows++ decode_rows(bd, ctx, prompts, budget, hook, done)+ else:+ decode_all(bd, ctx, prompts, budget, hook, done)+ wall = time.time() - t0+ with (out_dir / "runs.log").open("a", encoding="utf-8") as f:+ f.write(json.dumps({"data": args.data, "cond": cond, "n": len(todo), "batch": bd.batch,+ "wall_s": round(wall, 1), "s_per_problem": round(wall / len(todo), 2),+ "model": mpath, "bios": bios_id,+ "gpu": torch.cuda.get_device_name()}) + "\n")+ print(f"done {len(todo)} in {wall:.0f}s ({wall / len(todo):.1f} s/problem)", flush=True)+++# ---------------------------------------------------------------------------------- summary++def wilson(k: int, n: int, z: float = 1.96):+ if n == 0:+ return (float("nan"),) * 2+ p = k / n+ d = 1 + z * z / n+ c = (p + z * z / (2 * n)) / d+ h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d+ return c - h, c + h+++def mcnemar(b: int, c: int) -> float:+ """Exact two-sided McNemar p-value on discordant counts."""+ n = b + c+ if n == 0:+ return 1.0+ k = min(b, c)+ p = sum(math.comb(n, i) for i in range(k + 1)) / 2**n+ return min(1.0, 2 * p)+++def load_results(out: Path) -> dict:+ res = {}+ for f in sorted(out.glob("*.jsonl")):+ data, cond = f.name.split(".")[:2]+ recs = [json.loads(line) for line in f.read_text(encoding="utf-8").splitlines() if line]+ res[(data, cond)] = {r["id"]: r for r in recs}+ return res+++def failure(r: dict) -> str:+ if r["correct"]:+ return "ok"+ if r["stop"] != "eos":+ return "cutoff"+ if any(c["valid"] and not c["exact"] for c in r["calls"]):+ return "machine_wrong"+ if any(not c["valid"] for c in r["calls"]):+ return "invalid_call"+ if r.get("user_turn"):+ return "user_turn"+ if r.get("fake_resp") and r.get("correct_lenient"):+ return "answer_in_own_tool_response"+ if r["pred"] is None:+ return "no_answer"+ return "qwen_wrong_tools" if r["calls"] else "qwen_wrong"+++PAIRS = [("merged_inline_nudge-v11", "base_nudge"), ("merged_inline_nudge_fp-v11", "base_nudge"),+ ("merged_inline_nudge_fp-v11", "merged_inline_nudge-v11"), ("merged_inline-v11", "base"),+ ("merged_inline_nudge-v11", "oracle_nudge-close"),+ ("merged_inline_fp-v11", "merged_inline-v11"), ("merged_inline_fp-v11", "base"),+ ("oracle_nudge", "base_nudge"), ("oraclec_nudge", "base_nudge"),+ ("oracle_nudge-close", "base_nudge"), ("oracle_nudge2", "base_nudge"),+ ("oracle_nudge2-close", "base_nudge"),+ ("merged_inline_nudge", "oracle_nudge"), ("merged_inline_fp", "merged_inline"), ("merged_inline_nudge_fp", "merged_inline_nudge"),+ ("merged_inline_nudge_fp", "base_nudge"), ("merged_inline", "base"), ("merged_nb-v9", "base"), ("merged_inline", "merged_nb-v9"),+ ("merged_inline-v8", "base"), ("merged_inline_nudge-v8", "base_nudge"), ("merged_inline_nudge", "base"),+ ("merged_inline_nudge", "base_nudge"), ("merged", "base"), ("merged_nb", "base"), ("merged_nb", "merged"),+ ("merged_nudge", "base"), ("merged_nb_nudge", "base"), ("pytool_nudge", "base"),+ ("merged", "pytool_inline"), ("merged", "pytool"),+ ("pytool", "base"), ("pytool_inline", "base"), ("base_think", "base"),+ ("merged", "base_think"), ("merged_plain", "base")]+# pairs whose token streams should match (same Qwen text, merged vs base weights)+SAME_TEXT = [("merged", "pytool_inline"), ("merged_plain", "base")]+++def _plain(r):+ return MARK.sub("", r["text"])+++def _wall(out: Path) -> dict:+ """Wall seconds per problem from runs.log (all rows of a batch share the GPU)."""+ tot = {}+ log = out / "runs.log"+ if log.exists():+ for line in log.read_text(encoding="utf-8").splitlines():+ x = json.loads(line)+ w, n = tot.get((x["data"], x["cond"]), (0.0, 0))+ tot[(x["data"], x["cond"])] = (w + x["wall_s"], n + x["n"])+ return {k: w / n for k, (w, n) in tot.items() if n}+++NL = chr(10)+++def _clean(gold: str) -> bool:+ g = Fraction(Decimal(gold))+ return g >= 0 and g.denominator == 1+++def _looping(fires: list[dict], n: int = 3) -> bool:+ ex = [f["expr"] for f in fires]+ return any(len(set(ex[i:i + n])) == 1 for i in range(len(ex) - n + 1))+++def summary(args) -> None:+ out = Path(args.out)+ res = load_results(out)+ wall = _wall(out)+ lines = [("Accuracy with Wilson 95% CI. tool-call rate = problems with >= 1 call. valid = "+ "the tool exists and both arguments are non-negative integers. machine exact = "+ "printed result == true result (valid calls). args ok = the call is a planned step "+ "(arith) or its result is one of the reference solution's intermediate values "+ "(gsm8k). s/prob = wall time per problem at the run's batch size.")]+ datas = sorted({d for d, _ in res})+ for data in datas:+ conds = [c for c in CONDS if (data, c) in res] + sorted(+ c for d, c in res if d == data and c not in CONDS)+ lines.append(f"\n## {data}\n")+ lines.append("| cond | N | acc | 95% CI | lenient | tool-call rate | calls/prob | "+ "valid calls | machine exact | args ok | cutoffs | Qwen tok | chip tok | "+ "s/prob |")+ lines.append("|---|---|---|---|---|---|---|---|---|---|---|---|---|---|")+ for c in conds:+ rs = list(res[(data, c)].values())+ n, k = len(rs), sum(r["correct"] for r in rs)+ lo, hi = wilson(k, n)+ calls = [x for r in rs for x in r["calls"]]+ valid = [x for x in calls if x["valid"]]+ ex = [x["exact"] for x in valid if x["response"] is not None]+ argok = [x.get("planned", x.get("in_gold_steps")) for x in calls]+ argok = [a for a in argok if a is not None]++ def pct(xs):+ return f"{100 * sum(xs) / len(xs):.0f}% ({sum(xs)}/{len(xs)})" if xs else "-"++ w = wall.get((data, c))+ lines.append(+ f"| {c} | {n} | {100 * k / n:.1f}% | {100 * lo:.0f}-{100 * hi:.0f} | "+ f"{100 * sum(r.get('correct_lenient', r['correct']) for r in rs) / n:.1f}% | "+ f"{pct([bool(r['calls']) for r in rs])} | {len(calls) / n:.2f} | "+ f"{pct([x['valid'] for x in calls])} | {pct(ex)} | {pct(argok)} | "+ f"{sum(r['stop'] != 'eos' for r in rs)} | "+ f"{sum(r['n_qwen'] for r in rs) / n:.0f} | {sum(r['n_chip'] for r in rs) / n:.0f} | "+ f"{f'{w:.1f}' if w else '-'} |")+ inl = [c for c in conds if any(r.get("inline") for r in res[(data, c)].values())]+ if inl:+ lines.append("\nInline arithmetic (machine runs outside tool calls): exact = printed "+ "== value of the \"A op B =\" as written (empty for A - B < 0); "+ "unparsed = no \"A op B =\" found before the run.\n")+ lines.append("| cond | problems with a fire | fires/prob | machine exact | "+ "Qwen digits glued on (after R / A<B fill) | Qwen re-writes a number "+ "(same / different) | unparsed | result reused later | "+ "result is the answer | answer from a fire | "+ "looping (3+ same fires in a row) |")+ lines.append("|---|---|---|---|---|---|---|---|---|---|---|")+ for c in inl:+ rs = list(res[(data, c)].values())+ fs = [f for r in rs for f in r.get("inline", [])]+ ex = [f["exact"] for f in fs if "exact" in f]+ lines.append(+ f"| {c} | {sum(bool(r.get('inline')) for r in rs)}/{len(rs)} | "+ f"{len(fs) / len(rs):.2f} | {sum(ex)}/{len(ex)} | "+ f"{sum(bool(f.get('appended')) and f.get('true', '0')[0] != '-' for f in fs)} / "+ f"{sum(bool(f.get('appended')) and f.get('true', '0')[0] == '-' for f in fs)} | "+ f"{sum(f.get('echo') == 'same' for f in fs)} / "+ f"{sum(f.get('echo') == 'diff' for f in fs)} | "+ f"{sum('exact' not in f for f in fs)} | {sum(f['reused'] for f in fs)}/{len(fs)} | "+ f"{sum(f['is_answer'] for f in fs)}/{len(fs)} | "+ f"{sum(any(f['is_answer'] for f in r.get('inline', [])) for r in rs)}/{len(rs)} | "+ f"{sum(_looping(r.get('inline', [])) for r in rs)} |")+ lines.append("\nFailures (wrong answers by cause):\n")+ for c in conds:+ cnt = _count(failure(r) for r in res[(data, c)].values())+ lines.append(f"- {c}: " + (", ".join(f"{k} {v}" for k, v in sorted(cnt.items())+ if k != "ok") or "none"))+ if data == "gsmhard":+ lines.append(NL + "GSM-Hard subsets: many targets are negative or fractional (label "+ "artifacts of swapping in large numbers); 'clean' = non-negative integer "+ "target." + NL)+ lines.append("| cond | clean acc | clean 95% CI | other acc |")+ lines.append("|---|---|---|---|")+ for c in conds:+ rs = list(res[(data, c)].values())+ cl = [r["correct"] for r in rs if _clean(r["gold"])]+ ot = [r["correct"] for r in rs if not _clean(r["gold"])]+ lo, hi = wilson(sum(cl), len(cl))+ lines.append(f"| {c} | {sum(cl)}/{len(cl)} = {100 * sum(cl) / max(1, len(cl)):.1f}% "+ f"| {100 * lo:.0f}-{100 * hi:.0f} | {sum(ot)}/{len(ot)} |")+ if data == "arith":+ lines.append("\n| cond | " + " | ".join(ARITH_KINDS) + " |")+ lines.append("|---|" + "---|" * len(ARITH_KINDS))+ for c in conds:+ accs = []+ for j, kind in enumerate(ARITH_KINDS):+ rs = [r for r in res[(data, c)].values()+ if int(r["id"].split("-")[1]) % len(ARITH_KINDS) == j]+ accs.append(f"{sum(r['correct'] for r in rs)}/{len(rs)}")+ lines.append(f"| {c} | " + " | ".join(accs) + " |")+ rows = []+ for a, b in PAIRS:+ if (data, a) not in res or (data, b) not in res:+ continue+ ids = sorted(set(res[(data, a)]) & set(res[(data, b)]))+ if not ids:+ continue+ ra, rb = res[(data, a)], res[(data, b)]+ xa = [ra[i]["correct"] for i in ids]+ xb = [rb[i]["correct"] for i in ids]+ n = len(ids)+ bo = sum(x and not y for x, y in zip(xa, xb))+ co = sum(y and not x for x, y in zip(xa, xb))+ d = (bo - co) / n+ se = math.sqrt(max(0.0, (bo + co) / n - d * d) / n)+ same = "-"+ if (a, b) in SAME_TEXT:+ same = f"{sum(_plain(ra[i]) == _plain(rb[i]) for i in ids)}/{n}"+ rows.append(f"| {a} vs {b} | {n} | {100 * sum(xa) / n:.1f}% | "+ f"{100 * sum(xb) / n:.1f}% | {100 * d:+.1f} | "+ f"{100 * (d - 1.96 * se):+.1f}..{100 * (d + 1.96 * se):+.1f} | {bo} | {co} | "+ f"{mcnemar(bo, co):.3g} | {same} |")+ if rows:+ lines.append("\n| paired (A vs B) | N | A | B | A-B (pts) | 95% CI | A only | B only | "+ "McNemar p | same text |")+ lines.append("|---|---|---|---|---|---|---|---|---|---|")+ lines += rows+ text = "\n".join(lines)+ print(text)+ (out / "summary.md").write_text(text + "\n", encoding="utf-8")+++def _count(xs):+ d = {}+ for x in xs:+ d[x] = d.get(x, 0) + 1+ return d+++def main() -> None:+ ap = argparse.ArgumentParser()+ sub = ap.add_subparsers(dest="cmd", required=True)+ r = sub.add_parser("run")+ r.add_argument("cond", type=cond_arg, help=f"one of {', '.join(CONDS)} (or KIND_FLAGS)")+ r.add_argument("data", choices=("gsm8k", "gsmhard", "arith"))+ r.add_argument("--n", type=int, default=100)+ r.add_argument("--seed", type=int, default=0)+ r.add_argument("--model")+ r.add_argument("--out", default=str(ROOT / "dist" / "evals"))+ r.add_argument("--budget", type=int, help="Qwen-written tokens (default 2048, thinking 8192)")+ r.add_argument("--max-len", type=int, help="cache length (default: prompt + budget + room)")+ r.add_argument("--batch", type=int, default=8)+ r.add_argument("--shard", help="k/m: run every m-th remaining problem from k")+ r.add_argument("--no-compile", action="store_true")+ r.add_argument("--no-selftest", action="store_true")+ r.add_argument("--fast", action="store_true", help="merged: compact chip layers + skip_qwen")+ r.add_argument("--tag", help="model tag appended to the condition name (e.g. v6)")+ r.add_argument("--class-model", default=str(ROOT / "dist" / "q4-v9"),+ help="oracle: merged checkpoint whose text classes define the inline grammar")+ r.add_argument("--oracle-fmt", help="oracle: injected text, {r} = result, {rc} = with commas "+ "(default ' {r} ', oraclec ' {rc} '); name the variant with --tag")+ for name in ("summary", "rescore"):+ s = sub.add_parser(name)+ s.add_argument("--out", default=str(ROOT / "dist" / "evals"))+ args = ap.parse_args()+ {"run": run, "summary": summary, "rescore": rescore}[args.cmd](args)+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/fast.py b/chipformer/q35/fast.pynew file mode 100644index 0000000..f16d71a--- /dev/null+++ b/chipformer/q35/fast.py@@ -0,0 +1,295 @@+"""Fast greedy decoding of a q35 checkpoint with stock modules: a static cache plus one+compiled (CUDA-graph) single-token step that also takes the argmax on the GPU.++Pure decoding: every token the machine writes is the model's own greedy prediction; the host+(chipformer.session.Session) only injects the boot prompt, TICKs and key writes, exactly as with+any other model. Long sessions are bounded by ``max_len``; past it the session re-boots from its+mirror of the cells (the trace's own rollover record), which also resets the cache.++ runner = Runner(model, max_len=65536)+ s = runner.session(rom, quirks)+ s.tick(); s.step(); ...+"""++import json+from pathlib import Path++import torch+import torch.nn.functional as F++from chipformer.session import Session+from chipformer.trace import boot_tokens++# --- weight blocks ---------------------------------------------------------------------------+#+# Most of a merged checkpoint's weight bytes are exact zeros: the chip layers use a few hundred+# of their 3584+ input dims and a fraction of their units, Qwen's widened matrices never touch+# the chip dims (bar one mute column), and the output head reads no chip dim past the output+# code. Decoding at batch 1 is bound by reading weights, so reading only each matrix's nonzero+# block is the largest exact speedup there is. The checkpoint itself stays as it is; this only+# changes which kernels run.+++def exact_matmuls() -> None:+ """Make cuBLAS reduce split-K partial sums in fp32 (torch's default allows bf16).++ The circuit's GEMMs are exact only if their fp32 accumulation is: with bf16 partial sums a+ split-K kernel rounds e.g. 128 - 33280 to -33280 before adding 65536. Which kernel cuBLAS+ picks depends on the shapes (batch size, a compacted K), so every exact path sets this.+ """+ torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False+++exact_matmuls() # importing this module makes the process's matmuls safe for the circuit+++def nonzero_cols(w: torch.Tensor, chunk: int = 16384) -> torch.Tensor:+ """``(w != 0).any(0)`` without materializing the whole mask (output heads are large)."""+ nz = torch.zeros(w.shape[1], dtype=torch.bool, device=w.device)+ for i in range(0, w.shape[0], chunk):+ nz |= (w[i:i + chunk] != 0).any(0)+ return nz+++def _hull(mask: torch.Tensor) -> tuple[int, int]:+ idx = mask.nonzero().flatten()+ return (int(idx[0]), int(idx[-1]) + 1) if len(idx) else (0, 0)+++class SparseLinear(torch.nn.Module):+ """``F.linear(x, W)`` for a weight that is zero outside the block ``W[r0:r1, c0:c1]``;+ reads only the block. Equal to the dense product up to the order of the accumulation over+ the block's columns, so exactly equal wherever that sum is exact (all chip arithmetic)."""++ def __init__(self, w: torch.Tensor, rows=None, cols=None, block=None, dense_above=None):+ super().__init__()+ nz = w != 0 if rows is None or cols is None else None+ self.out_features, self.in_features = w.shape+ self.r0, self.r1 = rows or _hull(nz.any(1))+ self.c0, self.c1 = cols or _hull(nz.any(0))+ # dense_above: inputs of more rows than this (prefills) use the dense weight, so they+ # stay bit-identical to the stock model; the block is then a view (no extra memory)+ self.dense_above = dense_above+ self.dense = w if dense_above is not None else None+ if block is None:+ block = w[self.r0:self.r1, self.c0:self.c1]+ if dense_above is None:+ block = torch.nn.Parameter(block.contiguous(), requires_grad=False)+ self.weight = block++ def forward(self, x):+ if self.dense is not None and x.numel() > self.dense_above * x.shape[-1]:+ return F.linear(x, self.dense)+ if self.r1 == self.r0 or self.c1 == self.c0:+ return x.new_zeros(*x.shape[:-1], self.out_features)+ y = F.linear(x[..., self.c0:self.c1], self.weight)+ if self.r0 or self.r1 != self.out_features:+ y = F.pad(y, (self.r0, self.out_features - self.r1))+ return y++ def extra_repr(self):+ return (f"{self.out_features}x{self.in_features} -> rows {self.r0}:{self.r1}, "+ f"cols {self.c0}:{self.c1}")+++class SparseEmbedding(torch.nn.Module):+ """Embedding whose table is zero outside columns ``c0:c1`` (shared with the output head)."""++ def __init__(self, block: torch.nn.Parameter, c0: int, dim: int):+ super().__init__()+ self.weight, self.c0, self.dim = block, c0, dim++ def forward(self, ids):+ e = F.embedding(ids, self.weight)+ return F.pad(e, (self.c0, self.dim - self.c0 - self.weight.shape[1]))+++def chip_meta(model) -> dict | None:+ """The checkpoint's chipformer.json (merged checkpoints only)."""+ meta = Path(getattr(model.config, "_name_or_path", "") or ".") / "chipformer.json"+ return json.loads(meta.read_text()) if meta.exists() else None+++def _n_chip_layers(model) -> int:+ meta = chip_meta(model)+ if meta is not None:+ return meta["chip_layers"]+ cfg = getattr(model.config, "text_config", model.config)+ return cfg.num_hidden_layers # a chip-only checkpoint (tiny.py)++[email protected]_grad()+def compact(model, qwen: bool | str = True, n_chip: int | None = None) -> dict:+ """Replace the text stack's linear maps by their nonzero blocks, in place.++ ``qwen=False`` touches only the chip layers, whose arithmetic is exact, so every logit stays+ bit-identical. ``qwen="rows"`` also drops the zero output rows of Qwen's matrices (the chip+ dims they never write) in decode steps (at most 16 rows): each output is the same dot+ product, so it stays bit-identical too (dist/spikes/b19_qwenblocks.py). ``qwen=True``+ compacts Qwen's layers fully, plus the embedding and the output head: the dropped terms are+ exact zeros, but the shorter sums round differently in Qwen's own (inexact) text arithmetic,+ like any change of GEMM kernel. Returns weight counts.+ """+ exact_matmuls()+ inner = model.model+ lm = getattr(inner, "language_model", inner)+ n_chip = _n_chip_layers(model) if n_chip is None else n_chip+ before = after = 0+ heads = []+ for i, layer in enumerate(lm.layers):+ if i >= n_chip and not qwen:+ continue+ if (i < n_chip and hasattr(layer, "linear_attn")+ and isinstance(layer.linear_attn.out_proj, torch.nn.Linear)):+ heads.append(prune_heads(layer.linear_attn))+ for parent in list(layer.modules()):+ for name, child in list(parent.named_children()):+ if isinstance(child, torch.nn.Linear) and child.bias is None:+ if i >= n_chip and qwen == "rows":+ r0, r1 = _hull((child.weight != 0).any(1))+ if (r0, r1) == (0, child.out_features):+ continue+ s = SparseLinear(child.weight, rows=(r0, r1),+ cols=(0, child.in_features), dense_above=16)+ else:+ s = SparseLinear(child.weight)+ before += child.weight.numel()+ after += s.weight.numel()+ setattr(parent, name, s)+ if qwen is True and isinstance(model.lm_head, torch.nn.Linear):+ w = model.lm_head.weight+ emb = lm.embed_tokens.weight+ c0, c1 = _hull(nonzero_cols(w))+ if w.data_ptr() == emb.data_ptr():+ ce0, ce1 = _hull((emb != 0).any(0))+ c0, c1 = min(c0, ce0), max(c1, ce1)+ block = torch.nn.Parameter(w[:, c0:c1].contiguous(), requires_grad=False)+ before += w.numel()+ after += block.numel()+ if w.data_ptr() == emb.data_ptr():+ lm.embed_tokens = SparseEmbedding(block, c0, w.shape[1])+ model.lm_head = SparseLinear(w, rows=(0, w.shape[0]), cols=(c0, c1), block=block)+ torch.cuda.empty_cache()+ return {"dense_weights": before, "compact_weights": after, "chip_deltanet_heads": heads}++[email protected]_grad()+def prune_heads(la) -> tuple[int, int]:+ """Drop the value heads of a Gated DeltaNet whose output columns are all zero (with the key+ heads only they read). Heads are independent, so the kept heads compute exactly what they+ did; the recurrent state shrinks with them. Returns (kept, original) value heads."""+ nv, nk, dk, dv = la.num_v_heads, la.num_k_heads, la.head_k_dim, la.head_v_dim+ r = nv // nk+ out = la.out_proj.weight+ used = (out.view(out.shape[0], nv, dv) != 0).any(0).any(-1).view(nk, r).any(-1)+ k = used.nonzero().flatten()+ if len(k) == nk:+ return nv, nv+ if len(k) == 0:+ k = torch.zeros(1, dtype=torch.long, device=out.device) # keep one (inert) head+ v = (k[:, None] * r + torch.arange(r, device=k.device)).flatten()+ kd = (k[:, None] * dk + torch.arange(dk, device=k.device)).flatten()+ vd = (v[:, None] * dv + torch.arange(dv, device=k.device)).flatten()+ qkv = torch.cat([kd, la.key_dim + kd, 2 * la.key_dim + vd])++ def keep(mod, idx, dim=0):+ w = mod.weight.index_select(dim, idx).contiguous()+ mod.weight = torch.nn.Parameter(w, requires_grad=False)+ if isinstance(mod, torch.nn.Linear):+ mod.out_features, mod.in_features = w.shape++ keep(la.in_proj_qkv, qkv)+ keep(la.conv1d, qkv)+ la.conv1d.in_channels = la.conv1d.out_channels = la.conv1d.groups = len(qkv)+ keep(la.in_proj_z, vd)+ keep(la.in_proj_a, v)+ keep(la.in_proj_b, v)+ keep(la.out_proj, vd, 1)+ la.A_log = torch.nn.Parameter(la.A_log[v].contiguous(), requires_grad=False)+ la.dt_bias = torch.nn.Parameter(la.dt_bias[v].contiguous(), requires_grad=False)+ la.num_k_heads, la.num_v_heads = len(k), len(v)+ la.key_dim, la.value_dim = len(k) * dk, len(v) * dv+ la.conv_dim = 2 * la.key_dim + la.value_dim+ return len(v), nv+++class Runner:+ def __init__(self, model, max_len: int = 65536, token_offset: int = 0, n_chip: int = 8708,+ compile: bool = True):+ from transformers import StaticCache++ self.model = model+ self.off = token_offset+ self.n_chip = n_chip+ self.max_len = max_len+ self.dev = next(model.parameters()).device+ cfg = getattr(model.config, "text_config", model.config)+ self.cache = StaticCache(config=cfg, max_cache_len=max_len)+ # Run the text stack directly (a multimodal wrapper would recompute vision-aware rope+ # indices every step) and score only the chip rows of the tied output head: in a machine+ # record the predicted token is always a chip token (logit ~240 vs ~0 for text), so the+ # greedy choice is the same as over the full vocabulary.+ inner = model.model+ self.lm = getattr(inner, "language_model", inner)+ lm_head = model.lm_head+ self.head = lm_head.weight[token_offset:token_offset + n_chip]+ self.cols = (slice(lm_head.c0, lm_head.c1) if isinstance(lm_head, SparseLinear)+ else slice(None))+ self.pos = 0+ self.tok = torch.zeros(1, 1, dtype=torch.long, device=self.dev)+ self.cpos = torch.zeros(1, dtype=torch.long, device=self.dev)+ self._step = torch.compile(self._forward, mode="reduce-overhead", fullgraph=False) \+ if compile else self._forward++ def _forward(self, tok, cpos):+ h = self.lm(input_ids=tok, past_key_values=self.cache, cache_position=cpos,+ use_cache=True).last_hidden_state[0, -1]+ return (self.head @ h[self.cols]).argmax(-1)++ @torch.no_grad()+ def prefill(self, toks: list[int]) -> int:+ x = torch.tensor([[t + self.off for t in toks]], device=self.dev)+ cpos = torch.arange(self.pos, self.pos + len(toks), device=self.dev)+ h = self.lm(input_ids=x, past_key_values=self.cache, cache_position=cpos,+ use_cache=True).last_hidden_state[0, -1]+ self.pos += len(toks)+ return int((self.head @ h[self.cols]).argmax())++ @torch.no_grad()+ def feed(self, token: int) -> int:+ self.tok.fill_(token + self.off)+ self.cpos.fill_(self.pos)+ nxt = self._step(self.tok, self.cpos)+ self.pos += 1+ return int(nxt)++ def reset(self) -> None:+ self.cache.reset()+ self.pos = 0++ def session(self, rom: bytes, quirks=None) -> Session:+ """A Session whose boot prompt goes through one prefill; rolls over before max_len."""+ runner = self+ boot: list[int] = []++ class Fast(Session):+ def _generate_record(self):+ if runner.pos > runner.max_len - 200: # room for the longest record+ runner.reset()+ dump = boot_tokens(self.cells)+ self.tokens += dump+ self.pred = runner.prefill(dump)+ return super()._generate_record()++ collecting = [True]++ def feed(tok: int) -> int:+ if collecting[0]:+ boot.append(tok)+ return -1+ return runner.feed(tok)++ s = Fast(feed, rom, quirks)+ collecting[0] = False+ s.pred = runner.prefill(boot)+ return sdiff --git a/chipformer/q35/host.py b/chipformer/q35/host.pynew file mode 100644index 0000000..9e68ea4--- /dev/null+++ b/chipformer/q35/host.py@@ -0,0 +1,61 @@+"""Drive a q35 checkpoint with chipformer.session.Session (prefill + cached decode).++``Feeder.feed(token)`` processes one position through the HF model with its cache and returns the+greedy next token. Prompts can be pushed in one prefill with ``prefill``; the host's TICK and key+injections then go through the normal decode path.+"""++import torch++from chipformer.session import Session+++class Feeder:+ def __init__(self, model, token_ids=None):+ """``token_ids`` maps a chip token to the model's vocabulary (identity for tiny models)."""+ self.model = model+ self.ids = token_ids+ self.chip = None if token_ids is None else torch.tensor(token_ids)+ self.cache = None+ self.pending: list[int] = []+ self.n = 0++ def _id(self, t: int) -> int:+ return t if self.ids is None else self.ids[t]++ def _run(self, toks: list[int]) -> int:+ x = torch.tensor([[self._id(t) for t in toks]], device=self.model.device)+ with torch.no_grad():+ out = self.model(input_ids=x, past_key_values=self.cache, use_cache=True)+ self.cache = out.past_key_values+ self.n += len(toks)+ logits = out.logits[0, -1].float()+ if self.chip is not None:+ return int(logits[self.chip].argmax())+ return int(logits.argmax())++ def feed(self, token: int) -> int:+ return self._run([token])++ def reset(self) -> None:+ self.cache = None+ self.n = 0+++def session(model, rom: bytes, quirks=None, token_ids=None) -> Session:+ """A Session whose boot prompt goes through one prefill, then decodes token by token."""+ f = Feeder(model, token_ids)+ boot: list[int] = []++ def feed(tok: int) -> int:+ if boot is not None and collecting[0]:+ boot.append(tok)+ return -1+ return f.feed(tok)++ collecting = [True]+ s = Session(feed, rom, quirks)+ collecting[0] = False+ s.pred = f._run(boot)+ s.feeder = f+ return sdiff --git a/chipformer/q35/ir.py b/chipformer/q35/ir.pynew file mode 100644index 0000000..0e380ce--- /dev/null+++ b/chipformer/q35/ir.py@@ -0,0 +1,575 @@+"""Circuit graph for the Qwen3.5 target: SwiGLU units and Gated DeltaNet heads.++Like ``chipformer.dsl.graph``, every value is a scalar dim of the residual stream and linear+forms (``Expr``) are free. The node types are the two sublayers of a Qwen3.5 linear-attention+layer, described closely enough that the emitter only has to scale weights:++``FFN`` a sum of units ``coef * a * relu(b)``, emitted as ``silu(K b) * (a / K)``. silu equals+ relu on every value the circuit produces: 0, >= 18 (sigmoid rounds to 1) or <= -104+ (sigmoid underflows to 0).+``Head`` one Gated DeltaNet key head with its value head(s), channel by channel: q/k/v+ channels are linear forms with conv taps over the last 4 tokens followed by silu;+ keys must be one-hot (an exact overwrite of one state row) or zero, queries+ multi-hot or zero; ``a`` picks the per-token decay (keep / halve / clear); ``z``+ gates each output column. Rows carry a ballast in their ``ref`` columns that+ dominates the per-head output RMSNorm, so reads come back exactly rescaled.++Values live in IR units: a residual value x reaches every sublayer as exactly x * 2^-r because+chip tokens carry a large ballast (see spike.py), and the emitter folds 2^r into the weights.+The constant ONE is that ballast. On text tokens every chip dim, ONE included, is 0, so every+node outputs exactly 0 there.++Tokens are ints: chip tokens, text classes, and (for the object store) a text class with a Qwen+token id above it (``qid << 16 | cls``). Token features see ``t & 0xFFFF`` unless declared+``full``, so only features that read Qwen's own token id (the store's id code) can tell two text+tokens of one class apart.++Exactness is checked numerically: ``evaluate`` runs a token sequence through the graph and+asserts that every intermediate the hardware materializes is a bf16 value, that silu inputs are+in its exact range, and that keys, queries and decays have the allowed shapes.+"""++from dataclasses import dataclass, field++import numpy as np++from chipformer.dsl.graph import ONE, Expr, Num, as_expr++K = 128 # silu gate scale: silu(K n) = K relu(n) for integers n+A_REF = 2**24 # ballast value of a state row (two columns)+A_BIT = 2**7 # value of a set bit in a state row: (A_BIT / A_REF)^2 * 126 is absorbed+REF_COLS = (0, 1)+DECAY = {0.0: 1.0, 110.0: 0.5, 238.0: 0.0} # a -> exp(g) with dt_bias = -110, A_log = 0+HD = 128 # key/value head dim+TOKEN_MASK = 0xFFFF # the part of a token that ordinary features see (see above)+++@dataclass+class Dim:+ name: str+ kind: str # "one" | "token" | "ffn" | "head"+ level: int+++@dataclass+class FFN:+ name: str+ terms: list[tuple[Num, Expr, Expr]] # coef * a * relu(b)+ out: int+ level: int+ into: bool = False # out is an existing dim that this node adds to+++@dataclass+class Chan:+ expr: Expr+ taps: tuple[float, float, float, float] = (1.0, 0.0, 0.0, 0.0) # x_t, x_{t-1}, ...+++@dataclass+class VHead:+ """A further value head of a key head (Qwen3.5-4B has two per key head): the same queries+ and keys, its own values, decay, write gate and output gates, so it holds a second table+ with the same rows."""++ v: dict[int, Chan]+ a: Expr+ z: dict[int, Expr]+ out: list[tuple[int, int, float]]+ beta: Expr | None = None # write gate (0/1); None: always 1+ into: bool = False # outputs add onto existing dims+++@dataclass+class Head:+ name: str+ q: dict[int, Chan] # key-dim channel -> query+ k: dict[int, Chan]+ v: dict[int, Chan] # value column -> value (columns REF_COLS are the ballast)+ a: Expr # decay selector, see DECAY+ z: dict[int, Expr] # value column -> output gate silu(z)+ out: list[tuple[int, int, float]] # (value column, residual dim, weight)+ level: int+ n_cols: int = HD+ tiny: float = 0.0 # outputs below this may round (the counter's 2^-j for j >> JMAX)+ beta: Expr | None = None # write gate of the first value head (0/1); None: always 1+ into: bool = False # outputs add onto existing dims (several heads summed into one dim)+ vx: list[VHead] = field(default_factory=list) # further value heads of this key head++ def vheads(self) -> list[VHead]:+ return [VHead(self.v, self.a, self.z, self.out, self.beta, self.into), *self.vx]+++@dataclass+class Target:+ """Per-layer capacity of the host model."""++ key_heads: int = 16+ value_heads: int = 16+ intermediate: int = 3584++ @property+ def cols(self) -> int:+ return HD # value columns a head uses (one value head; the output norm is per head)+++@dataclass+class Graph:+ target: Target = field(default_factory=Target)++ def __post_init__(self) -> None:+ self.dims: list[Dim] = [Dim("one", "one", 0)]+ self.token_fns: dict[int, object] = {}+ self.full_fns: set[int] = set() # features that see the whole token (TOKEN_MASK)+ # dims that can hold large values (> 2^8): their slot is never handed over by a+ # subtraction in the MLP that writes the new value (1 - 2^26 is not exact in fp32)+ self.big: set[int] = set()+ self.nodes: list[FFN | Head] = []+ self.one = Expr({ONE: 1})+ self.units: dict[int, int] = {} # sublayer -> FFN units used+ self.heads: dict[int, int] = {} # sublayer -> key heads used++ # --- dims -----------------------------------------------------------------++ def token_feature(self, name: str, fn, full: bool = False) -> Expr:+ d = self._dim(name, "token", 0)+ self.token_fns[d] = fn+ if full:+ self.full_fns.add(d)+ return Expr({d: 1})++ def feature_value(self, d: int, t: int):+ fn = self.token_fns[d]+ return fn(t) if d in self.full_fns else fn(t & TOKEN_MASK)++ def _dim(self, name: str, kind: str, level: int) -> int:+ self.dims.append(Dim(name, kind, level))+ return len(self.dims) - 1++ def level(self, *exprs: Expr) -> int:+ return max((self.dims[d].level for e in exprs for d in e.terms), default=0)++ # --- placement ------------------------------------------------------------++ def _place(self, lvl: int, odd: bool, need: int, used: dict[int, int], cap: int) -> int:+ sub = lvl + 1+ while (sub % 2 == 1) != odd or used.get(sub, 0) + need > cap:+ sub += 1+ used[sub] = used.get(sub, 0) + need+ return sub++ # --- nodes ----------------------------------------------------------------++ def ffn(self, terms, name: str = "ffn", into: Expr | None = None, min_level: int = 0) -> Expr:+ terms = [(c, as_expr(a), as_expr(b)) for c, a, b in terms if c != 0]+ lvl = max(self.level(*(e for _, a, b in terms for e in (a, b))), min_level)+ sub = self._place(lvl, False, len(terms), self.units, self.target.intermediate)+ if into is not None:+ (d,) = into.terms+ self.dims[d].level = max(self.dims[d].level, sub)+ else:+ d = self._dim(name, "ffn", sub)+ self.nodes.append(FFN(name, terms, d, sub, into is not None))+ return Expr({d: 1})++ def head(self, name: str, q: dict, k: dict, v: dict, a: "Expr | Num", z: dict,+ outs: list[tuple[int, float]], min_level: int = 0, tiny: float = 0.0,+ beta=None, into: list | None = None, vx: list | None = None) -> list[Expr]:+ """Add a head; ``outs`` lists (value column, weight) pairs, one new dim each (or, with+ ``into``, onto these existing dims). ``vx``: further value heads, dicts with keys v, a,+ z, outs and optionally beta / into, sharing this head's queries and keys; their output+ dims follow the first value head's in the returned list."""+ def chans(ch):+ return {i: c if isinstance(c, Chan) else Chan(as_expr(c)) for i, c in ch.items()}++ q, k, v = chans(q), chans(k), chans(v)+ a = as_expr(a)+ z = {i: as_expr(e) for i, e in z.items()}+ beta = None if beta is None else as_expr(beta)+ vx = [dict(x) for x in (vx or [])]+ assert len(vx) < self.target.value_heads // self.target.key_heads or not vx, \+ "more value heads than the key head has"+ for x in vx:+ x["v"], x["a"] = chans(x["v"]), as_expr(x["a"])+ x["z"] = {i: as_expr(e) for i, e in x["z"].items()}+ x["beta"] = None if x.get("beta") is None else as_expr(x["beta"])+ exprs = [c.expr for ch in (q, k, v) for c in ch.values()] + [a, *z.values()]+ exprs += [] if beta is None else [beta]+ for x in vx:+ exprs += [c.expr for c in x["v"].values()] + [x["a"], *x["z"].values()]+ exprs += [] if x["beta"] is None else [x["beta"]]+ lvl = max(self.level(*exprs), min_level)+ sub = self._place(lvl, True, 1, self.heads, self.target.key_heads)++ def out_dims(tag, outs_, into_):+ if into_ is None:+ return [self._dim(f"{name}{tag}[{col}]", "head", sub) for col, _ in outs_]+ assert len(into_) == len(outs_)+ dims_ = []+ for e in into_:+ (d_,) = e.terms+ self.dims[d_].level = max(self.dims[d_].level, sub)+ dims_.append(d_)+ return dims_++ dims = out_dims("", outs, into)+ extra, ret = [], [Expr({d: 1}) for d in dims]+ for i, x in enumerate(vx):+ if isinstance(x.get("into"), str) and x["into"] == "main": # onto the first's dims+ x["into"] = [Expr({d: 1}) for d in dims]+ xd = out_dims(f".v{i + 1}", x["outs"], x.get("into"))+ extra.append(VHead(x["v"], x["a"], x["z"],+ [(col, d, w) for (col, w), d in zip(x["outs"], xd)], x["beta"],+ x.get("into") is not None))+ ret += [Expr({d: 1}) for d in xd]+ self.nodes.append(Head(name, q, k, v, a, z,+ [(col, d, w) for (col, w), d in zip(outs, dims)], sub,+ self.target.cols, tiny, beta, into is not None, extra))+ return ret++ def n_layers(self) -> int:+ top = max((n.level for n in self.nodes), default=0)+ return (top + 1) // 2++ def reschedule(self, unit_cap: int | None = None) -> None:+ """Place every node again by list scheduling: sublayer by sublayer, the ready nodes on+ the longest remaining path go first (the build-time placement is greedy in build order,+ so a node off the critical path can take the room a critical one needed). Nodes named+ ``exit.*`` go to the first MLP sublayer above all others, as at build time.++ A dim several nodes write (``into``) keeps its build-order meaning: a reader sees the+ writes built before it and none built after, and later writers never go below the+ dim's first writer. ``unit_cap`` caps FFN units per sublayer; the default leaves a+ sixteenth of each MLP for the allocator's handover units (emit.allocate), whose+ absence in a full sublayer costs residual slots.+ """+ inter = self.target.intermediate+ cap_units = inter - inter // 16 if unit_cap is None else unit_cap+ nodes = self.nodes+ n = len(nodes)++ def outs(nd):+ if isinstance(nd, FFN):+ return [nd.out]+ return [d for vh in nd.vheads() for _, d, _ in vh.out]++ def reads_of(nd):+ if isinstance(nd, FFN):+ es = [e for _, a, b in nd.terms for e in (a, b)]+ else:+ es = [c.expr for ch in (nd.q, nd.k) for c in ch.values()]+ for vh in nd.vheads():+ es += [c.expr for c in vh.v.values()] + [vh.a, *vh.z.values()]+ es += [] if vh.beta is None else [vh.beta]+ return {d for e in es for d in e.terms if d != ONE}++ writes = [outs(nd) for nd in nodes]+ writers: dict[int, list[int]] = {}+ for i, ws in enumerate(writes):+ for d in ws:+ writers.setdefault(d, []).append(i)+ strict = [set() for _ in range(n)] # i -> nodes that must be strictly below i+ loose = [set() for _ in range(n)] # i -> nodes that must be at i's level or below+ for i, nd in enumerate(nodes):+ for d in reads_of(nd):+ for w in writers.get(d, []):+ if w < i:+ strict[i].add(w)+ elif w > i:+ loose[w].add(i) # a later writer must not show up in i's input+ for d in writes[i]:+ first = writers[d][0]+ if first != i:+ loose[i].add(first)+ exits = {i for i, nd in enumerate(nodes) if nd.name.startswith("exit.")}+ # longest remaining path in sublayers: a successor of the other kind can sit one+ # sublayer up, one of the same kind two (a loose one: in the same sublayer)+ is_head = [isinstance(nd, Head) for nd in nodes]+ succ = [[] for _ in range(n)]+ for i in range(n):+ for p in strict[i]:+ succ[p].append((i, 1 if is_head[i] != is_head[p] else 2))+ for p in loose[i] - strict[i]:+ succ[p].append((i, 1 if is_head[i] != is_head[p] else 0))+ tail = [0] * n+ for i in reversed(range(n)): # build order is topological for these edges+ tail[i] = max((g + tail[s] for s, g in succ[i]), default=0)+ level = [None] * n+ units: dict[int, int] = {}+ heads: dict[int, int] = {}+ todo = set(range(n)) - exits+ sub = 1+ while todo:+ assert sub <= 4 * n + 4, "reschedule: a node never fits"+ odd = sub % 2 == 1+ placed = True+ while placed: # a node placed here can make a later writer of its dim ready here+ placed = False+ cand = {i for i in todo if is_head[i] == odd+ and all(level[p] is not None and level[p] < sub for p in strict[i])}+ ready = set()+ grew = True+ while grew: # loose predecessors: placed already, or ready here too+ grew = False+ for i in cand - ready:+ if all((level[p] is not None and level[p] <= sub) or p in ready+ for p in loose[i]):+ ready.add(i)+ grew = True+ for i in sorted(ready, key=lambda i: (-tail[i], i)):+ if not all(level[p] is not None and level[p] <= sub for p in loose[i]):+ continue # its loose predecessor did not fit here+ if odd:+ if heads.get(sub, 0) + 1 > self.target.key_heads:+ continue+ heads[sub] = heads.get(sub, 0) + 1+ else:+ need = len(nodes[i].terms)+ if units.get(sub, 0) + need > cap_units:+ continue+ units[sub] = units.get(sub, 0) + need+ level[i] = sub+ todo.discard(i)+ placed = True+ sub += 1+ top = max((lv for lv in level if lv is not None), default=0)+ for i in sorted(exits):+ lo = max([top] + [level[p] for p in strict[i]]) + 1+ lo += lo % 2 # an MLP sublayer+ level[i] = lo+ units[lo] = units.get(lo, 0) + len(nodes[i].terms)+ assert len({level[i] for i in exits}) <= 1, "exit nodes in different sublayers"+ for i, nd in enumerate(nodes):+ nd.level = level[i]+ for d, ws in writers.items():+ self.dims[d].level = max(level[w] for w in ws)+ self.units, self.heads = units, heads++ def stats(self) -> dict[str, int]:+ return {+ "layers": self.n_layers(),+ "dims": len(self.dims),+ "heads": sum(isinstance(n, Head) for n in self.nodes),+ "ffn_units": sum(len(n.terms) for n in self.nodes if isinstance(n, FFN)),+ "max_units": max(self.units.values(), default=0),+ "max_heads": max(self.heads.values(), default=0),+ }+++# --- exactness checks -------------------------------------------------------------+#+# The evaluator computes every node twice: ideally (exact float64) and as the bf16 hardware does+# (every materialized tensor rounded to bf16, sums accumulated in fp32). The two must agree on+# every node output; fp32 hardware has no rounding and computes the ideal values.+++class ExactnessError(AssertionError):+ pass+++def bf16(x: np.ndarray) -> np.ndarray:+ """Round to the nearest bf16 value (ties to even), returned as float64."""+ f = np.asarray(x, dtype=np.float32).copy()+ u = f.view(np.uint32)+ u += 0x7FFF + ((u >> 16) & 1)+ u &= 0xFFFF0000+ return f.astype(np.float64)+++def is_bf16(x: np.ndarray) -> np.ndarray:+ return (bf16(x) == x) & np.isfinite(x)+++def require(ok, what: str, x) -> None:+ ok, x = np.asarray(ok), np.asarray(x)+ if not np.all(ok):+ t = np.unravel_index(int(np.argmin(ok)), ok.shape)+ raise ExactnessError(f"{what}: bad value {x[t]!r} at {t}")+++def silu_ok(x: np.ndarray) -> np.ndarray:+ """silu(x) == relu(x) exactly (fp32 and bf16)."""+ return (x == 0) | (x >= 18) | (x <= -104)+++def dead(x: np.ndarray) -> np.ndarray:+ """x is so negative that silu gives exactly 0 however the hardware rounds it (a gate or key+ switched off on text tokens by overlay.py: x need not be a bf16 value)."""+ return bf16(x) <= -104+++# --- evaluation ---------------------------------------------------------------------+++def ev(e: Expr, m: np.ndarray) -> np.ndarray:+ if not e.terms:+ return np.zeros(m.shape[0])+ idx = np.fromiter(e.terms.keys(), dtype=np.int64, count=len(e.terms))+ coef = np.array([float(c) for c in e.terms.values()])+ return m[:, idx] @ coef+++def conv(c: Chan, m: np.ndarray, check: bool, carry: dict | None = None,+ key=None) -> np.ndarray:+ """silu(conv(projection)) as computed in bf16; must equal the exact value. With ``carry``+ the projection's last values of the previous chunk come first (see evaluate)."""+ x = ev(c.expr, m)+ # without lag taps a dead projection only reaches its own silu+ off = dead(c.taps[0] * bf16(x)) if not any(c.taps[1:]) else np.zeros(len(x), bool)+ if check:+ require(is_bf16(x) | off, "projection", x)+ n = len(x)+ if carry is not None and any(c.taps[1:]):+ prev = carry.get(key, np.zeros(0))+ xx = np.concatenate([prev, x])+ carry[key] = xx[-3:].copy()+ else:+ xx = x+ p = len(xx) - n+ out = np.zeros(len(xx))+ for lag, w in enumerate(c.taps):+ if w:+ out[lag:] += w * xx[: len(xx) - lag]+ out = out[p:]+ if check:+ require((is_bf16(out) & silu_ok(out)) | off, "conv output (silu input)", out)+ return np.maximum(out, 0.0)+++def evaluate(g: Graph, tokens: list[int], check: bool = True,+ carry: dict | None = None) -> np.ndarray:+ """Residual stream after every node: (n_tokens, n_dims), exact in float64.++ Text tokens (ids outside the chip vocabulary) can be passed as -1: all chip dims are 0.+ ``carry`` (a dict, empty at the start of a stream) evaluates a long stream chunk by chunk:+ head states and conv windows pass from one call to the next.+ """+ n = len(tokens)+ m = np.zeros((n, len(g.dims)))+ toks = np.asarray(tokens, dtype=np.int64)+ chip = toks >= 0+ m[:, ONE] = chip+ uniq, inv = np.unique(toks, return_inverse=True)+ for d in g.token_fns:+ col = np.array([float(g.feature_value(d, int(u))) if u >= 0 else 0.0 for u in uniq])[inv]+ if check:+ require(is_bf16(col), f"token feature {g.dims[d].name}", col)+ m[:, d] = col+ for i, node in enumerate(g.nodes):+ try:+ if isinstance(node, FFN):+ eval_ffn(node, m, chip, check)+ else:+ eval_head(node, m, chip, check, carry, i)+ except ExactnessError as e:+ raise ExactnessError(f"{node.name}: {e}") from None+ if check:+ # every RMSNorm divides by the ballast alone: other residual values must vanish next to+ # it (x^2 far below m * 2^80)+ big = np.abs(m[:, 1:]).max(0) if m.shape[0] else np.zeros(0)+ if len(big) and big.max() > RESIDUAL_MAX:+ d = int(big.argmax()) + 1+ raise ExactnessError(f"{g.dims[d].name}: residual value {big[d - 1]!r} would move "+ f"the norms (> {RESIDUAL_MAX})")+ return m+++RESIDUAL_MAX = 2.0**28 # x^2 <= 2^56 is lost in fp32 next to the ballast's m * 2^80+++def eval_ffn(node: FFN, m: np.ndarray, chip: np.ndarray, check: bool) -> None:+ ideal = np.zeros(m.shape[0])+ hw = np.zeros(m.shape[0])+ for c, a, b in node.terms:+ bv, av = ev(b, m), ev(a, m)+ ideal += float(c) * av * np.maximum(bv, 0.0)+ if check:+ gate = bf16(K * bv)+ require(silu_ok(gate), "gate outside silu's exact range", gate)+ up = bf16(av / K)+ hw += float(c) * bf16(np.maximum(gate, 0.0) * up)+ if check:+ hw = bf16(hw)+ require(hw == ideal, "bf16 output differs from exact", np.stack([hw, ideal], 1)[:, 0])+ if node.into:+ ideal = m[:, node.out] + ideal+ if check:+ require(is_bf16(ideal), "residual add", ideal)+ m[:, node.out] = ideal+def eval_head(node: Head, m: np.ndarray, chip: np.ndarray, check: bool,+ carry: dict | None = None, ni: int = 0) -> None:+ n = m.shape[0]+ q = (np.stack([conv(c, m, check, carry, (ni, "q", s)) for s, c in node.q.items()], 1)+ if node.q else np.zeros((n, 0)))+ k = (np.stack([conv(c, m, check, carry, (ni, "k", s)) for s, c in node.k.items()], 1)+ if node.k else np.zeros((n, 0)))+ qi, ki = list(node.q), list(node.k)+ if check:+ nz = (k > 0).sum(1)+ require(nz <= 1, "key must be one-hot", nz)+ require(~((nz > 0) & ~chip), "key on a text token", nz)+ for t in np.nonzero((q > 0).sum(1) > 1)[0]:+ vals = q[t][q[t] > 0]+ require(vals == vals[0], "query must be multi-hot", vals)+ has_key = k.any(1) if k.shape[1] else np.zeros(n, dtype=bool)+ key_row = [ki[int(np.argmax(k[t]))] if has_key[t] else None for t in range(n)]+ has_q = q.any(1) if q.shape[1] else np.zeros(n, dtype=bool)+ for x_, vh in enumerate(node.vheads()):+ v = np.zeros((n, node.n_cols))+ for col, c in vh.v.items():+ v[:, col] = conv(c, m, check, carry, (ni, "v", x_, col))+ a = ev(vh.a, m)+ z = {col: ev(e, m) for col, e in vh.z.items()}+ beta = np.ones(n) if vh.beta is None else ev(vh.beta, m)+ writes = has_key & (beta > 0)+ if check:+ require(is_bf16(a) & np.isin(a, list(DECAY)), "decay selector a", a)+ require(np.isin(beta, [0.0, 1.0]), "write gate beta", beta)+ for col, zz in z.items():+ require((is_bf16(zz) & silu_ok(zz)) | dead(zz), f"gate z[{col}]", zz)+ written = v[writes]+ if len(written):+ data = np.delete(written, REF_COLS, 1)+ require(np.isin(data, [0.0, A_BIT]), "state row data column", data)+ ref = written[:, list(REF_COLS)]+ require((ref == 0) | (ref == A_REF), "state row ballast column", ref)+ state: dict[int, np.ndarray] = {} if carry is None else carry.get((ni, "S", x_), {})+ out = np.zeros((n, node.n_cols))+ decay = np.vectorize(DECAY.get)(a)+ # only tokens that decay, write or read change anything+ events = np.nonzero((decay != 1.0) | writes | has_q)[0]+ for t in events:+ if decay[t] == 0.0:+ state.clear()+ elif decay[t] == 0.5:+ state = {s: row * 0.5 for s, row in state.items()}+ if writes[t]:+ state[key_row[t]] = v[t].copy()+ if has_q[t]:+ o = sum((state[s] for s, w in zip(qi, q[t]) if w > 0 and s in state),+ np.zeros(node.n_cols))+ ref = (o[list(REF_COLS)] ** 2).sum()+ if ref == 0:+ if check:+ require(o == 0, "read without ballast", o)+ continue+ if check:+ data = (np.delete(o, REF_COLS) ** 2).sum()+ require(np.array([data <= ref * 2.0**-26]), "data not absorbed by ballast",+ np.array([data / ref]))+ out[t] = o / np.sqrt(ref / node.n_cols)+ if carry is not None:+ carry[ni, "S", x_] = state+ for col, d, w in vh.out:+ zz = z.get(col, np.zeros(n))+ val = out[:, col] * np.maximum(zz, 0.0)+ if check:+ small = np.abs(out[:, col]) < node.tiny+ require(is_bf16(out[:, col]) | small, f"normed column {col}", out[:, col])+ require(is_bf16(val * w) | small, f"head output column {col}", val * w)+ if vh.into:+ m[:, d] += val * w+ if check:+ require(is_bf16(m[:, d]), "head output sum", m[:, d])+ else:+ m[:, d] = val * wdiff --git a/chipformer/q35/lib/__init__.py b/chipformer/q35/lib/__init__.pynew file mode 100644index 0000000..96c2340--- /dev/null+++ b/chipformer/q35/lib/__init__.py@@ -0,0 +1 @@+"""The library the merged model's computer runs: an assembler and BCD arithmetic routines."""diff --git a/chipformer/q35/lib/asm.py b/chipformer/q35/lib/asm.pynew file mode 100644index 0000000..20c7e20--- /dev/null+++ b/chipformer/q35/lib/asm.py@@ -0,0 +1,155 @@+"""A small two-pass CHIP-8 assembler (Cowgod mnemonics, plus PRINT for the bridge's 00FD).++ label: define a label (its address)+ db 1, 2, 0x03 raw bytes+ org 0x600 set the address (gaps are zero-filled)+ ; comment++Instructions: CLS RET PRINT JP CALL SE SNE LD ADD OR AND XOR SUB SHR SUBN SHL RND DRW SKP SKNP.+Operands: V0..VF, I, [I], DT, ST, K, F, B, numbers (decimal / 0x / 0b) and labels.+"""++import re++START = 0x200+++def _num(tok: str, labels: dict[str, int]) -> int:+ tok = tok.strip()+ if tok in labels:+ return labels[tok]+ return int(tok, 0)+++def _reg(tok: str) -> int | None:+ m = re.fullmatch(r"[vV]([0-9a-fA-F])", tok.strip())+ return int(m[1], 16) if m else None+++def assemble(src: str, start: int = START) -> tuple[bytes, dict[str, int]]:+ """Returns (bytes from ``start``, labels)."""+ lines = []+ for raw in src.splitlines():+ line = raw.split(";")[0].strip()+ while line:+ m = re.match(r"([A-Za-z_][\w.]*):\s*(.*)", line)+ if not m:+ break+ lines.append(("label", m[1]))+ line = m[2].strip()+ if line:+ op, _, rest = line.partition(" ")+ args = [a.strip() for a in rest.split(",")] if rest.strip() else []+ lines.append((op.upper(), args))+ labels: dict[str, int] = {}+ for pass_ in range(2):+ pc = start+ out = bytearray()+ for op, args in lines:+ if op == "label":+ labels[args] = pc+ continue+ if op == "ORG":+ target = _num(args[0], labels)+ assert target >= pc, f"org {target:#x} goes backwards from {pc:#x}"+ out += bytes(target - pc)+ pc = target+ continue+ if op == "DB":+ data = [_num(a, labels) & 0xFF for a in args] if pass_ else [0] * len(args)+ out += bytes(data)+ pc += len(data)+ continue+ word = _encode(op, args, labels) if pass_ else 0+ out += bytes([word >> 8, word & 0xFF])+ pc += 2+ return bytes(out), labels+++def _encode(op: str, args: list[str], labels: dict[str, int]) -> int:+ r = [_reg(a) for a in args]+ a = [x.upper() for x in args]++ def n(i, bits=12):+ v = _num(args[i], labels)+ assert 0 <= v < 1 << bits, (op, args, v)+ return v++ if op == "CLS":+ return 0x00E0+ if op == "RET":+ return 0x00EE+ if op == "PRINT":+ return 0x00FD+ if op in ("BADD", "BSUB", "BRSUB"): # bignum macros (macros.py), with a jump target+ t = n(0)+ assert 0x200 <= t < 0x400 and t % 2 == 0, (op, args)+ return 0x5000 | (t - 0x200) // 2 << 4 | {"BADD": 1, "BSUB": 4, "BRSUB": 7}[op]+ if op == "BZERO":+ return 0x5002+ if op == "BMUL":+ return 0x5003 | r[0] << 8+ if op == "BDIV":+ return 0x5005 | r[0] << 8+ if op == "BADB":+ return 0x5006 | r[0] << 8+ if op == "BMUL0":+ return 0x5013 | r[0] << 8+ if op == "BTRIM":+ return 0x5008+ if op == "BLD": # bld a, vX / bld b, vX+ return 0x500B | {"A": 0, "B": 1}[a[0]] << 4 | r[1] << 8+ if op == "BSTA":+ return 0x500C+ if op == "JP":+ if len(args) == 2:+ return 0xB000 | n(1)+ return 0x1000 | n(0)+ if op == "CALL":+ return 0x2000 | n(0)+ if op in ("SE", "SNE"):+ x = r[0]+ if r[1] is not None:+ return (0x5000 if op == "SE" else 0x9000) | x << 8 | r[1] << 4+ return (0x3000 if op == "SE" else 0x4000) | x << 8 | n(1, 8)+ if op == "LD":+ if a[0] == "I":+ return 0xA000 | n(1)+ if a[0] == "[I]":+ return 0xF055 | r[1] << 8+ if a[1] == "[I]":+ return 0xF065 | r[0] << 8+ if a[0] == "DT":+ return 0xF015 | r[1] << 8+ if a[0] == "ST":+ return 0xF018 | r[1] << 8+ if a[0] == "F":+ return 0xF029 | r[1] << 8+ if a[0] == "B":+ return 0xF033 | r[1] << 8+ if a[1] == "DT":+ return 0xF007 | r[0] << 8+ if a[1] == "K":+ return 0xF00A | r[0] << 8+ if r[1] is not None:+ return 0x8000 | r[0] << 8 | r[1] << 4+ return 0x6000 | r[0] << 8 | n(1, 8)+ if op == "ADD":+ if a[0] == "I":+ return 0xF01E | r[1] << 8+ if r[1] is not None:+ return 0x8004 | r[0] << 8 | r[1] << 4+ return 0x7000 | r[0] << 8 | n(1, 8)+ alu = {"OR": 1, "AND": 2, "XOR": 3, "SUB": 5, "SHR": 6, "SUBN": 7, "SHL": 0xE}+ if op in alu:+ y = r[1] if len(r) > 1 and r[1] is not None else 0+ return 0x8000 | r[0] << 8 | y << 4 | alu[op]+ if op == "RND":+ return 0xC000 | r[0] << 8 | n(1, 8)+ if op == "DRW":+ return 0xD000 | r[0] << 8 | r[1] << 4 | n(2, 4)+ if op == "SKP":+ return 0xE09E | r[0] << 8+ if op == "SKNP":+ return 0xE0A1 | r[0] << 8+ raise ValueError(f"unknown instruction {op} {args}")diff --git a/chipformer/q35/lib/bcd.py b/chipformer/q35/lib/bcd.pynew file mode 100644index 0000000..b50ee46--- /dev/null+++ b/chipformer/q35/lib/bcd.py@@ -0,0 +1,601 @@+"""Exact decimal arithmetic for the computer inside the model ("the BIOS").++Calling convention (what the circuit's CALL record writes, see bridge.py):+ A = digits of the first argument, most significant first, at A_ADDR (LA of them at LEN)+ B = digits of the second argument at B_ADDR (LB at LEN + 1)+ PC = the routine's entry+Each routine leaves the result's digits at R_ADDR and ends with PRINT (00FD), which prints V0+digits from I and halts. Leading zeros are stripped. Nothing ticks during a call, so the routines+never use the timers or wait for keys.+"""++from pathlib import Path++from chipformer import spec+from chipformer.q35.lib.asm import assemble++A_ADDR, B_ADDR, LEN, R_ADDR = 0x600, 0x620, 0x640, 0x660+# 0: a tool call (<tool_response>); 1: an inline call (" digits"), | 2 inside "$", | 4 inside+# "$$", | 8 the result is negative (set by sub's BRSUB)+MODE = LEN + 2+DSTOP = LEN + 3 # BDIV / BADB stop at t = LA + DSTOP (div: DEC; mod, gcd: 0)+# div: W = 0, A's digits (the CALL record writes them at W_ADDR + 1), DEC zeros; the quotient goes+# to R_ADDR as P[t] (t <= LA), "." (10) at P[LA + 1], P[t + 1] (t > LA); not ending: frac+W_ADDR = 0x6BF+DEC = 10 # decimals of an inexact quotient+TMP = 0x6E4 # gcd: the remainder while A and B move (after W; 0x700.. holds eval's code)+T10, MUL, DIV, MOD = 0x700, 0x710, 0x780, 0x800+MAX_DIGITS = 24+QUIRKS = ("loadstore",) # FX55/FX65 leave I unchanged+++SRC = f"""+idle: jp idle ; PC rests here between calls (not a PRINT)++; ---------------------------------------------------------------- add: R = A + B+add: badd trim ; macro: R = A + B, v0 = length, i = R, then trim+ db 0, 0 ; (entries are compiled into the weights)++; ---------------------------------------------------------------- sub: R = A - B+sub: bsub trim ; macro: R = A - B, v0 = length, i = R; trim unless A < B+ jp sub_neg++; ---------------------------------------------------------------- mul: R = A * B+mul: ld v1, 0 ; rows done+ bmul0 v1 ; macro: the first schoolbook row (no BZERO)+ bmul v1 ; macro: the next row; repeats until the last row,+ ; which sets v0 = LA + LB, i = R++; ---------------------------------------------------------------- print mem[i..i+v0)+trim: btrim ; macro: drop a leading 0 (keeping one digit); repeats++; ---------------------------------------------------------------- A - B < 0: B - A+; MODE |= 8: the print starts with " -" (a tool call: "<tool_response> -")+sub_neg: brsub trim++; ---------------------------------------------------------------- div: A / B+; the quotient if it ends within DEC decimals, else the reduced fraction p/q (frac, in low+; memory); "..." if B starts with 0+div: bzero ; P[0 .. LA+LB) = 0+ ld i, {LEN}+ ld v1, [i] ; v0 = LA, v1 = LB+ ld va, v0+ ld vb, v1+ ld i, {B_ADDR}+ ld v0, [i]+ sne v0, 0+ jp div_z+ ld i, {W_ADDR + 1}+ bld a, va ; A = A's digits (div works on W; frac needs them)+ ld i, {R_ADDR + 1}+ add i, va+ ld v0, 10+ ld [i], v0 ; P[LA + 1] = "."+ ld i, {DSTOP}+ ld [i], v0 ; DS = DEC (= 10, as the ".")+ ld i, zeros+ ld v9, [i] ; v0..v9 = 0+ ld i, {W_ADDR}+ ld [i], v0 ; W[0] = 0+ ld i, {W_ADDR + 1}+ add i, va+ ld [i], v9 ; W[LA+1 .. LA+DEC] = 0+ ld vc, vb ; t = LB+ ld v0, va+ add v0, {DEC}+ sub v0, vc ; VF = [LA + DEC >= LB]+ se vf, 0+ jp div_lp+ ld vc, va+ add vc, {DEC} ; B is too long: the quotient is 0.000..., t = LA + DEC+ jp div_rem+div_lp: bdiv vc ; repeats itself, one quotient digit per record+ se vf, 1+ jp div_end ; VF = 2 (exact) or 3 (DEC decimals)+ badb vc ; repeats itself while the window is negative+ se vf, 0+ jp div_end+ jp div_lp+div_end: se vf, 3+ jp div_ex+div_rem: jp frac ; no end within DEC decimals: the reduced fraction p/q+div_ex: ld ve, vc+ se vc, va+ add ve, 1 ; t > LA: P[t + 1]; t == LA: no "."+div_pr: ld v6, vb ; first digit: P[LB] (P[LA] if LB > LA), then skip zeros+ ld v0, va+ sub v0, v6+ sne vf, 0+ ld v6, va+dp_lp: se v6, va+ jp dp_chk+ jp dp_go+dp_chk: ld i, {R_ADDR}+ add i, v6+ ld v0, [i]+ se v0, 0+ jp dp_go+ add v6, 1+ jp dp_lp+dp_go: ld i, {R_ADDR}+ add i, v6+ ld v0, ve+ sub v0, v6+ add v0, 1+div_z: ld i, {R_ADDR} ; B starts with 0: "..." (every result prints something)+ ld v0, 11+ ld [i], v0+ ld v0, 1+zeros: db 0, 0, 0, 0, 0, 0, 0, 0, 0, 0++"""++# mod and gcd (v16) next to v18's evaluator: the 4 KB are full, so their code fills four gaps+# (eval.asm's {MODGCD1..4}): the end of eval's first part (below A_ADDR), two holes in the data+# page (after LEN's cells, after R's 48 digits; nothing writes them), the end of eval's second+# part (below TAPE) and the end of chain mode's code (below H). Asserts check each one.+GAP_LEN, GAP_R = LEN + 4, R_ADDR + 48 # data-page holes: up to R_ADDR, up to W_ADDR+MODGCD1 = f"""+; ---------------------------------------------------------------- mod: A mod B+; (A at W + 1, as for div); "..." if B starts with 0+mod: call args+ sne v0, 0+ jp div_z+ call rem+rem_pr: ld i, {W_ADDR} ; print W[vd .. ve]+ add i, vd+ ld v0, ve+ sub v0, vd+ add v0, 1+mg1_end:+"""+MODGCD2 = f"""+ org {GAP_LEN}+; ---------------------------------------------------------------- the shared parts+args: ld v0, 0 ; DS = 0 (the integer part only), W[0] = 0,+ ld i, {DSTOP} ; va = LA, vb = LB, v0 = B[0]+ ld [i], v0+ ld i, {W_ADDR}+ ld [i], v0+ ld i, {LEN}+ ld v1, [i]+ ld va, v0+ ld vb, v1+ ld i, {B_ADDR}+ ld v0, [i]+ ret+args_end:+ org {GAP_R}+; ---------------------------------------------------------------- gcd: gcd(A, B), by Euclid+gcd: call args+ se v0, 0+ jp gcd_lp+ ld vd, 1 ; B = 0: the answer is A+ ld ve, va+ call strip+ jp rem_pr+gcd_lp: call rem ; W[vd .. ve] = A mod B+ ld i, {W_ADDR}+ add i, vd+ ld v0, [i]+ sne v0, 0+ jp gcd_b ; 0: the answer is B+ ld vc, ve ; vc = the remainder's length (the next LB)+ sub vc, vd+ add vc, 1+ ld i, {W_ADDR} ; the moves go through A with v18's BLD / BSTA macros+ add i, vd+ bld a, vc ; A = the remainder+ ld i, {TMP}+ bsta ; TMP = the remainder+ ld i, {B_ADDR}+ jp gcd_c+gcd1_end:+"""+MODGCD3 = f"""+gcd_c: bld a, vb ; A = B, LA = LB (the next A)+ ld i, {W_ADDR + 1}+ bsta ; W + 1 = B+ ld i, {TMP}+ bld b, vc ; B = the remainder, LB = its length+ ld va, vb+ ld vb, vc+ jp gcd_lp+gcd_b: jp gcd_e ; (low memory: frac's gcd returns instead)+"""+MODGCD4 = f"""+; rem: A mod B for A at W[1 .. va] (W[0] = 0), B at B[0 .. vb) (B[0] != 0), LEN = LA, LB and+; DS = 0: the remainder, without leading zeros (one digit if it is 0), is W[vd .. ve]. BDIV and+; BADB run the integer part of A / B and leave the remainder in the last window.+rem: ld v0, va+ sub v0, vb ; VF = [LA >= LB]+ se vf, 0+ jp rem_go+ ld vd, 1 ; B is longer: the remainder is A+ ld ve, va+ jp strip+rem_go: ld vc, vb ; t = LB+rem_lp: bdiv vc ; repeats itself, one quotient digit per record+ se vf, 1+ jp rem_end ; VF = 2 (a 0 remainder) or 3, at t = LA+ badb vc ; repeats itself while the window is negative+ se vf, 0+ jp rem_end+ jp rem_lp+rem_end: ld ve, va+ ld vd, ve+ sub vd, vb+ add vd, 1 ; the window's top digit is 0 (the remainder is below B)+strip: se vd, ve ; skip zeros in W[vd .. ve), keeping the last digit+ jp st_chk+ ret+st_chk: ld i, {W_ADDR}+ add i, vd+ ld v0, [i]+ se v0, 0+ ret+ add vd, 1+ jp strip+"""++# ---------------------------------------------------------------- eval: a whole expression (v18)+# The CALL record writes the tape (one code per token of the expression: bridge.TAPE_CODE, digits+# 0..9, the symbols . + - * / ( ) ^ as 10..17, fused tokens such as ")/(" as 18.., which the+# dispatch table expands), its length and MODE. eval (eval.asm) parses it by shunting-yard into+# exact fractions n / d and prints the value: the integer, or n / d as div prints it, or "..."+# when it can't (more than MAX_DIGITS digits anywhere, a division by 0, a malformed expression,+# more than NVAL pending values). A value lives in a slot: H + 32 s = [sign, n length, n's place+# (0: its digits follow, else a number on the tape), d length (0: d = 1), n's digits], D + 32 s+# = d's digits; BLD / BSTA (macros.py) move digits around.+# Memory: eval's code from 0x2AE to A_ADDR and from CODE2 to TAPE, then its data, then DBUF (10^k).+CODE2 = 0x700+TAPE, TLEN, MSAVE, CUR, DXY, SGN, EXP = 0xC00, 0xC30, 0xC31, 0xC32, 0xC34, 0xC36, 0xC38+OPST, XTAB, H, D = 0xC50, 0xC80, 0xE00, 0xF00 # operator stack, ROM tables, value slots+NVAL = 6 # value slots (offsets 0 .. 160); T6 and T7 are temporaries+T6, T7 = 192, 224+END = 63 # after the tape+H1, H3, H4, SGN1, EXP1, LEN1, MAXD1 = H + 1, H + 3, H + 4, SGN + 1, EXP + 1, LEN + 1, MAX_DIGITS + 1+TAPE_M1 = TAPE - 1 # a number's place p: its digits from TAPE + p - 1+# chain mode (chain.asm): its sign parity, accumulator lengths, the first number, accumulators+GP, L0, L1, PEND, FOP, ACC0, ACC1, FST, CODE3 = (0xC3A, 0xC3B, 0xC3C, 0xC3D, 0xC40, 0x0A0, 0x0C0,+ OPST, 0xCA0)+VMAX, W1 = 32 * NVAL, W_ADDR + 1+T6_DL, T6_D, T7_NL, T7_N = H + T6 + 3, D + T6, H + T7 + 1, H + T7 + 4+SYM_SUB = dict(zip(".+-*/()^", ("s_pt", "s_add", "s_sub", "s_mul", "s_div", "s_lp", "s_rp", "s_pow")))+++def _dispatch() -> tuple[str, str, str]:+ """The tape codes' handlers (each symbol of a fused token in turn), the jump table on codes+ and the one on operators (reduce)."""+ from chipformer.q35 import bridge++ handlers, jtab = [], []+ for code in range(64):+ if code < 10:+ jtab.append(" jp dig")+ elif code in bridge.EXPANSION:+ calls = [f" call {SYM_SUB[ch]}" for ch in bridge.EXPANSION[code]]+ handlers.append(f"h{code}:" + "\n".join(calls)[3:] + "\n jp ev_lp")+ jtab.append(f" jp h{code}")+ else:+ jtab.append(" jp h_end" if code == END else " jp err")+ ops = {4: "r_add", 5: "r_sub", 8: "r_mul", 9: "r_div", 16: "r_pow"}+ rtab = [f" jp {ops.get(code, 'err')}" for code in range(17)]+ return "\n".join(handlers), "\n".join(jtab), "\n".join(rtab)+++HANDLERS, JTAB, RTAB = _dispatch()+_NAMES = {k: v for k, v in globals().items() if k.isupper()}+EVAL_SRC = (Path(__file__).with_name("eval.asm").read_text(encoding="utf-8")+ .format(**_NAMES, DBUF_DB=", ".join(["1"] + ["0"] * MAX_DIGITS), **dict(zip(("CHAIN1", "CHAIN2"), Path(__file__).with_name("chain.asm")+ .read_text(encoding="utf-8").format(**_NAMES)+ .split("; ---- part 2")))))++# ---------------------------------------------------------------- frac: a reduced fraction+# Below PROGRAM_START (0x0E0 .. 0x1FF: after the font and chain mode's accumulators) the 4 KB+# still had room: the boot record writes this segment too (ROM.segments, ref/emu.Chip8).+# div jumps here when A / B does not end within DEC decimals: p = A / g, q = B / g for+# g = gcd(A, B) (v16's gcd, which returns to frac instead of printing), printed as "p/q" (12 is+# "/"), MODE's sign as for the quotient. A's and B's digits wait in the tape's cells (eval is done+# with them when it prints through div).+LOW_ORG = 0x0E0+SAVE_A, SAVE_B = TAPE, TAPE + MAX_DIGITS+FRAC_F, LA0, LB0, LP = LEN + 28, LEN + 29, LEN + 30, LEN + 31 # (after args' code in the hole)+OUT = W_ADDR # the print: p, "/", q (W and TMP are free by then)+SLASH = 12 # the print's code for "/"+LOW_SRC = f"""+frac: ld i, {SAVE_A}+ bsta ; SAVE_A = A (div saved it in A at its start)+ ld i, {B_ADDR}+ bld a, vb+ ld i, {SAVE_B}+ bsta ; SAVE_B = B+ ld v0, va+ ld v1, vb+ ld i, {LA0}+ ld [i], v1 ; LA0, LB0+ ld i, {SAVE_A}+ bld a, va+ ld i, {W_ADDR + 1}+ bsta ; W + 1 = A, as gcd takes it+ ld v0, 1+ ld i, {FRAC_F}+ ld [i], v0+ call gcd ; B = g, LB its length+ ld v0, 0+ ld i, {FRAC_F}+ ld [i], v0+ ld i, {LA0}+ ld v0, [i]+ ld i, {SAVE_A}+ bld a, v0+ ld i, {W_ADDR + 1}+ bsta ; W + 1 = A+ call quo ; I = p's first digit, v0 = its digits+ bld a, v0+ ld i, {LP}+ ld [i], v0 ; LP = lp+ ld i, {SAVE_A}+ bsta ; SAVE_A = p+ ld i, {LB0}+ ld v0, [i]+ ld i, {SAVE_B}+ bld a, v0+ ld i, {W_ADDR + 1}+ bsta ; W + 1 = B+ call quo ; I = q's first digit, v0 = its digits+ ld v3, v0+ bld a, v3 ; A = q+ ld i, {LP}+ ld v0, [i]+ ld v2, v0 ; v2 = lp, v3 = lq+ ld i, {OUT + 1}+ add i, v2+ bsta ; OUT + lp + 1 = q+ ld i, {OUT}+ add i, v2+ ld v0, {SLASH}+ ld [i], v0 ; "/"+ ld i, {SAVE_A}+ bld a, v2+ ld i, {OUT}+ bsta ; OUT = p+ ld v0, v2+ add v0, v3+ add v0, 1+ ld i, {OUT}++; quo: A / B for A at W + 1 and B dividing it: I = the quotient's first digit (P[LB .. LA]+; without leading zeros), v0 = its digits+quo: call args+ call rem+ ld v6, vb+q_lp: se v6, va+ jp q_chk+ jp q_go+q_chk: ld i, {R_ADDR}+ add i, v6+ ld v0, [i]+ se v0, 0+ jp q_go+ add v6, 1+ jp q_lp+q_go: ld v0, va+ sub v0, v6+ add v0, 1+ ld i, {R_ADDR}+ add i, v6+ ret++; gcd's end: the gcd op prints B; frac's call returns with it+gcd_e: ld i, {FRAC_F}+ ld v0, [i]+ se v0, 0+ ret+ ld i, {B_ADDR}+ ld v0, vb+low_end:+ org {spec.PROGRAM_START}+"""+++class Image(bytes):+ """A ROM (loaded at PROGRAM_START) and its segments below it: [(address, bytes)]."""++ segments: tuple = ()+++_out, LABELS = assemble(LOW_SRC + SRC + EVAL_SRC, start=LOW_ORG)+ROM = Image(_out[spec.PROGRAM_START - LOW_ORG:])+ROM.segments = ((LOW_ORG, bytes(_out[:LABELS["low_end"] - LOW_ORG])),)+assert LABELS["idle"] == spec.PROGRAM_START and LABELS["low_end"] <= spec.PROGRAM_START+OPS = {op: LABELS[op] for op in ("add", "mul", "sub", "div", "mod", "gcd", "eval")}+# The entries are compiled into the circuit's CALL record (chip8_q35.py), so they are part of the+# weights: moving a routine means rebuilding every checkpoint. Add new code after the entries.+assert {k: v for k, v in OPS.items() if k in ("add", "sub", "mul")} == {+ "add": 0x202, "sub": 0x206, "mul": 0x20A}, OPS+W_OPS = ("div", "mod", "gcd") # A goes to W + 1: they work on it in place+assert LABELS["sm_t"] < 0x400 and LABELS["am_t"] < 0x400, "macro targets must sit below 0x400"+# compiled into the weights (the expression CALL record): eval's entry, the tape and its length+# (q35-int: div, mod and gcd come first, so eval sits later than v18's 0x2CA)+EVAL_ENTRY = 0x2CC+assert (OPS["eval"], TAPE, TLEN) == (EVAL_ENTRY, 0xC00, 0xC30), (hex(OPS["eval"]), TAPE, TLEN)+assert W_ADDR + 1 + MAX_DIGITS + DEC <= TMP and TMP + MAX_DIGITS <= CODE2, "gcd's TMP"+assert LABELS["code1_end"] <= A_ADDR and LABELS["eval_end"] <= TAPE, "eval's code overflows"+assert LABELS["dbuf"] + MAX_DIGITS < CODE3 and spec.FONT_BASE + 80 <= ACC0 < ACC1 + 32 <= 0x200+assert LABELS["chain_end"] <= H, "chain mode's code overflows"+# mod / gcd's pieces in the data page: clear of LEN's cells, R's 48 digits (mul) and W+assert DSTOP < GAP_LEN and LABELS["args_end"] <= FRAC_F, "args overflows its hole"+assert LABELS["gcd1_end"] <= W_ADDR, "gcd overflows its hole"+++class EvalError(Exception):+ pass+++def eval_value(tape: list[int]):+ """What eval computes, as (sign, n, d) with n, d strings (d None for 1), or EvalError: the+ reference for eval.asm, including its limits."""+ from chipformer.q35 import bridge++ syms = []+ for code in tape:+ syms += [(None, code)] if code < 10 else [(s, None) for s in bridge.EXPANSION[code]]+ vals, ops = [], []+ num, dec, operand = None, None, False # digits, decimals (None: no ".")++ def fits(s):+ s = s.lstrip("0") or "0"+ if len(s) > MAX_DIGITS:+ raise EvalError("too long")+ return s++ def commit():+ nonlocal num, dec, operand+ if len(vals) >= NVAL:+ raise EvalError("too many values")+ k = dec or 0+ vals.append([0, fits(num), "1" + "0" * k if k else None])+ num, dec, operand = None, None, True++ def mul(a, b):+ return fits(str(int(a) * int(b)))++ def reduce():+ op = ops.pop()+ if op == "neg":+ if not vals:+ raise EvalError("no operand")+ vals[-1][0] ^= 1+ return+ if len(vals) < 2:+ raise EvalError("no operand")+ y = vals.pop()+ x = vals[-1]+ sx, nx, dx = x+ sy, ny, dy = y+ if op in "+-":+ if op == "-":+ sy ^= 1+ if dx is None and dy is None:+ t6, t7, d = nx, ny, None+ else:+ t6 = nx if dy is None else mul(nx, dy)+ t7 = ny if dx is None else mul(ny, dx)+ d = dx if dy is None else dy if dx is None else mul(dx, dy)+ if sx == sy:+ n, s = fits(str(int(t6) + int(t7))), sx+ else:+ a, b = int(t6), int(t7)+ n, s = fits(str(abs(a - b))), sx if a >= b else sy+ x[:] = [s, n, d]+ elif op == "*":+ x[:] = [sx ^ sy, mul(nx, ny), dx if dy is None else dy if dx is None else mul(dx, dy)]+ elif op == "/":+ if int(ny) == 0:+ raise EvalError("division by 0")+ x[:] = [sx ^ sy, nx if dy is None else mul(nx, dy), ny if dx is None else mul(dx, ny)]+ else: # ^+ if dy is not None or len(ny) > 2:+ raise EvalError("exponent")+ e = int(ny)+ if e == 0:+ x[:] = [0, "1", None]+ return+ n, d = nx, dx+ for _ in range(e - 1):+ n = mul(n, nx)+ d = None if dx is None else mul(d, dx)+ s = sx if e % 2 else 0+ if sy:+ if int(n) == 0:+ raise EvalError("division by 0")+ n, d = d or "1", n+ x[:] = [s, n, d]++ PREC = {"+": 1, "-": 1, "*": 2, "/": 2, "neg": 3, "^": 4}+ for s, d in syms:+ if s is None:+ num = (num or "") + str(d)+ if dec is not None:+ dec += 1+ continue+ if s == ".":+ if num is None or dec is not None:+ raise EvalError("bad .")+ dec = 0+ continue+ if num is not None:+ commit()+ if s == "(":+ if operand:+ push_binary("*", ops, PREC, reduce)+ ops.append("(")+ operand = False+ elif s == ")":+ while ops and ops[-1] != "(":+ reduce()+ if ops:+ ops.pop()+ operand = True+ elif not operand:+ if s == "-":+ ops.append("neg")+ elif s != "+":+ raise EvalError("operator without an operand")+ else:+ push_binary(s, ops, PREC, reduce)+ operand = False+ if num is not None:+ commit()+ while ops:+ if ops[-1] == "(":+ ops.pop()+ else:+ reduce()+ if len(vals) != 1:+ raise EvalError("not one value")+ return tuple(vals[0])+++def push_binary(s, ops, prec, reduce):+ while ops and ops[-1] != "(" and (prec[ops[-1]] > prec[s]+ or (prec[ops[-1]] == prec[s] and s != "^")):+ reduce()+ ops.append(s)+++def eval_text(tape: list[int]) -> tuple[bool, str]:+ """(negative, digits) as eval prints them: the integer, the quotient as div prints it, or+ "..." when eval gives up."""+ from chipformer.q35 import evals++ try:+ s, n, d = eval_value(tape)+ except EvalError:+ return False, "..."+ neg = bool(s) and int(n) != 0+ if d is None:+ return neg, n+ return neg, evals.div_text(int(n), int(d))+++def eval_writes(tape: list[int], mode: int) -> list[tuple[int, int]]:+ """The CALL record of an expression: the tape, its length, the mode, then PC = eval."""+ return [(TAPE + i, c) for i, c in enumerate(tape)] + [(TLEN, len(tape)), (MODE, mode),+ (spec.PC, OPS["eval"])]+++def call_writes(op: str, a: str, b: str, mode: int = 0) -> list[tuple[int, int]]:+ """The writes of a CALL record: the arguments' digits, their lengths, the mode, then PC."""+ da, db = [int(ch) for ch in a], [int(ch) for ch in b]+ assert 1 <= len(da) <= MAX_DIGITS and 1 <= len(db) <= MAX_DIGITS, (a, b)+ a_at = W_ADDR + 1 if op in W_OPS else A_ADDR+ w = [(a_at + i, d) for i, d in enumerate(da)] + [(B_ADDR + i, d) for i, d in enumerate(db)]+ return w + [(LEN, len(da)), (LEN + 1, len(db)), (MODE, mode), (spec.PC, OPS[op])]diff --git a/chipformer/q35/lib/chain.asm b/chipformer/q35/lib/chain.asmnew file mode 100644index 0000000..dab227d--- /dev/null+++ b/chipformer/q35/lib/chain.asm@@ -0,0 +1,283 @@++; ================================================================ eval's chain mode (v18)+; Most expressions are chains of one precedence: only * and / or only + and -, with brackets+; and signs. Their value is (-1)^GP ACC0 / ACC1 (products: every number multiplies the+; numerator ACC0 or the denominator ACC1) or ACC0 - ACC1 (sums: every number adds to the+; positive or the negative terms), so each number costs one macro and no operator stack.+; Anything else (a ".", "^", the other kind of operator, a 0 in a product, an implicit product,+; more than MAX_DIGITS digits) restarts the general evaluator (g_init), which is the reference.+;+; registers: vd = 1 (chain mode), v7 = mode (0 undecided, 1 products, 2 sums), vc = the current+; bracket's bits and v8 = the next operand's (bit 0: negated (sums), bit 1: a divisor+; (products)), va = open brackets (their vc at FST), v6 = the accumulator A holds (its copy in+; memory is stale; 255: none), vb, ve, v9 as in the general evaluator. The first number waits+; in PEND until the first operator decides the mode.++f_init: ld v0, 0+ ld v1, 0+ ld v2, 0+ ld v3, 0+ ld v4, 0+ ld i, {GP}+ ld [i], v4 ; GP, L0, L1 = 0, PEND = none+ ld vd, 1+ ld v6, 255+ ld v7, 0+ ld vc, 0+ ld v8, 0+ ld v9, 0+ ld va, 0+ ld vb, 255+ ld ve, 0+ jp ev_lp++fcons: sne vb, 255 ; the number that just ended, if any+ ret+ se v9, 0+ jp g_init ; right after an operand+ ld v5, ve+ add v5, 255+ sub v5, vb ; v5 = its digits (vb: the first)+ ld i, {TAPE}+ add i, vb+ ld v0, [i]+ sne v0, 0+ call fc_z ; leading zeros+ ld v0, v5+ ld v1, {MAXD1}+ sub v0, v1+ se vf, 0+ jp g_init+ ld v2, vc ; v2 = its bits+ xor v2, v8+ sne v7, 0+ jp fc_p+ sne v7, 1+ shr v2, v2 ; products: bit 1 picks the accumulator+ ld v0, 1+ and v2, v0+ call facc+fc_e: ld vb, 255+ ld v8, 0+ ld v9, 1+ ret+fc_p: ld v0, 1 ; undecided: PEND = (first digit, digits, negated)+ and v2, v0+ ld v0, vb+ ld v1, v5+ ld i, {PEND}+ ld [i], v2+ jp fc_e+fc_z: sne v5, 1+ ret+ ld i, {TAPE}+ add i, vb+ ld v0, [i]+ se v0, 0+ ret+ add vb, 1+ add v5, 255+ jp fc_z++facc: ld i, {TAPE} ; ACC[v2] op= the number (v5 digits from tape index vb)+ add i, vb+ bld b, v5+ sne v7, 1+ call fa_0+ ld i, {L0}+ add i, v2+ ld v0, [i]+ sne v0, 0+ jp fa_new+ se v6, v2+ call fa_ld+ sne v7, 1+ call mulm+ sne v7, 2+ call addm+ ld v1, v0+ ld v3, {MAXD1}+ sub v1, v3+ se vf, 0+ jp g_init+ bld a, v0+ jp fa_len+fa_new: se v6, 255 ; empty: ACC = the number+ call fa_sp+ ld i, {B_ADDR}+ bld a, v5+ ld v0, v5+fa_len: ld v6, v2 ; A holds ACC[v2]+ ld i, {L0}+ add i, v2+ ld [i], v0+ ret+fa_ld: se v6, 255 ; A = ACC[v2] (v0 digits), the other one back to memory+ call fa_sp+ ld i, {ACC0}+ se v2, 0+ ld i, {ACC1}+ bld a, v0+ ret+fa_sp: ld i, {ACC0} ; ACC[v6] = A+ se v6, 0+ ld i, {ACC1}+ bsta+ ret+fa_0: se v5, 1 ; products: a 0 goes to the general evaluator+ ret+ ld i, {B_ADDR}+ ld v0, [i]+ sne v0, 0+ jp g_init+ ret++; ---- part 2+f_lp: call fcons ; "("+ se v9, 0+ jp g_init ; a product without an operator+ ld v0, vc+ ld i, {FST}+ add i, va+ ld [i], v0+ add va, 1+ xor vc, v8+ ld v8, 0+ ret+f_rp: call fcons ; ")"+ sne va, 0+ jp fr_d+ add va, 255+ ld i, {FST}+ add i, va+ ld v0, [i]+ ld vc, v0+fr_d: ld v8, 0+ ld v9, 1+ ret+f_op: ld i, {FOP} ; an operator (v0 = 4 + 5 - 8 * 9 /)+ ld [i], v0+ call fcons+ ld i, {FOP}+ ld v0, [i]+ ld v3, v0+ se v9, 0+ jp fo_bin+ sne v3, 5 ; not after an operand: "-" negates, "+" is nothing+ jp fo_neg+ sne v3, 4+ ret+ jp g_init+fo_neg: ld v0, 1+ xor v8, v0+ ld i, {GP}+ ld v0, [i]+ ld v1, 1+ xor v0, v1+ ld [i], v0+ ret+fo_bin: ld v1, v3 ; its mode: 1 (* /) or 2 (+ -)+ ld v2, 8+ sub v1, v2+ ld v2, 2+ se vf, 0+ ld v2, 1+ se v7, 0+ jp fo_m+ ld v7, v2 ; the first operator decides+ ld i, {PEND}+ ld v2, [i] ; v0 = its first digit, v1 = digits, v2 = negated+ sne v1, 0+ jp g_init+ ld vb, v0+ ld v5, v1+ se v7, 2+ ld v2, 0 ; products: the numerator+ call facc+ ld vb, 255+ ld i, {FOP}+ ld v0, [i]+ ld v3, v0+ jp fo_p+fo_m: se v7, v2+ jp g_init+fo_p: ld v8, 0 ; the next operand: "-" negates it, "/" divides by it+ sne v3, 5+ ld v8, 1+ sne v3, 9+ ld v8, 2+ ld v9, 0+ ret++f_end: call fcons ; the end+ se v6, 255+ call fa_sp+ se v7, 0+ jp fe_m+ ld i, {PEND} ; one number: its sign is GP+ ld v1, [i]+ sne v1, 0+ jp g_init+ ld i, {TAPE}+ add i, v0+ bld a, v1+ ld v5, v1+ ld v6, 0+ ld i, {GP}+ ld v0, [i]+ ld v7, v0+ jp fe_pr+fe_m: ld i, {GP}+ ld v2, [i] ; v0 = GP, v1 = L0, v2 = L1+ ld v5, v1+ ld v6, v2+ se v7, 1+ jp fe_sum+ ld v7, v0 ; products: (-1)^GP ACC0 / ACC1+ ld i, {ACC0}+ bld a, v5+ jp fe_pr+fe_sum: ld v7, 0 ; sums: ACC0 - ACC1+ se v6, 0+ jp fe_s2+ ld i, {ACC0}+ bld a, v5+ jp fe_pr+fe_s2: ld i, {ACC1}+ bld b, v6+ sne v5, 0+ jp fe_neg+ ld i, {ACC0}+ bld a, v5+ call subm+ ld v7, 1+ se v3, 0+ ld v7, 0+ bld a, v0+ ld v5, v0+ ld v6, 0+ jp fe_pr+fe_neg: ld i, {ACC1} ; no positive terms: -ACC1+ bld a, v6+ ld v5, v6+ ld v6, 0+ ld v7, 1+fe_pr: ld i, {A_ADDR} ; A = the value's n (v5 digits), v6 = d's (0: none), v7 = sign+ ld v0, [i]+ se v5, 1+ jp fe_s+ se v0, 0+ jp fe_s+ ld v7, 0 ; 0: no sign, no d+ ld v6, 0+fe_s: call setmode+ se v6, 0+ jp fe_div+ ld i, {A_ADDR}+ ld v0, v5+fe_div: ld i, {W1} ; n / d: div prints it+ bsta+ ld i, {ACC1}+ bld b, v6+ jp divdiff --git a/chipformer/q35/lib/eval.asm b/chipformer/q35/lib/eval.asmnew file mode 100644index 0000000..d42959e--- /dev/null+++ b/chipformer/q35/lib/eval.asm@@ -0,0 +1,754 @@++; ================================================================ eval: a whole expression (v18)+; (bcd.py fills in the names in braces and generates the dispatch tables; see its notes)+;+; The tape: one code per token (digits 0..9, symbols 10..17, fused tokens 18..), END after it.+; A value lives in slot s (offset 32 s): H + 32 s = [sign, n length, n's place, d length (0: d = 1),+; n's digits]; n's place 0: its digits follow (H4 + 32 s), else they are a number on the tape+; (TAPE + place - 1); D + 32 s = d's digits. Slots 0 .. NVAL-1 are the value stack, T6 and T7+; temporaries.+;+; registers while parsing: ve = tape index, vd = 32 * values, va = operators (vc = the top),+; vb = where the number being read starts (255: none; v8 = its "." + 1, or 0), v9 = 1 after an+; operand; in reduce v6 / v7 = the offsets of X / Y (X op Y -> X); v0..v5 are scratch.+; operators: "(" 0, + 4, - 5, * 8, / 9, negation 12, ^ 16. An operator reduces the ones on top+; of the stack while they are >= its threshold (+ - 4, * / 8, ^ 17: right-associative).++; macros with jump targets first (they must sit below 0x400): R = A op B with the leading zeros+; skipped, so v0 = the length and I = its first digit+mulm: ld v4, 0+ bmul0 v4+ bmul v4+ btrim+ ret+addm: badd am_t+am_t: btrim+ ret+subm: ld v3, 1 ; |A - B|, v3 = [A >= B]+ bsub sm_t+ ld v3, 0+ brsub sm_t+sm_t: btrim+ ret++; ---------------------------------------------------------------- the parser+eval: ld i, {MODE}+ ld v0, [i]+ ld i, {MSAVE}+ ld [i], v0+ ld v1, 1+ or v0, v1 ; MODE bit 0 while it runs (so BRSUB sets v0)+ ld i, {MODE}+ ld [i], v0+ ld i, {TLEN}+ ld v0, [i]+ ld i, {TAPE}+ add i, v0+ ld v0, {END}+ ld [i], v0+ jp f_init ; chain mode first (it falls back to g_init)+g_init: ld ve, 0 ; the general evaluator+ ld vd, 0+ ld va, 0+ ld vb, 255+ ld v8, 0+ ld v9, 0+ev_lp: ld i, {TAPE}+ add i, ve+ ld v0, [i]+ add ve, 1+ add v0, v0+ jp v0, jtab+dig: se vb, 255+ jp ev_lp+ ld vb, ve ; a number starts+ add vb, 255+ jp ev_lp++s_pt: sne vd, 1 ; "." (chain mode: the general evaluator)+ jp g_init+ sne vb, 255+ jp err+ se v8, 0+ jp err+ ld v8, ve+ ret+s_lp: sne vd, 1 ; "(" (right after an operand: a product)+ jp f_lp+ ld v7, 0+ se vb, 255+ call commit+ se v9, 0+ call imul+ ld v0, 0+ jp push+imul: ld v0, 8+ ld v1, 8+ ld i, {CUR}+ ld [i], v1+ jp bo_lp+s_rp: sne vd, 1 ; ")" (none open: ignored)+ jp f_rp+ ld v7, 0+ se vb, 255+ call commit+rp_lp: sne va, 0+ jp rp_d+ sne vc, 0+ jp rp_p+ call reduce+ jp rp_lp+rp_p: call pop+rp_d: ld v9, 1+ ret+s_add: ld v0, 4+ ld v1, 4+ jp s_op+s_sub: ld v0, 5+ ld v1, 4+ jp s_op+s_mul: ld v0, 8+ ld v1, 8+ jp s_op+s_div: ld v0, 9+ ld v1, 8+ jp s_op+s_pow: sne vd, 1+ jp g_init+ ld v0, 16+ ld v1, 17+s_op: sne vd, 1+ jp f_op+ ld i, {CUR} ; CUR = the operator, its threshold+ ld [i], v1+ ld v7, v0+ se vb, 255+ call commit+ se v9, 0+ jp bo_lp+ sne v7, 5 ; not after an operand: "-" negates, "+" is nothing+ jp s_neg+ sne v7, 4+ ret+ jp err+s_neg: ld v0, 12+ jp push+bo_lp: ld i, {CUR} ; reduce the operators on top >= the threshold, then push+ ld v1, [i]+ sne va, 0+ jp push+ ld v2, vc+ sub v2, v1+ sne vf, 0+ jp push+ call reduce+ jp bo_lp+push: ld v1, v0 ; push v0: memory keeps the one below, vc the top+ ld v0, vc+ ld i, {OPST}+ add i, va+ ld [i], v0+ add va, 1+ ld vc, v1+ ld v9, 0+ ret+pop: add va, 255 ; vc = the one below+ ld i, {OPST}+ add i, va+ ld v0, [i]+ ld vc, v0+ ret++commit: sne vd, {VMAX} ; push the number on the tape from vb to ve - 2+ jp err+ ld v5, ve+ add v5, 255+ sub v5, vb ; v5 = its tokens+ ld v4, 0 ; v4 = its decimals+ sne v8, 0+ jp cm_z+ ld v4, ve ; move the decimals left over the "."+ add v4, 255+ sub v4, v8+ ld i, {TAPE}+ add i, v8+ bld a, v4+ ld v0, v8+ add v0, 255+ ld i, {TAPE}+ add i, v0+ bsta+ add v5, 255+cm_z: sne v5, 1 ; skip leading zeros+ jp cm_c+ ld i, {TAPE}+ add i, vb+ ld v0, [i]+ se v0, 0+ jp cm_c+ add vb, 1+ add v5, 255+ jp cm_z+cm_c: ld v0, v5+ ld v1, {MAXD1}+ sub v0, v1+ se vf, 0+ jp err+ ld v2, v4+ sne v2, 0+ jp cm_n+ add v2, 1+ ld i, dbuf ; d = 10^decimals+ bld a, v2+ ld i, {D}+ add i, vd+ bsta+cm_n: ld v3, 0 ; a negation right before it (not before "^") folds into it+ sne va, 0+ jp cm_h+ se vc, 12+ jp cm_h+ sne v7, 16+ jp cm_h+ ld v3, 1+ call pop+cm_h: ld v0, v3 ; header: sign, n length, place, d length+ ld v1, v5+ ld v3, v2+ ld v2, vb+ add v2, 1+ ld i, {H}+ add i, vd+ ld [i], v3+ add vd, 32+ ld vb, 255+ ld v8, 0+ ld v9, 1+ ret++; ---------------------------------------------------------------- the end: reduce all, print+h_end: sne vd, 1+ jp f_end+ ld v7, 0+ se vb, 255+ call commit+he_lp: sne va, 0+ jp he_d+ sne vc, 0+ jp he_p+ call reduce+ jp he_lp+he_p: call pop+ jp he_lp+he_d: se vd, 32+ jp err+ ld v3, 0+ call na ; A = n (v0 = its length)+ ld i, {H}+ ld v3, [i] ; v0 sign, v1 n length, v2 place, v3 d length+ ld v7, v0+ ld v5, v1+ ld v6, v3+ ld i, {A_ADDR}+ ld v0, [i]+ se v5, 1+ jp he_s+ se v0, 0+ jp he_s+ ld v7, 0 ; 0: no sign, no d+ ld v6, 0+he_s: call setmode+ se v6, 0+ jp he_div+ ld i, {A_ADDR}+ ld v0, v5+he_div: ld i, {W1} ; n / d: div prints it+ bsta+ ld i, {D}+ bld b, v6+ jp div+setmode: ld i, {MSAVE} ; MODE as the CALL wrote it, | 8 if v7 (negative)+ ld v0, [i]+ ld v1, 0+ sne v7, 0+ jp sm_m+ ld v1, 8 ; (a tool response prints " -" too, since v16)+sm_m:+ or v0, v1+ ld i, {MODE}+ ld [i], v0+ ret+err: ld i, {MSAVE} ; prints "..."+ ld v0, [i]+ ld i, {MODE}+ ld [i], v0+ ld v0, 11+ ld i, {R_ADDR}+ ld [i], v0+ ld v0, 1++; ---------------------------------------------------------------- reduce: the top operator+reduce: ld v5, vc+ call pop+ ld v0, v5+ sne v0, 12+ jp r_neg+ sne vd, 32+ jp err+ sne vd, 0+ jp err+ ld v6, vd+ add v6, 192+ ld v7, vd+ add v7, 224+ add v0, v0+ jp v0, rtab+r_neg: sne vd, 0+ jp err+ ld v3, vd+ add v3, 224+ ld i, {H}+ add i, v3+ ld v0, [i]+ ld v1, 1+ xor v0, v1+ ld [i], v0+ ret+pop1: add vd, 224+ ret+{CHAIN1}+{MODGCD1}+code1_end:+{MODGCD2}+ org {CODE2} ; (the data from A_ADDR to the end of W sits in between)++hdrs: ld i, {H} ; v3 = sign(Y), DXY = d lengths (X, Y); v0 = sign(X), I = X's header+ add i, v7+ ld v3, [i]+ ld v4, v0+ ld v5, v3+ ld i, {H}+ add i, v6+ ld v3, [i]+ ld v2, v0+ ld v0, v3+ ld v1, v5+ ld i, {DXY}+ ld [i], v1+ ld v0, v2+ ld v3, v4+ ld i, {H}+ add i, v6+ ret+r_mul: call hdrs+ xor v0, v3+ ld [i], v0 ; sign(X) ^= sign(Y)+ ld v3, v6+ call na+ ld v3, v7+ call nb+ call mulm+ ld v3, v6+ call rs_n ; n(X) = n(X) n(Y)+ ld i, {DXY}+ ld v1, [i]+ sne v1, 0+ jp pop1 ; d(Y) = 1+ se v0, 0+ jp rm_dd+ call dcp ; d(X) = 1: d(X) = d(Y)+ jp pop1+rm_dd: ld v3, v6+ call da+ ld v3, v7+ call db+ call mulm+ ld v3, v6+ call rs_d+ jp pop1++r_div: ld v3, v7 ; Y = 0: "..."+ call nb+ se v0, 1+ jp dv_ok+ ld i, {B_ADDR}+ ld v0, [i]+ sne v0, 0+ jp err+dv_ok: call hdrs+ xor v0, v3+ ld [i], v0+ ld i, {DXY}+ ld v1, [i]+ sne v1, 0+ jp dv_d+ ld v3, v6 ; n(X) = n(X) d(Y)+ call na+ ld v3, v7+ call db+ call mulm+ ld v3, v6+ call rs_n+dv_d: ld i, {DXY}+ ld v0, [i]+ se v0, 0+ jp dv_dd+ call ndcp ; d(X) = 1: d(X) = n(Y)+ jp pop1+dv_dd: ld v3, v6 ; d(X) = d(X) n(Y)+ call da+ ld v3, v7+ call nb+ call mulm+ ld v3, v6+ call rs_d+ jp pop1++r_sub: ld i, {H} ; X - Y = X + (-Y)+ add i, v7+ ld v0, [i]+ ld v1, 1+ xor v0, v1+ ld [i], v0+r_add: call hdrs+ ld v1, v3+ ld i, {SGN}+ ld [i], v1 ; SGN = sign(X), sign(Y)+ ld i, {DXY}+ ld v1, [i]+ or v0, v1+ se v0, 0+ jp ad_fr+ ld v3, v6 ; whole numbers+ call na+ ld v3, v7+ call nb+ jp sadd+ad_fr: ld v3, v6 ; T6 = n(X) d(Y)+ call na+ ld i, {DXY}+ ld v1, [i]+ sne v1, 0+ jp a1_n+ ld v3, v7+ call db+ call mulm+ ld v3, {T6}+ call rs_n+ jp a2+a1_n: ld v3, {T6}+ call ast_n+a2: ld v3, v7 ; T7 = n(Y) d(X)+ call na+ ld i, {DXY}+ ld v0, [i]+ sne v0, 0+ jp a2_n+ ld v3, v6+ call db+ call mulm+ ld v3, {T7}+ call rs_n+ jp a3+a2_n: ld v3, {T7}+ call ast_n+a3: ld i, {DXY} ; d(X) = d(X) d(Y)+ ld v1, [i]+ sne v1, 0+ jp a4+ se v0, 0+ jp a3_m+ call dcp+ jp a4+a3_m: ld v3, v6+ call da+ ld v3, v7+ call db+ call mulm+ ld v3, v6+ call rs_d+a4: ld v3, {T6}+ call na+ ld v3, {T7}+ call nb+sadd: ld i, {SGN} ; n(X) = sign(X) A + sign(Y) B+ ld v1, [i]+ se v0, v1+ jp sa_df+ call addm+ ld v3, v6+ call rs_n+ jp pop1+sa_df: call subm+ ld v5, v3+ ld v3, v6+ call rs_n+ se v5, 0+ jp pop1 ; |A| >= |B|: X's sign+ ld i, {SGN1}+ ld v0, [i]+ ld i, {H}+ add i, v6+ ld [i], v0 ; else Y's+ jp pop1++r_pow: ld i, {H} ; Y: a whole number of at most 2 digits+ add i, v7+ ld v3, [i] ; v0 sign, v1 length, v2 place, v3 d length+ se v3, 0+ jp err+ ld v5, v0+ ld v0, v1+ ld v2, 3+ sub v0, v2+ se vf, 0+ jp err+ ld v3, v7+ call nb ; B = its digits, v0 = how many+ ld v4, v0+ ld i, {B_ADDR}+ ld v1, [i]+ se v4, 1+ jp pw_3+ ld v1, v0 ; one digit: e = B[0]+ ld v0, 0+pw_3: ld v2, v0 ; e = 10 v0 + v1+ shl v2, v2+ ld v3, v2+ shl v3, v3+ shl v3, v3+ add v2, v3+ add v2, v1+ ld v0, v2+ ld v1, v5+ ld i, {EXP}+ ld [i], v1 ; EXP = e, EXP + 1 = [e < 0]+ ld v3, v6 ; T6 = X+ call na+ ld v3, {T6}+ call ast_n+ ld i, {H3}+ add i, v6+ ld v0, [i]+ ld i, {T6_DL}+ ld [i], v0+ sne v0, 0+ jp pw_x+ ld i, {D}+ add i, v6+ bld a, v0+ ld i, {T6_D}+ bsta+pw_x: ld i, {EXP}+ ld v0, [i]+ se v0, 0+ jp pw_s+ ld v0, 0 ; e = 0: X = 1+ ld v1, 1+ ld v2, 0+ ld v3, 0+ ld v4, 1+ ld i, {H}+ add i, v6+ ld [i], v4+ jp pop1+pw_s: ld v1, 1 ; even e: no sign+ and v0, v1+ se v0, 0+ jp pw_lp+ ld i, {H}+ add i, v6+ ld [i], v0+pw_lp: ld i, {EXP}+ ld v0, [i]+ add v0, 255+ ld [i], v0+ sne v0, 0+ jp pw_e+ ld v3, v6 ; X = X * T6+ call na+ ld v3, {T6}+ call nb+ call mulm+ ld v3, v6+ call rs_n+ ld i, {T6_DL}+ ld v0, [i]+ sne v0, 0+ jp pw_lp+ ld v3, v6+ call da+ ld v3, {T6}+ call db+ call mulm+ ld v3, v6+ call rs_d+ jp pw_lp+pw_e: ld i, {EXP1}+ ld v0, [i]+ se v0, 0+ call recip+ jp pop1+recip: ld v3, v6 ; X = 1 / X; X = 0: "..."+ call na ; T7 = n(X)+ se v0, 1+ jp rc_ok+ ld i, {A_ADDR}+ ld v0, [i]+ sne v0, 0+ jp err+rc_ok: ld v3, {T7}+ call ast_n+ ld i, {H3}+ add i, v6+ ld v0, [i]+ se v0, 0+ jp rc_d+ ld v0, 1 ; d(X) = 1: n(X) = 1+ ld i, {H4}+ add i, v6+ ld [i], v0+ ld v0, 1+ ld v1, 0+ jp rc_h+rc_d: ld i, {D} ; n(X) = d(X)+ add i, v6+ bld a, v0+ ld i, {H4}+ add i, v6+ bsta+ ld v1, 0+rc_h: ld i, {H1}+ add i, v6+ ld [i], v1 ; n length, place 0+ ld i, {T7_NL} ; d(X) = T7+ ld v0, [i]+ ld i, {T7_N}+ bld a, v0+ ld i, {D}+ add i, v6+ bsta+ ld i, {H3}+ add i, v6+ ld [i], v0+ ret++; ---------------------------------------------------------------- moves (slot offset v3)+na: ld i, {H1} ; A = n(slot), v0 = its length+ add i, v3+ ld v1, [i]+ sne v1, 0+ jp na_s+ ld i, {TAPE_M1}+ add i, v1+ bld a, v0+ ret+na_s: ld i, {H4}+ add i, v3+ bld a, v0+ ret+nb: ld i, {H1} ; B = n(slot), v0 = its length+ add i, v3+ ld v1, [i]+ sne v1, 0+ jp nb_s+ ld i, {TAPE_M1}+ add i, v1+ bld b, v0+ ret+nb_s: ld i, {H4}+ add i, v3+ bld b, v0+ ret+da: ld i, {H3} ; A = d(slot)+ add i, v3+ ld v0, [i]+ ld i, {D}+ add i, v3+ bld a, v0+ ret+db: ld i, {H3} ; B = d(slot)+ add i, v3+ ld v0, [i]+ ld i, {D}+ add i, v3+ bld b, v0+ ret+ast_n: ld i, {LEN} ; n(slot) = A+ ld v0, [i]+ ld i, {H4}+ add i, v3+ bsta+ ld v1, 0+ ld i, {H1}+ add i, v3+ ld [i], v1+ ret+rs_n: ld v1, v0 ; n(slot) = R (over MAX_DIGITS digits: "...")+ ld v2, {MAXD1}+ sub v1, v2+ se vf, 0+ jp err+ bld a, v0+ ld i, {H4}+ add i, v3+ bsta+ ld v1, 0+ ld i, {H1}+ add i, v3+ ld [i], v1+ ret+rs_d: ld v1, v0 ; d(slot) = R+ ld v2, {MAXD1}+ sub v1, v2+ se vf, 0+ jp err+ bld a, v0+ ld i, {D}+ add i, v3+ bsta+ ld i, {H3}+ add i, v3+ ld [i], v0+ ret+dcp: ld i, {H3} ; d(X) = d(Y)+ add i, v7+ ld v0, [i]+ ld i, {D}+ add i, v7+ bld a, v0+ ld i, {D}+ add i, v6+ bsta+ ld i, {H3}+ add i, v6+ ld [i], v0+ ret+ndcp: ld v3, v7 ; d(X) = n(Y)+ call na+ ld i, {D}+ add i, v6+ bsta+ ld i, {H3}+ add i, v6+ ld [i], v0+ ret++; ---------------------------------------------------------------- dispatch+{HANDLERS}+jtab:+{JTAB}+rtab:+{RTAB}+{MODGCD3}+eval_end:++ org {XTAB}+dbuf: db {DBUF_DB}++ org {CODE3}+{CHAIN2}+{MODGCD4}+chain_end:diff --git a/chipformer/q35/lib/macros.py b/chipformer/q35/lib/macros.pynew file mode 100644index 0000000..80a6f2b--- /dev/null+++ b/chipformer/q35/lib/macros.py@@ -0,0 +1,179 @@+"""Bignum macro instructions: the circuit computes a whole row of digits in one record.++They live in CHIP-8's unused 5XYN space (5XY0 is the usual skip) and work on the BIOS layout+(bcd.py): digits most significant first at A_ADDR / B_ADDR, lengths at LEN / LEN+1, result at+R_ADDR. Writes go least significant first so a carry flows from one write to the next. T is a+jump target in the instruction itself, T = 0x200 + 2 * XY.++ 5XY1 BADD R[0..n) = A + B, n = max(LA, LB) + 1, then V0 = n - z, I = R_ADDR + z+ (z = [R[0] == 0]: the print skips a leading zero); PC = T+ 5XY2 BZERO R[0..n) = 0, n = LA + LB, then V7 = n+ 5XY4 BSUB R[0..n) = A - B mod 10^n, n = max(LA, LB) + 1, then V0 and I as BADD;+ PC = T if A >= B, else the next instruction+ 5XY7 BRSUB R[0..n) = B - A (for B >= A), then V0 and I as BADD, MODE |= 8 (the print+ starts with " -"); PC = T+ 5X03 BMUL one schoolbook row of A * B; VX = rows done, row r = LB - 1 - VX:+ for i = LA-1 .. 0: R[i+r+1] = (R[i+r+1] + A[i] * B[r] + carry) mod 10, carry = ..div 10+ then R[r] = carry, VX += 1, and after the last row V0 = LA + LB - z,+ I = R_ADDR + z (z = [R[0] == 0]); PC stays (the next row) until the last row+ 5X13 BMUL0 the first row (VX = 0): the same with R read as 0 (no BZERO needed); PC = the+ next instruction (a BMUL), or the one after it if LB = 1+ 5008 BTRIM if mem[I] == 0, V0 >= 2 and I is in the result: V0 -= 1, I += 1, PC stays;+ else PC = the next instruction (strips leading zeros before a PRINT)++ 5X0B BLD A[j] = mem[I + j] for j < VX, then LA = VX (5X1B: into B and LB); PC = next+ 500C BSTA mem[I + j] = A[j] for j < LA; PC = next (the expression evaluator's moves)++A record whose PC stays has no PC write (the circuit fetches from the unchanged PC).++Each record's values come from the state before the record plus the carry chain inside it, so a+multiplication costs LB records of LA + 4 writes.++Long division works in place on W (bcd.W_ADDR: a 0, the digits of A, then DEC zeros), with the+divisor B and the quotient's digits in P (bcd.R_ADDR; P[t] for t <= LA, P[t + 1] after the "."):++ 5XY5 BDIV one quotient digit, t = VX: the remainder window W[t-LB .. t] -= q * B, P[.] = q,+ with q estimated from the window's top 3 digits and B's top 2 (q <= true + 2)+ 5XY6 BADB the add-back after an estimate that was too big: W[t-LB .. t] += B, P[.] -= 1++Both end with VF = 1 (negative: add back), 2 (done, exact), 3 (done, DS decimals and a+remainder) or 0 (next digit, VX = t + 1). BDIV repeats itself (PC unchanged) on 0 and BADB on 1,+so a division needs loop instructions only around its add-backs. DS = mem[bcd.DSTOP] (div:+DEC; mod and gcd: 0, the remainder of the integer part).+"""++from chipformer import spec+from chipformer.q35.lib import bcd++BADD, BZERO, BMUL, BSUB, BDIV, BADB, BRSUB, BTRIM = 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8+BLD, BSTA = 0xB, 0xC # the expression evaluator's moves (eval.asm)+KINDS = (BADD, BZERO, BMUL, BSUB, BDIV, BADB, BRSUB, BTRIM, BLD, BSTA)+R_CELLS = 2 * bcd.MAX_DIGITS # the result region BTRIM can read+++def target(op: int) -> int:+ return 0x200 + 2 * (op >> 4 & 0xFF)+++def div_estimate(W: list[int], B: list[int], t: int) -> int:+ """BDIV's quotient digit: never below the true one, at most 2 above it (for B[0] != 0)."""+ lb = len(B)+ if lb == 1:+ r3, b2, delta = 10 * W[t - 1] + W[t], B[0], 0+ else:+ r3 = 100 * W[t - lb] + 10 * W[t - lb + 1] + W[t - lb + 2]+ b2, delta = 10 * B[0] + B[1], int(lb >= 3)+ return min(9, (r3 + delta) // b2) if b2 else 9+++def _div_record(cells: list[int], op: int, kind: int) -> list[tuple[int, int]]:+ la, lb = cells[bcd.LEN], cells[bcd.LEN + 1]+ B = [cells[bcd.B_ADDR + i] for i in range(lb)]+ x = op >> 8 & 0xF+ t = cells[spec.V + x]+ W = [cells[(bcd.W_ADDR + i) & 0xFFF] for i in range(64)]+ p_addr = bcd.R_ADDR + t + int(t > la)+ q = div_estimate(W, B, t) if kind == BDIV else 0+ # the quotient digit goes first: later tokens of the record read it from the token stream+ w = [(p_addr, q if kind == BDIV else (cells[p_addr] - 1) & 0xFF)]+ carry, digits = 0, []+ for k in range(lb): # least significant first+ i = lb - 1 - k+ if kind == BDIV:+ v = W[t - k] - q * B[i] - carry+ d = v % 10+ carry = (d - v) // 10 # the borrow+ else:+ v = W[t - k] + B[i] + carry+ d, carry = v % 10, v // 10+ w.append((bcd.W_ADDR + t - k, d))+ digits.append(d)+ v = W[t - lb] + (carry if kind == BADB else -carry)+ d = v % 10+ w.append((bcd.W_ADDR + t - lb, d))+ digits.append(d)+ # BDIV: negative if the top digit borrowed; BADB: still negative unless it carried out+ neg = v < 0 if kind == BDIV else v < 10+ zero = not neg and not any(digits)+ flag = 1 if neg else 2 if zero and t >= la else 3 if t == la + cells[bcd.DSTOP] else 0+ w.append((spec.V + x, t + (flag == 0)))+ w.append((spec.V + 15, flag))+ pc = cells[spec.PC]+ again = flag == (0 if kind == BDIV else 1)+ return w + [(spec.PC, pc if again else (pc + 2) & 0xFFF)]+++def macro_writes(cells: list[int], op: int):+ """The record for opcode ``op`` if it is a macro, else None."""+ if op >> 12 != 5 or op & 0xF not in KINDS:+ return None+ if op & 0xF in (BDIV, BADB):+ return _div_record(cells, op, op & 0xF)+ la, lb = cells[bcd.LEN], cells[bcd.LEN + 1]+ A = [cells[bcd.A_ADDR + i] for i in range(la)]+ B = [cells[bcd.B_ADDR + i] for i in range(lb)]+ pc = cells[spec.PC]+ nxt = (pc + 2) & 0xFFF+ kind = op & 0xF+ w = []+ if kind == BLD:+ n, i, to_b = cells[spec.V + (op >> 8 & 0xF)] & 31, cells[spec.I], op >> 4 & 1+ base = bcd.B_ADDR if to_b else bcd.A_ADDR+ w = [(base + j, cells[(i + j) & 0xFFF]) for j in range(n)] + [(bcd.LEN + to_b, n)]+ return w + [(spec.PC, nxt)]+ if kind == BSTA:+ i = cells[spec.I]+ w = [((i + j) & 0xFFF, cells[bcd.A_ADDR + j] & 15) for j in range(la & 31)]+ return w + [(spec.PC, nxt)]+ if kind in (BADD, BSUB, BRSUB):+ n = max(la, lb) + 1+ carry = 0+ for k in range(n): # w-th write is digit k from the right+ a = A[la - 1 - k] if k < la else 0+ b = B[lb - 1 - k] if k < lb else 0+ if kind == BRSUB:+ a, b = b, a+ if kind == BADD:+ s = a + b + carry+ w.append((bcd.R_ADDR + n - 1 - k, s % 10))+ carry = s // 10+ else: # carry is the borrow+ s = a - b - carry+ w.append((bcd.R_ADDR + n - 1 - k, s % 10))+ carry = int(s < 0)+ mode = cells[bcd.MODE]+ z = int(w[-1][1] == 0) # the top digit, R[0]+ w.append((spec.V, n - z))+ w.append((spec.I, bcd.R_ADDR + z))+ if kind == BRSUB:+ w.append((bcd.MODE, mode | 8))+ new_pc = nxt if kind == BSUB and carry else target(op)+ elif kind == BZERO:+ n = la + lb+ w = [(bcd.R_ADDR + k, 0) for k in range(n)] + [(spec.V + 7, n)]+ new_pc = nxt+ elif kind == BMUL:+ x, first = op >> 8 & 0xF, op >> 4 & 0xF == 1+ d = cells[spec.V + x]+ row = lb - 1 - d+ b = B[row] if 0 <= row < lb else 0+ carry = 0+ for i in range(la - 1, -1, -1):+ k = i + row + 1+ t = (0 if first else cells[bcd.R_ADDR + k]) + A[i] * b + carry+ w.append((bcd.R_ADDR + k, t % 10))+ carry = t // 10+ w.append((bcd.R_ADDR + row, carry))+ w.append((spec.V + x, (d + 1) & 0xFF))+ last = d + 1 == lb+ if last:+ z = int(carry == 0) # R[0]+ w += [(spec.V, la + lb - z), (spec.I, bcd.R_ADDR + z)]+ new_pc = ((nxt + 2) & 0xFFF if last else nxt) if first else nxt if last else pc+ else: # BTRIM+ i, v0 = cells[spec.I], cells[spec.V]+ z = bcd.R_ADDR <= i < bcd.R_ADDR + R_CELLS and cells[i] & 15 == 0 and v0 >= 2+ if z:+ w = [(spec.V, v0 - 1), (spec.I, i + 1)]+ new_pc = pc if z else nxt+ return w if new_pc == pc else w + [(spec.PC, new_pc)]diff --git a/chipformer/q35/merge.py b/chipformer/q35/merge.pynew file mode 100644index 0000000..a3fa5ce--- /dev/null+++ b/chipformer/q35/merge.py@@ -0,0 +1,379 @@+"""Widen a Qwen3.5 checkpoint and append chip layers, keeping Qwen's own computation intact.++Qwen's matrices are zero-padded to the new hidden size, so its dims never read or write chip+dims. The RMSNorms that see the full residual (input, post-attention, final) average over more+dims after widening; their gains are scaled by sqrt(D / D') so Qwen's normalized inputs stay the+same up to one bf16 rounding. The MTP head is dropped: HF ignores it, and its shapes would change.+"""++import copy+import json+import re+from pathlib import Path++import torch+from safetensors import safe_open++TEXT = "model.language_model."+# which axis of each weight is the hidden dim, by name+IN_COLS = re.compile(+ r"(in_proj_(qkv|z|a|b)|q_proj|k_proj|v_proj|gate_proj|up_proj)\.weight$|embed_tokens\.weight$"+)+OUT_ROWS = re.compile(r"(out_proj|o_proj|down_proj)\.weight$|visual\.merger\.linear_fc2\.(weight|bias)$")+WIDE_NORMS = re.compile(r"(input_layernorm|post_attention_layernorm)\.weight$|language_model\.norm\.weight$")++IM_START = 248045 # <|im_start|> in Qwen3.5's tokenizer+++def load(path):+ path = Path(path)+ cfg = json.loads((path / "config.json").read_text())+ sd = {}+ for f in sorted(path.glob("*.safetensors")):+ with safe_open(f, "pt") as fh:+ for k in fh.keys(): # noqa: SIM118 - safe_open is not iterable+ if not k.startswith("mtp."):+ sd[k] = fh.get_tensor(k)+ return cfg, sd+++def widen(cfg, sd, hidden, n_chip_layers=0):+ """Return (config, state_dict) with hidden size `hidden` and n zero chip layers appended."""+ tc = cfg["text_config"]+ d = tc["hidden_size"]+ assert hidden >= d+ scale = (d / hidden) ** 0.5+ out = {}+ for k, w in sd.items():+ if IN_COLS.search(k):+ w = torch.cat([w, w.new_zeros(*w.shape[:-1], hidden - d)], -1)+ elif OUT_ROWS.search(k):+ w = torch.cat([w, w.new_zeros(hidden - d, *w.shape[1:])], 0)+ elif WIDE_NORMS.search(k):+ g = (1.0 + w.float()) * scale - 1.0 # zero-centred gain: (1 + w)+ w = torch.cat([g.to(w.dtype), w.new_zeros(hidden - d)])+ out[k] = w+ cfg = copy.deepcopy(cfg)+ tc = cfg["text_config"]+ tc["hidden_size"] = hidden+ cfg["vision_config"]["out_hidden_size"] = hidden+ tc["mtp_num_hidden_layers"] = 0+ tc.pop("full_attention_interval", None)+ n0 = tc["num_hidden_layers"]+ tc["layer_types"] = tc["layer_types"] + ["linear_attention"] * n_chip_layers+ tc["num_hidden_layers"] = n0 + n_chip_layers+ for i in range(n0, n0 + n_chip_layers):+ out.update(empty_chip_layer(tc, i, out[f"{TEXT}layers.0.input_layernorm.weight"].dtype))+ return cfg, out+++def empty_chip_layer(tc, i, dtype):+ """Zero weights for a linear-attention layer: on text tokens it adds exactly 0."""+ h, nk, nv = tc["hidden_size"], tc["linear_num_key_heads"], tc["linear_num_value_heads"]+ dk, dv, inter = tc["linear_key_head_dim"], tc["linear_value_head_dim"], tc["intermediate_size"]+ p = f"{TEXT}layers.{i}."+ z = lambda *s: torch.zeros(*s, dtype=dtype)+ return {+ p + "input_layernorm.weight": z(h),+ p + "post_attention_layernorm.weight": z(h),+ p + "linear_attn.in_proj_qkv.weight": z(2 * nk * dk + nv * dv, h),+ p + "linear_attn.in_proj_z.weight": z(nv * dv, h),+ p + "linear_attn.in_proj_a.weight": z(nv, h),+ p + "linear_attn.in_proj_b.weight": z(nv, h),+ p + "linear_attn.conv1d.weight": z(2 * nk * dk + nv * dv, 1, tc["linear_conv_kernel_dim"]),+ p + "linear_attn.dt_bias": z(nv),+ p + "linear_attn.A_log": z(nv),+ p + "linear_attn.norm.weight": torch.ones(dv, dtype=dtype),+ p + "linear_attn.out_proj.weight": z(h, nv * dv),+ p + "mlp.gate_proj.weight": z(inter, h),+ p + "mlp.up_proj.weight": z(inter, h),+ p + "mlp.down_proj.weight": z(h, inter),+ }+++def instantiate(cfg, sd, dtype=torch.bfloat16):+ from transformers import Qwen3_5Config, Qwen3_5ForConditionalGeneration++ config = Qwen3_5Config(**{k: v for k, v in cfg.items() if k not in ("architectures", "model_type")})+ with torch.device("meta"):+ model = Qwen3_5ForConditionalGeneration(config)+ missing, unexpected = model.load_state_dict(sd, strict=False, assign=True)+ missing = [k for k in missing if k != "lm_head.weight"] # tied to embed_tokens+ assert not missing and not unexpected, (missing[:5], unexpected[:5])+ model.tie_weights()+ # rotary tables are non-persistent buffers, still on the meta device: rebuild them+ from transformers.models.qwen3_5 import modeling_qwen3_5 as q++ inner = model.model+ inner.language_model.rotary_emb = q.Qwen3_5TextRotaryEmbedding(config=config.text_config)+ inner.visual.rotary_pos_emb = q.Qwen3_5VisionRotaryEmbedding(config.vision_config)+ return model.to(dtype).eval()+++# --- the merged checkpoint -----------------------------------------------------------------++TAU_TEXT = 2.0**-40 # tau in every text token's embedding row+TAU_GAIN = 2.0**60 # final-norm gain on tau: chip logits at text positions ~ -2^20 / rms+++def chip_token_names() -> list[str]:+ from chipformer import spec++ names = []+ for t in range(spec.VOCAB_SIZE):+ if spec.is_val(t):+ names.append(f"<|chip={t - spec.VAL_BASE:#05x}|>")+ elif spec.is_addr(t):+ names.append(f"<|chip@{t - spec.ADDR_BASE:#06x}|>")+ else:+ names.append({spec.STEP: "<|chip_step|>", spec.TICK: "<|chip_tick|>",+ spec.BOOT: "<|chip_boot|>"}[t])+ return names+++def extend_tokenizer(src: Path, dst: Path, first_id: int) -> None:+ """Copy the tokenizer, padding with reserved tokens up to first_id, then the chip tokens."""+ import shutil++ from tokenizers import AddedToken, Tokenizer++ tok = Tokenizer.from_file(str(src / "tokenizer.json"))+ n = tok.get_vocab_size(with_added_tokens=True)+ pad = [AddedToken(f"<|reserved_{i}|>", special=True) for i in range(first_id - n)]+ chip = [AddedToken(s, special=True) for s in chip_token_names()]+ tok.add_special_tokens(pad + chip)+ assert tok.token_to_id(chip_token_names()[0]) == first_id+ dst.mkdir(parents=True, exist_ok=True)+ tok.save(str(dst / "tokenizer.json"))+ for f in ("chat_template.jinja", "preprocessor_config.json",+ "video_preprocessor_config.json", "generation_config.json", "LICENSE"):+ if (src / f).exists():+ shutil.copy(src / f, dst / f)+ tc = json.loads((src / "tokenizer_config.json").read_text(encoding="utf-8"))+ dec = tc.setdefault("added_tokens_decoder", {})+ for i, t in enumerate(pad + chip):+ dec[str(n + i)] = {"content": t.content, "lstrip": False, "normalized": False,+ "rstrip": False, "single_word": False, "special": True}+ (dst / "tokenizer_config.json").write_text(json.dumps(tc, indent=2, ensure_ascii=False),+ encoding="utf-8")+++def pick_hidden(base: int, n_chip: int) -> tuple[int, int]:+ """Smallest (hidden, m) with hidden = m * 4^k >= base + m + n_chip, a multiple of 128, and+ 3 <= m <= 64 ballast dims (m * B^2 / hidden must be a power of 4)."""+ best = None+ for m in range(3, 65):+ k = 0+ while m * 4**k < base + m + n_chip or (m * 4**k) % 128:+ k += 1+ h = m * 4**k+ if best is None or h < best[0] or (h == best[0] and m > best[1]):+ best = (h, m)+ return best+++ID_GAIN = 4.0 # the store's id code: final-norm gain, relative to the class code's+++def store_row_features(c, lay, v0: int) -> dict[int, torch.Tensor]:+ """Residual index -> value per Qwen row (v0 rows) of the features that read a token's own+ id (the object store's): its id bits and id code, newline and tag detectors."""+ from chipformer.q35 import store++ vocab = c.store.vocab+ out = {}+ toks = [store.qtok(q, vocab.cls(q)) for q in range(v0)]+ for d in sorted(c.full_fns):+ col = torch.tensor([float(c.feature_value(d, t)) for t in toks])+ out[lay.dim(d)] = col+ return out+++def build_merged(src, dst, hidden: int | None = None, m: int | None = None,+ mute_qwen: bool = True, overlay: bool = True, store: bool = False):+ """Write Qwen3.5 + the CHIP-8 circuit as one checkpoint ("v2" layout); returns (circuit,+ layout).++ The chip layers come first. Every token carries the ballast and the features of its+ class (chip token, or one of bridge.py's text classes); the last chip sublayer clears all+ of it, leaving each token's Qwen part, mu (1 on chip tokens), the output code and tau, so+ Qwen's layers see their usual input on text and a muted, empty token at chip positions.++ With ``overlay`` the circuit dims that only chip tokens need live in Qwen's dims+ (overlay.py), which makes the checkpoint much narrower.+ """+ from safetensors.torch import save_file+ from transformers import AutoTokenizer++ from chipformer import spec+ from chipformer.q35 import bridge, chip8_q35, tiny+ from chipformer.q35.emit import Layout, allocate, check_bf16, embedding_rows, emit_layers+ from chipformer.q35.ir import Target++ src, dst = Path(src), Path(dst)+ cfg, sd = load(src)+ tc = cfg["text_config"]+ d0, v0, inter = tc["hidden_size"], tc["vocab_size"], tc["intermediate_size"]+ vocab = None+ if store:+ from chipformer.q35.store import Vocab++ vocab = Vocab(src / "tokenizer.json")+ target = Target(tc["linear_num_key_heads"], tc["linear_num_value_heads"], inter)+ per_dim = (2 * tc["linear_num_key_heads"] * tc["linear_key_head_dim"]+ + 3 * tc["linear_num_value_heads"] * tc["linear_value_head_dim"] + 3 * inter)++ def circuit(schedule):+ c = chip8_q35.build(target, store=vocab, schedule=schedule)+ keep = set(c.keep)+ plan = None+ if overlay:+ from chipformer.q35 import overlay as ov++ plan = ov.build(c, keep)+ print(plan.summary(), flush=True)+ alloc = allocate(c, None, keep, inter, overlay=plan.dims if plan else None,+ overlay_cap=d0)+ h, m_ = pick_hidden(d0, 1 + alloc.n_slots) if hidden is None else (hidden, m)+ # weights (roughly): every layer and the embedding grow with the width+ size = h * ((tc["num_hidden_layers"] + c.n_layers()) * per_dim + v0)+ return size, c, keep, plan, alloc, h, m_++ # The list scheduler (ir.Graph.reschedule) saves chip layers, but packs work closer, so more+ # circuit dims are live at once: build both and keep the smaller checkpoint (4B: scheduled;+ # 0.8B, whose overlay pool is full: build order)+ options = [circuit(True)] + ([] if hidden is not None else [circuit(False)])+ size, c, keep, plan, alloc, hidden, m = min(options, key=lambda o: o[0])+ print(f"circuit: {c.n_layers()} layers, {alloc.n_slots} chip slots, hidden {hidden}",+ flush=True)+ code = [next(iter(e.terms)) for e in c.code]+ id_code = [next(iter(e.terms)) for e in c.store.yc] if store else []+ tau, mu = next(iter(c.tau.terms)), next(iter(c.mu.terms))+ lay = Layout(hidden=hidden, off=d0, m=m, slots=alloc.slots, over=alloc.over)+ assert d0 + m + 1 + alloc.n_slots <= hidden, (alloc.n_slots, hidden)+ lay.scale # noqa: B018 - asserts m * B^2 / hidden is a power of 4++ n_chip = c.n_layers()+ cfg, sd = widen(cfg, sd, hidden)+ tc = cfg["text_config"]+ n0 = tc["num_hidden_layers"]+ # Qwen's layers move behind the chip layers+ shift = re.compile(r"^(model\.language_model\.layers\.)(\d+)\.")+ sd = {shift.sub(lambda mt: f"{mt[1]}{int(mt[2]) + n_chip}.", k): v for k, v in sd.items()}+ tc["layer_types"] = ["linear_attention"] * n_chip + tc["layer_types"]+ tc["num_hidden_layers"] = n_chip + n0+ vocab = -(-(v0 + spec.VOCAB_SIZE) // 64) * 64+ tc["vocab_size"] = vocab++ if mute_qwen:+ from chipformer.q35.mute import fit_anti_sink, mute++ dev = "cuda" if torch.cuda.is_available() else "cpu"+ sinks = fit_anti_sink(src, dev)+ mute(sd, tc, n0, lay.dim(mu), hidden**0.5, sinks=sinks, first=n_chip)+ layers = emit_layers(c, lay, tc, 0, TEXT, alloc, keep, chip=c.mu)+ sd.update({k: v.to(torch.bfloat16) for k, v in layers.items()})+ del layers++ # embeddings: Qwen's rows keep their Qwen part and get their text class's chip features+ tok = AutoTokenizer.from_pretrained(src)+ text_ids = {}+ for cls, string in bridge.QWEN_TEXT.items():+ ids = tok.encode(string, add_special_tokens=False)+ assert len(ids) == 1, (string, ids)+ text_ids[cls] = ids[0]+ classes = list(range(bridge.TEXT_BASE, bridge.TEXT_BASE + bridge.N_TEXT))+ rows = dict(zip(classes, embedding_rows(c, lay, classes, hidden).to(torch.bfloat16)))+ full = torch.zeros(vocab, hidden, dtype=torch.bfloat16)+ full[:v0, :d0] = sd[TEXT + "embed_tokens.weight"][:, :d0]+ full[:v0, d0:] = rows[bridge.OTHER][d0:]+ for cls, i in text_ids.items():+ full[i, d0:] = rows[cls][d0:]+ for cls, strings in bridge.EXTRA_TEXT.items():+ for string in strings:+ (i,) = tok.encode(string, add_special_tokens=False)+ full[i, d0:] = rows[cls][d0:]+ if store: # every Qwen row also carries its own id bits and id code (store_q35.py)+ for col, vals in store_row_features(c, lay, v0).items():+ full[:v0, col] = vals.to(torch.bfloat16)+ chip = embedding_rows(c, lay, list(range(spec.VOCAB_SIZE)), hidden)+ full[v0 : v0 + spec.VOCAB_SIZE] = chip.to(torch.bfloat16)+ sd[TEXT + "embed_tokens.weight"] = full+ # image tokens come from the vision merger: give them the OTHER class too+ bias = "model.visual.merger.linear_fc2.bias"+ if bias in sd:+ sd[bias] = sd[bias].clone()+ sd[bias][d0:] = rows[bridge.OTHER][d0:].to(sd[bias].dtype)++ norm = sd[TEXT + "norm.weight"].float()+ norm[d0:] = -1.0 # chip dims are invisible to the lm_head ...+ for d in code: # ... except the output code and tau+ norm[lay.dim(d)] = tiny.code_gain(lay)+ for d in id_code: # (an any-print beats its 16-of-18-bit neighbours by 8 code units)+ norm[lay.dim(d)] = ID_GAIN * tiny.code_gain(lay)+ norm[lay.dim(tau)] = TAU_GAIN+ check_bf16("final norm", norm[d0:])+ sd[TEXT + "norm.weight"] = norm.to(torch.bfloat16)++ dst.mkdir(parents=True, exist_ok=True)+ shard, size, shards = {}, 0, []+ for k in sorted(sd):+ shard[k] = sd[k].contiguous()+ size += sd[k].numel() * 2+ if size > 2**31:+ shards.append(shard)+ shard, size = {}, 0+ shards.append(shard)+ index = {"metadata": {"total_size": sum(t.numel() * 2 for t in sd.values())}, "weight_map": {}}+ for i, sh in enumerate(shards):+ name = f"model-{i + 1:05d}-of-{len(shards):05d}.safetensors"+ save_file(sh, str(dst / name), metadata={"format": "pt"})+ index["weight_map"].update({k: name for k in sh})+ (dst / "model.safetensors.index.json").write_text(json.dumps(index, indent=2))+ (dst / "config.json").write_text(json.dumps(cfg, indent=2))+ extend_tokenizer(src, dst, v0)+ gen_path = dst / "generation_config.json"+ gen = json.loads(gen_path.read_text()) if gen_path.exists() else {}+ if "eos_token_id" not in gen:+ eos = cfg["text_config"].get("eos_token_id", 248044)+ gen.update(eos_token_id=[248046, eos], pad_token_id=eos)+ # After the circuit prints a tool response inside the assistant turn, Qwen sometimes opens a+ # new turn (<|im_start|>user ...) instead of going on. Qwen never writes <|im_start|> in its+ # own output, so the merged model bans it; the circuit never predicts it either.+ gen["bad_words_ids"] = [[IM_START]]+ gen_path.write_text(json.dumps(gen, indent=2))+ meta = {"layout": "v2", "chip_token_offset": v0, "chip_vocab": spec.VOCAB_SIZE,+ "chip_layers": n_chip, "qwen_layers_from": n_chip, "hidden": hidden,+ "chip_dims_offset": d0, "ballast_dims": m, "circuit": c.stats(),+ "slots": alloc.n_slots, "overlay_slots": len(alloc.final_over),+ "overlay": plan.summary() if plan else None, "muted": mute_qwen,+ "text_ids": {str(k): v for k, v in text_ids.items()}}+ if store:+ from chipformer.q35 import store_q35++ meta["store"] = {"chunks": c.store.n_chunks, "tokens_per_row": c.store.tpr,+ "persistent_heads": store_q35.persistent_heads(c)}+ (dst / "chipformer.json").write_text(json.dumps(meta, indent=2))+ return c, lay+++def main() -> None:+ import argparse++ ap = argparse.ArgumentParser(description="Merge the CHIP-8 circuit into a Qwen3.5 checkpoint")+ ap.add_argument("src", help="Qwen3.5 checkpoint directory")+ ap.add_argument("--out", default="dist/qwen3.5-0.8b-chip8")+ ap.add_argument("--hidden", type=int, default=None)+ ap.add_argument("--ballast", type=int, default=None)+ ap.add_argument("--no-mute", action="store_true", help="leave Qwen's layers active on chip tokens")+ ap.add_argument("--no-overlay", action="store_true",+ help="give every circuit dim its own dim (the v15 layout, much wider)")+ ap.add_argument("--store", action="store_true", help="add the object store (store.py)")+ args = ap.parse_args()+ c, lay = build_merged(args.src, args.out, args.hidden, args.ballast, not args.no_mute,+ not args.no_overlay, args.store)+ print(f"{args.out}: hidden {lay.hidden}, {c.n_layers()} chip layers, {c.stats()}")+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/mute.py b/chipformer/q35/mute.pynew file mode 100644index 0000000..be78979--- /dev/null+++ b/chipformer/q35/mute.py@@ -0,0 +1,98 @@+"""Make Qwen's own layers ignore chip tokens.++On a chip token every RMSNorm output on the ballast dims is a known constant (the normed+ballast, ``Layout.scale * B``), and on a text token it is exactly 0. Weights on one ballast dim can+therefore act on chip tokens alone, leaving text untouched bit for bit:++ Gated DeltaNet a = -1024: softplus(a + dt_bias) = 0, so g = 0 and the state does not decay+ b = -1024: beta = 0, so nothing is written+ z = -1024: silu(z) = 0, so the output is exactly 0+ full attention output gate = -1024: sigmoid = 0, so the output is exactly 0++So Qwen's recurrent state passes through a run of chip tokens unchanged, and chip positions add+nothing to Qwen's residual. Chip positions are still keys in Qwen's full attention (with values+~0); an "anti-sink" key, pointing against the mean query of each KV head (fitted on text), keeps+them from soaking up attention. What remains is a fixed boundary effect of Qwen's 3-token conv+window, independent of the trace length (``report.py``'s post-trace section).+"""++import re++import torch++MUTE = -1024.0+++ROTARY = 64 # dims of each attention head that RoPE rotates; the anti-sink key avoids them+SINK_NORM = 16.0++[email protected]_grad()+def fit_anti_sink(base_dir, device: str = "cpu") -> dict[int, torch.Tensor]:+ """Per full-attention layer, (n_kv_heads, head_dim) keys opposite to the mean text query."""+ from transformers import AutoModelForImageTextToText, AutoTokenizer++ from chipformer.q35.report import PROMPTS, encode++ tok = AutoTokenizer.from_pretrained(base_dir)+ model = AutoModelForImageTextToText.from_pretrained(base_dir, dtype=torch.bfloat16).to(device)+ lm, tc = model.model.language_model, model.config.text_config+ layers = {i: l.self_attn for i, l in enumerate(lm.layers)+ if tc.layer_types[i] == "full_attention"}+ sums = {i: 0 for i in layers}+ count = 0+ hooks = [a.q_norm.register_forward_hook(+ lambda m, inp, out, i=i: sums.__setitem__(i, sums[i] + out.float().flatten(0, -3).sum(0)))+ for i, a in layers.items()]+ for kind, text in PROMPTS:+ ids = encode(tok, kind, text)+ model(input_ids=torch.tensor([ids], device=device))+ count += len(ids)+ for h in hooks:+ h.remove()+ hd, nkv = tc.head_dim, tc.num_key_value_heads+ grp = tc.num_attention_heads // nkv+ out = {}+ for i, a in layers.items():+ mean = sums[i] / count # (heads, head_dim)+ gain = 1.0 + a.k_norm.weight.float()+ keys = torch.zeros(nkv, hd)+ for g in range(nkv):+ u = -mean[g * grp:(g + 1) * grp].mean(0)[ROTARY:].cpu() / gain[ROTARY:].cpu()+ keys[g, ROTARY:] = u / u.norm() * SINK_NORM+ out[i] = keys+ return out+++def mute(sd: dict, tc: dict, n_qwen: int, ballast_dim: int, normed_ballast: float,+ prefix: str = "model.language_model.", sinks: dict | None = None,+ first: int = 0) -> None:+ """Patch Qwen's layers (indices first .. first+n_qwen-1) of ``sd`` in place.++ ``ballast_dim`` is any dim that is a known constant on chip tokens and exactly 0 on text+ (the ballast in the v1 layout, the mu marker after the chip layers' exit in v2);+ ``normed_ballast`` is its value after the RMSNorm. ``sinks`` is keyed by Qwen's own index.+ """+ w = MUTE / normed_ballast+ hd = tc["head_dim"]+ for i in range(n_qwen):+ p = f"{prefix}layers.{first + i}."+ if tc["layer_types"][first + i] == "linear_attention":+ for name in ("in_proj_a", "in_proj_b", "in_proj_z"):+ t = sd[f"{p}linear_attn.{name}.weight"]+ t[:, ballast_dim] = w+ else:+ q = sd[f"{p}self_attn.q_proj.weight"] # rows: per head [query (hd), gate (hd)]+ for h in range(q.shape[0] // (2 * hd)):+ q[h * 2 * hd + hd : (h + 1) * 2 * hd, ballast_dim] = w+ if sinks is not None:+ k = sd[f"{p}self_attn.k_proj.weight"]+ k[:, ballast_dim] = (sinks[i].flatten() / normed_ballast).to(k.dtype)+++def is_muted(sd: dict, ballast_dim: int) -> bool:+ keys = [k for k in sd if re.search(r"linear_attn\.in_proj_b\.weight$", k)]+ return bool(keys) and bool((sd[keys[0]][:, ballast_dim] != 0).all())+++__all__ = ["MUTE", "is_muted", "mute", "torch"]diff --git a/chipformer/q35/overlay.py b/chipformer/q35/overlay.pynew file mode 100644index 0000000..bd97f19--- /dev/null+++ b/chipformer/q35/overlay.py@@ -0,0 +1,448 @@+"""Overlay: circuit dims that live only on chip tokens share Qwen's dims in a merged checkpoint.++The chip layers run first, and while they do every text token carries its Qwen embedding in dims+[0, D0); on chip tokens those dims are 0. So a circuit dim that is 0 at every text position can+use a Qwen dim as its residual slot (emit.Layout.over): a chip token sees the circuit's value+there, a text token sees q, a Qwen embedding value. That stays exact when++1. nothing writes into an overlaid slot at a text position, so Qwen's input is bit-identical:+ every unit and head column that writes an overlaid dim is switched off on text tokens;+2. whatever reads an overlaid slot at a text position is switched off there, or cannot matter:+ - an FFN unit: its gate gets -KILL * is_text, so silu is exactly 0 whatever q is;+ - a head that writes no key on text tokens: its key channels that read the overlay get the+ same switch, so the key stays 0 (no state write), and every output gate z of a column+ that is 0 on text does too (a query of q then reads nothing out);+3. nothing that matters on text tokens reads an overlaid dim at all: units and head columns that+ are nonzero there, keys written there (the capture), decays, and channels with conv lag taps+ (which carry a text token's projection into the next token).++The handovers and the exit clear of overlaid slots are gated by mu (chip tokens) instead of ONE+(emit.emit_layers), so a text token keeps q.++What is 0 or switched off on text tokens is measured on ``corpus()``, mixed traces of Qwen text,+inline expressions, tool calls and prints. ``apply`` writes the switches into the graph, so the+circuit's own IR follows them: a trace where a switched-off unit should have fired shows up as+an IR mismatch against the reference like any other circuit bug.+"""++import random+from dataclasses import dataclass, field++import numpy as np++from chipformer.dsl.graph import ONE, Expr+from chipformer.q35 import bridge as B+from chipformer.q35 import ir+from chipformer.q35.ir import FFN, K, Chan, Graph, Head, conv, ev++KILL = 2**20 # -KILL * is_text switches a gate, key or output gate off on text tokens+++@dataclass+class Plan:+ dims: set[int] = field(default_factory=set) # dims that may use Qwen's dims+ terms: set[tuple[int, int]] = field(default_factory=set) # (node, term): FFN units+ keys: set[tuple[int, int]] = field(default_factory=set) # (node, key channel)+ gates: set[tuple[int, int]] = field(default_factory=set) # (node, value column): z+ q_bound: float = 0.0 # the largest |q| every switched-off value still absorbs++ def summary(self) -> str:+ return (f"overlay: {len(self.dims)} dims, {len(self.terms)} units, {len(self.keys)} key "+ f"channels and {len(self.gates)} output gates switched off on text, |q| <= "+ f"{self.q_bound:.4g}")+++# --- the corpus ------------------------------------------------------------------------------++FORMS = {"add": [[B.PLUS]], "sub": [[B.MINUS], [B.MINUS2]],+ "mul": [[B.TIMES], [B.BSL, B.TIMESW, B.SPACE]],+ "div": [[B.DIV], [B.DIV2], [B.BSL, B.FN_DIV, B.SPACE]]}+LEADS = [[B.SPACE], [B.DOLLAR], [B.SDOLLAR, B.SPACE], [B.DDOLLAR], [B.SDDOLLAR], [B.NL],+ [B.WS], [B.LPAREN], [B.OTHER], [B.EQ, B.SPACE]]+++def _digits(rng: random.Random, n: int, commas: bool) -> list[int]:+ s = str(rng.randrange(10 ** (n - 1), 10**n))+ out = []+ for i, ch in enumerate(s):+ if commas and i and (len(s) - i) % 3 == 0:+ out.append(B.COMMA)+ out.append(B.HEX[int(ch)])+ return out+++def _expr(rng: random.Random) -> list[int]:+ op = rng.choice(list(FORMS))+ commas = rng.random() < 0.3+ a = _digits(rng, rng.randint(1, 7), commas)+ b = _digits(rng, rng.randint(1, 5), commas)+ gap = [B.SPACE] if rng.random() < 0.5 else []+ return rng.choice(LEADS) + a + rng.choice(FORMS[op]) + gap + b + [B.EQ]+++def _number(rng: random.Random, n: int) -> str:+ return str(rng.randrange(10 ** (n - 1), 10**n))+++# v16 / v18 forms: the overlay is planned from what the circuit does on text in this corpus, so+# every capture path must run in it: signed numbers, mod and gcd (v16), whole expressions and+# the calc tool (v18)+EXPR_TOKENS = [" +", " -", " *", " /", " ^", "^{", "(", " (", ")", " )", "(-", " (-", ")*", ")/",+ ")+", ")-", "*(", "/(", "+(", "-(", ")*(", ")/(", "((", "))", "))*", "*(-", "/-",+ "+-", "*-", "**", "$(", "{", "}"]+++def _signed(rng: random.Random, digits: list[int]) -> list[int]:+ r = rng.random()+ if r < 0.4:+ return [B.MINUS2] + digits+ return [B.LPMINUS if r < 0.7 else B.SLPMINUS] + digits + [B.RPAREN]+++def _expr16(rng: random.Random) -> list[int]:+ """v16: signed mul / div, mod and gcd."""+ a = _digits(rng, rng.randint(1, 7), False)+ b = _digits(rng, rng.randint(1, 5), False)+ r = rng.random()+ lead = rng.choice(LEADS)+ if r < 0.4:+ op = rng.choice([[B.TIMES], [B.DIV], [B.DIV2], [B.RPDIV], [B.BSL, B.FN_DIV, B.SPACE]])+ if rng.random() < 0.5:+ a = _signed(rng, a)+ if rng.random() < 0.5:+ b = _signed(rng, b)+ return lead + a + op + [B.SPACE] * (rng.random() < 0.5) + b + [B.EQ]+ if r < 0.7:+ op = rng.choice([[B.MODOP], [B.BSL, B.FN_MOD, B.SPACE],+ [B.BSL, B.HEX[11], B.FN_MOD, B.SPACE]])+ return lead + a + op + [B.SPACE] + b + [B.EQ]+ pre = rng.choice([[B.FN_GCD], [B.BSL, B.FN_GCD]])+ if rng.random() < 0.3:+ b = [B.MINUS2] + b+ return lead + pre + [B.LPAREN] + a + [B.COMMA, B.SPACE] + b + [B.RPAREN, B.EQ]+++def _expr18(rng: random.Random) -> list[int]:+ """v18: a whole expression, valid or not (the evaluator prints "..." for a broken one)."""+ out = list(rng.choice(LEADS))+ for k in range(rng.randint(2, 4)):+ if k:+ out.append(B.CLASS_OF[rng.choice(EXPR_TOKENS)])+ if rng.random() < 0.3:+ out.append(B.SPACE)+ out += _digits(rng, rng.randint(1, 4), rng.random() < 0.1)+ if rng.random() < 0.15:+ out += [B.PERIOD] + _digits(rng, rng.randint(1, 2), False)+ return out + [B.EQ]+++def _tool(rng: random.Random, bm) -> None:+ """A tool call: the four single ops, mod and gcd (signed where the computer takes signs), or+ calc(expression)."""+ if rng.random() < 0.2:+ bm.calc_call([t for t in _expr18(rng)[:-1] if B.is_text(t) and t in B.TAPE_CODE])+ return+ op = rng.choice(list(FORMS) + ["mod", "gcd"])+ a, b = _number(rng, rng.randint(1, 6)), _number(rng, rng.randint(1, 4))+ if op in B.SIGNED and rng.random() < 0.4:+ a = "-" + a+ bm.tool_call(op, a, b)+++def corpus(seed: int = 7, docs: int = 24) -> list[list[int]]:+ """Token sequences (chip tokens and text classes) of mixed sessions."""+ from chipformer.q35.lib import bcd++ rng = random.Random(seed)+ text = [t for t in range(B.TEXT_BASE, B.TEXT_BASE + B.N_TEXT) if t != B.TC_CLOSE]+ common = [B.HEX[d] for d in range(10)] * 3 + [B.SPACE] * 6 + [B.OTHER] * 6 + text+ out = []+ for _ in range(docs):+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(rng.randint(1, 4))+ for _ in range(rng.randint(3, 8)):+ r = rng.random()+ if r < 0.45:+ form = rng.choice([_expr, _expr, _expr16, _expr18])+ bm.qwen(form(rng) + [rng.choice(common) for _ in range(rng.randint(1, 6))])+ elif r < 0.7:+ bm.qwen([rng.choice(common) for _ in range(rng.randint(5, 25))])+ else:+ _tool(rng, bm)+ while not bm.halted:+ bm.step()+ bm.text(rng.randint(1, 4))+ out.append(bm.tokens)+ return out+++# --- measuring and planning --------------------------------------------------------------------+++def _reads(e: Expr) -> set[int]:+ return {d for d in e.terms if d != ONE}+++def _text_mask(tokens: list[int]) -> np.ndarray:+ return np.array([B.is_text(t) for t in tokens])+++@dataclass+class Activity:+ """What is nonzero at text positions, over a corpus."""++ zero: np.ndarray # dim -> 0 at every text position+ terms: dict = field(default_factory=dict) # (node, term) -> coef * a * relu(b) != 0 somewhere+ gate_max: dict = field(default_factory=dict) # (node, term) -> max b+ keys: dict = field(default_factory=dict) # node -> a key written at some text position+ key_max: dict = field(default_factory=dict) # (node, channel) -> max conv output+ cols: dict = field(default_factory=dict) # (node, column) -> output nonzero somewhere+ z_max: dict = field(default_factory=dict) # (node, column) -> max z+++def activity(g: Graph, traces: list[list[int]]) -> Activity:+ act = Activity(np.ones(len(g.dims), bool))++ def upd(d, k, v, f=max):+ d[k] = f(d[k], v) if k in d else v++ for toks in traces:+ txt = _text_mask(toks)+ m = ir.evaluate(g, toks, check=False)+ act.zero &= ~(m[txt] != 0).any(0)+ for i, n in enumerate(g.nodes):+ if isinstance(n, FFN):+ for j, (coef, a, b) in enumerate(n.terms):+ bv = ev(b, m)[txt]+ v = float(coef) * ev(a, m)[txt] * np.maximum(bv, 0)+ upd(act.terms, (i, j), bool((v != 0).any()), lambda x, y: x or y)+ upd(act.gate_max, (i, j), float(bv.max(initial=-np.inf)))+ continue+ wrote = False+ for ch_i, ch in n.k.items():+ x = conv(ch, m, False)[txt]+ wrote |= bool((x > 0).any())+ upd(act.key_max, (i, ch_i), float(ev(ch.expr, m)[txt].max(initial=-np.inf)))+ upd(act.keys, i, wrote, lambda x, y: x or y)+ for x_, vh in enumerate(n.vheads()):+ for col, d, _ in vh.out:+ # (a dim several heads add into is judged by its total)+ upd(act.cols, (i, x_, col), bool((m[txt, d] != 0).any()), lambda x, y: x or y)+ if col in vh.z:+ upd(act.z_max, (i, x_, col),+ float(ev(vh.z[col], m)[txt].max(initial=-np.inf)))+ return act+++def _vh_reads(vh) -> set[int]:+ """What a value head's write gate and decay read: they act at text positions too."""+ return _reads(vh.a) | (set() if vh.beta is None else _reads(vh.beta))+++def needed(g: Graph, act: Activity, keep: set[int]) -> set[int]:+ """Dims whose value at text positions matters: the outputs the lm_head reads, what heads+ write into their state there (decays always; keys and values of heads that write keys on+ text), what conv lag taps carry to the next token, and everything those depend on."""+ need = set(keep) | _reads(g.is_text) | _reads(g.mu)+ for i, n in enumerate(g.nodes):+ if isinstance(n, Head):+ vhs = n.vheads()+ vchans = [ch for vh in vhs for ch in vh.v.values()]+ need |= set().union(*(_vh_reads(vh) for vh in vhs))+ chans = list(n.q.values()) + list(n.k.values()) + vchans+ need |= {d for ch in chans if any(ch.taps[1:]) for d in _reads(ch.expr)}+ if act.keys[i]:+ need |= {d for ch in list(n.k.values()) + vchans for d in _reads(ch.expr)}+ for n in sorted(g.nodes, key=lambda n: -n.level): # reads come from lower levels+ if isinstance(n, FFN):+ if n.out in need:+ need |= {d for _, a, b in n.terms for d in _reads(a) | _reads(b)}+ continue+ for vh in n.vheads():+ for col, d, _ in vh.out:+ if d in need:+ need |= {x for ch in n.q.values() for x in _reads(ch.expr)}+ need |= _reads(vh.z.get(col, Expr({})))+ return need+++def plan(g: Graph, act: Activity, keep: set[int]) -> Plan:+ """The overlay dims and the switch-offs they need.++ A unit or head column may be switched off on text tokens when its output does not matter+ there (``needed``) or is 0 there anyway (``activity``). A dim is overlaid when it is not+ needed on text or is 0 there, every unit and column writing it or reading it may be switched+ off, and no decay, conv lag tap or key of a head writing keys on text reads it.+ """+ need = needed(g, act, keep)+ fixed = set(keep) | _reads(g.is_text) | _reads(g.mu)+ ok = {d for d in range(1, len(g.dims))+ if d not in fixed and (d not in need or act.zero[d])}+ killable = {} # (node, term) / (node, ("col", c)) -> may be switched off+ for i, n in enumerate(g.nodes):+ if isinstance(n, FFN):+ for j in range(len(n.terms)):+ killable[i, j] = n.out not in need or not act.terms[i, j]+ else:+ for x_, vh in enumerate(n.vheads()):+ for col, d, _ in vh.out:+ killable[i, ("col", x_, col)] = d not in need or not act.cols[i, x_, col]+ for i, n in enumerate(g.nodes):+ if isinstance(n, FFN):+ for j, (_, a, b) in enumerate(n.terms):+ if not killable[i, j]:+ ok -= _reads(a) | _reads(b) | {n.out}+ continue+ vhs = n.vheads()+ vchans = [ch for vh in vhs for ch in vh.v.values()]+ for vh in vhs:+ ok -= _vh_reads(vh)+ for ch in list(n.q.values()) + list(n.k.values()) + vchans:+ if any(ch.taps[1:]):+ ok -= _reads(ch.expr)+ if act.keys[i]:+ ok -= {d for ch in list(n.k.values()) + vchans for d in _reads(ch.expr)}+ cols = [(x_, col, d) for x_, vh in enumerate(vhs) for col, d, _ in vh.out]+ if not all(killable[i, ("col", x_, col)] for x_, col, _ in cols):+ ok -= {d for ch in n.q.values() for d in _reads(ch.expr)}+ for x_, col, d in cols:+ if not killable[i, ("col", x_, col)]:+ ok -= _reads(vhs[x_].z.get(col, Expr({}))) | {d}+ p = Plan(ok)+ bounds = [np.inf]++ def bound(top, s, margin):+ if s:+ bounds.append((KILL - max(top, 0) - margin) / s)++ def noise(e):+ return sum(abs(float(c)) for d, c in e.terms.items() if d in ok)++ for i, n in enumerate(g.nodes):+ if isinstance(n, FFN):+ for j, (_, a, b) in enumerate(n.terms):+ if n.out in ok or (_reads(a) | _reads(b)) & ok:+ p.terms.add((i, j))+ bound(act.gate_max[i, j], noise(b), 104 / K)+ continue+ if not act.keys[i]:+ for ch_i, ch in n.k.items():+ if _reads(ch.expr) & ok:+ assert not any(ch.taps[1:]) and ch.taps[0] > 0+ p.keys.add((i, ch_i))+ bound(act.key_max[i, ch_i], noise(ch.expr), 104 / ch.taps[0])+ noisy = any(_reads(ch.expr) & ok for ch in n.q.values())+ for x_, vh in enumerate(n.vheads()):+ for col, d, _ in vh.out:+ if col in vh.z and (noisy or d in ok or _reads(vh.z[col]) & ok):+ p.gates.add((i, x_, col))+ bound(act.z_max.get((i, x_, col), 0), noise(vh.z[col]), 104)+ p.q_bound = float(min(bounds))+ return p+++def _vz(n, x_: int) -> dict:+ """The output gates of value head x_ of head n (the first value head's are n.z)."""+ return n.z if x_ == 0 else n.vx[x_ - 1].z+++def apply(g: Graph, p: Plan) -> None:+ """Write the switch-offs into the graph (in place); overlaid token features read 0 on text+ (the text rows carry Qwen's embedding there)."""+ off = KILL * g.is_text+ for i, j in p.terms:+ n = g.nodes[i]+ coef, a, b = n.terms[j]+ n.terms[j] = (coef, a, b - off)+ for i, ch_i in p.keys:+ ch = g.nodes[i].k[ch_i]+ g.nodes[i].k[ch_i] = Chan(ch.expr - off, ch.taps)+ for i, x_, col in p.gates:+ z = _vz(g.nodes[i], x_)+ z[col] = z[col] - off+ for d in p.dims & set(g.token_fns):+ g.token_fns[d] = (lambda f: lambda t: 0 if B.is_text(t) else f(t))(g.token_fns[d])+++def build(g: Graph, keep: set[int], traces: list[list[int]] | None = None) -> Plan:+ """Measure, plan and apply the overlay of ``g`` (in place)."""+ p = plan(g, activity(g, corpus() if traces is None else traces), keep)+ apply(g, p)+ return p+++# --- checking -----------------------------------------------------------------------------------+++def evaluate_noisy(g: Graph, p: Plan, tokens: list[int], noise: dict[int, np.ndarray]) -> np.ndarray:+ """``ir.evaluate`` with the overlay dims holding ``noise`` (Qwen's embedding values) at text+ positions. Raises unless every write into them there is exactly 0 and every switched-off+ gate, key and output gate is dead (silu exactly 0) despite the noise."""+ n = len(tokens)+ txt = _text_mask(tokens)+ toks = np.asarray(tokens, dtype=np.int64)+ m = np.zeros((n, len(g.dims)))+ chip = toks >= 0+ m[:, ONE] = chip+ uniq, inv = np.unique(toks, return_inverse=True)+ for d, fn in g.token_fns.items():+ m[:, d] = np.array([float(fn(int(u))) for u in uniq])[inv]+ over = sorted(d for d in p.dims if d in noise)+ for d in over:+ ir.require(m[txt, d] == 0, f"overlay token feature {g.dims[d].name} on text", m[txt, d])+ m[txt, d] = noise[d][txt]+ for i, node in enumerate(g.nodes):+ if isinstance(node, FFN):+ for j, (_, _, b) in enumerate(node.terms):+ if (i, j) in p.terms:+ gate = K * ev(b, m)[txt]+ ir.require(ir.dead(gate), f"{node.name}: unit not dead", gate)+ vals = np.zeros(n)+ for coef, a, b in node.terms:+ vals += float(coef) * ev(a, m) * np.maximum(ev(b, m), 0.0)+ m[:, node.out] = (m[:, node.out] + vals) if node.into else vals+ if node.out in noise:+ ir.require(vals[txt] == 0, f"{node.name}: write into an overlay dim on text", vals)+ m[txt, node.out] = noise[node.out][txt]+ continue+ for ch_i, ch in node.k.items():+ if (i, ch_i) in p.keys:+ x = ch.taps[0] * ev(ch.expr, m)[txt]+ ir.require(ir.dead(x), f"{node.name}: key channel not dead", x)+ vhs = node.vheads()+ for x_, vh in enumerate(vhs):+ for col, d, _ in vh.out:+ if (i, x_, col) in p.gates:+ zz = ev(vh.z[col], m)[txt]+ ir.require(ir.dead(zz), f"{node.name}: output gate not dead", zz)+ outs = {d for vh in vhs for _, d, _ in vh.out}+ # an overlaid dim several heads add into holds the noise at text positions: take it out+ # before the head adds (its share there must be 0) and put it back after+ keep = {d: m[txt, d].copy() for d in outs if d in noise}+ for d in keep:+ if any(vh.into for vh in vhs):+ m[txt, d] = 0+ ir.eval_head(node, m, chip, False)+ for d in outs:+ if d in noise:+ ir.require(m[txt, d] == 0, f"{node.name}: write into an overlay dim on text",+ m[txt, d])+ m[txt, d] = keep[d]+ return m+++def check_noisy(g: Graph, p: Plan, tokens: list[int], rng: np.random.Generator,+ scale: float = 1.0) -> None:+ """The overlay leaves the circuit exact: with random Qwen values in the overlay dims at text+ positions, every dim everywhere else equals the clean evaluation."""+ clean = ir.evaluate(g, tokens)+ txt = _text_mask(tokens)+ noise = {d: ir.bf16(rng.normal(0, scale, len(tokens))) for d in p.dims}+ m = evaluate_noisy(g, p, tokens, noise)+ mask = np.ones(m.shape, bool)+ mask[np.ix_(txt, sorted(p.dims))] = False+ bad = np.nonzero(mask & (m != clean))+ if len(bad[0]):+ t, d = int(bad[0][0]), int(bad[1][0])+ raise ir.ExactnessError(f"overlay changes {g.dims[d].name} at position {t}: "+ f"{m[t, d]!r} vs {clean[t, d]!r}")diff --git a/chipformer/q35/prims.py b/chipformer/q35/prims.pynew file mode 100644index 0000000..a9df485--- /dev/null+++ b/chipformer/q35/prims.py@@ -0,0 +1,225 @@+"""Exact gadgets for the q35 IR.++MLP gadgets are lists of unit terms ``(coef, a, b)`` (``coef * a * relu(b)``) so several can share+one node. They keep every positive gate value a bf16 integer (|b| <= 256), which makes silu an+exact relu; the negative side may be any size, since silu(<= -104) is exactly 0.++Values are represented as+ bits list of 0/1 Exprs, LSB first (an integer is their weighted sum, free)+ one-hot list of 0/1 Exprs, exactly one set (bits of the index are linear, free)+ int an Expr holding a small integer (|value| <= 256 inside a gate)+"""++from fractions import Fraction++from chipformer.dsl.graph import Expr+from chipformer.q35.ir import A_BIT, A_REF, REF_COLS, Chan, Graph, K++Terms = list[tuple]+++def cat(term_lists) -> Terms:+ """Concatenate unit-term lists."""+ return [t for ts in term_lists for t in ts]+++def from_bits(bits: list) -> Expr:+ return sum((b * (1 << i) for i, b in enumerate(bits)), Expr())+++def onehot_bits(oh: list, n: int) -> list:+ """Bits of the index of a one-hot."""+ return [sum((e for v, e in enumerate(oh) if v >> i & 1), Expr()) for i in range(n)]+++def onehot_int(oh: list) -> Expr:+ return sum((v * e for v, e in enumerate(oh)), Expr())+++class Circ(Graph):+ # --- unit terms -----------------------------------------------------------++ def t_const(self, c) -> Terms:+ return [(c, self.one, self.one)]++ def t_and(self, lits, coef=1, a=None) -> Terms:+ """coef * a * AND(lits) for 0/1 literals.++ The gate is 4 * (sum - n + 1/2): +2 when all are set, <= -2 otherwise, so a false AND+ is exactly 0 even when the literals carry fp32 noise (silu(-256) == -0).+ """+ a = self.one if a is None else a+ return [(Fraction(coef) / 2, a, 4 * sum(lits, Expr()) - (4 * len(lits) - 2))]++ def t_ge(self, z, theta, lo, hi, coef=1, lits=()) -> Terms:+ """coef * [z >= theta] (AND lits) for an integer z in [lo, hi]."""+ if theta > hi:+ return []+ n = len(lits)+ if theta <= lo and not n:+ return self.t_const(coef)+ if hi - theta + 1 <= 256:+ m = 1 << (hi - lo).bit_length() # > hi - lo, and a bf16 weight+ zz = z + m * (sum(lits, Expr()) - n) if n else z+ return [(coef, self.one, zz - theta + 1), (-coef, self.one, zz - theta)]+ assert theta - lo <= 256, (theta, lo, hi)+ base = self.t_and(lits, coef) if n else self.t_const(coef)+ return base + self.t_lt(z, theta, lo, hi, -coef, lits)++ def t_lt(self, z, theta, lo, hi, coef=1, lits=()) -> Terms:+ """coef * [z < theta] (AND lits) for an integer z in [lo, hi]."""+ if theta <= lo:+ return []+ n = len(lits)+ if theta > hi and not n:+ return self.t_const(coef)+ if theta - lo <= 256:+ m = 1 << (hi - lo).bit_length()+ zz = z - m * (sum(lits, Expr()) - n) if n else z+ return [(coef, self.one, theta - zz), (-coef, self.one, theta - zz - 1)]+ assert hi - theta + 1 <= 256, (theta, lo, hi)+ base = self.t_and(lits, coef) if n else self.t_const(coef)+ return base + self.t_ge(z, theta, lo, hi, -coef, lits)++ def t_in(self, z, a, b, lo, hi, coef=1, lits=()) -> Terms:+ """coef * [a <= z < b] (AND lits)."""+ a, b = max(a, lo), min(b, hi + 1)+ if a >= b:+ return []+ if a == lo:+ return self.t_lt(z, b, lo, hi, coef, lits)+ if b == hi + 1:+ return self.t_ge(z, a, lo, hi, coef, lits)+ if lits:+ return self.t_ge(z, a, lo, hi, coef, lits) + self.t_ge(z, b, lo, hi, -coef, lits)+ lt_cost = (b - lo, a - lo)+ if max(lt_cost) <= 256:+ return self.t_lt(z, b, lo, hi, coef) + self.t_lt(z, a, lo, hi, -coef)+ return self.t_ge(z, a, lo, hi, coef) + self.t_ge(z, b, lo, hi, -coef)++ def t_eq(self, z, v, lo, hi, coef=1, lits=()) -> Terms:+ return self.t_in(z, v, v + 1, lo, hi, coef, lits)++ # --- nodes ----------------------------------------------------------------++ def node(self, terms: Terms, name: str) -> Expr:+ if not terms:+ return Expr()+ return self.ffn(terms, name)++ def and_(self, lits, name: str) -> Expr:+ return self.node(self.t_and(lits), name)++ def onehot_of_bits(self, bits: list, name: str, n: int | None = None) -> list:+ """One-hot of the integer with these bits (values 0..2^len-1, or 0..n-1)."""+ size = n or 1 << len(bits)+ out = []+ for v in range(size):+ lits = [b if v >> i & 1 else self.one - b for i, b in enumerate(bits)]+ out.append(self.and_(lits, f"{name}=={v}"))+ return out++ def nib_onehot(self, z, lo, hi, name: str, div: int = 1) -> list:+ """One-hot of (z // div) mod 16 for an integer z in [lo, hi] (lo >= 0)."""+ out = []+ for x in range(16):+ terms = []+ k0 = lo // (16 * div)+ for k in range(k0, hi // (16 * div) + 1):+ a = div * (16 * k + x)+ terms += self.t_in(z, a, a + div, lo, hi)+ out.append(self.node(terms, f"{name}[{x}]"))+ return out++ def select_bits(self, cases: list[tuple[list, list]], n: int, name: str) -> list:+ """Bits of sum(cond * value) for one-hot conditions; values are bit lists or ints."""+ out = []+ for i in range(n):+ terms = []+ for lits, val in cases:+ if isinstance(val, int):+ if val >> i & 1:+ terms += self.t_and(lits)+ elif i < len(val):+ b = val[i]+ if isinstance(b, int):+ if b:+ terms += self.t_and(lits)+ else:+ terms += self.t_and([*lits, b])+ out.append(self.node(terms, f"{name}.b{i}"))+ return out++ # --- 12-bit arithmetic ----------------------------------------------------++ def add12(self, a_lo, a_hi_oh: list, d, d_max: int, name: str, a_lo_max: int = 255):+ """One-hot nibbles of (A + d) mod 4096, where A = a_lo + 256 * index(a_hi_oh)."""+ lo_hi = a_lo_max + d_max+ L = a_lo + d+ n0 = self.nib_onehot(L, 0, lo_hi, f"{name}.n0")+ n1 = self.nib_onehot(L, 0, lo_hi, f"{name}.n1", div=16)+ n2 = []+ for x in range(16):+ terms = (self.t_lt(L, 256, 0, lo_hi, lits=[a_hi_oh[x]])+ + self.t_ge(L, 256, 0, lo_hi, lits=[a_hi_oh[(x - 1) % 16]]))+ n2.append(self.node(terms, f"{name}.n2[{x}]"))+ return n0, n1, n2++ def add12_bits(self, a_lo, a_hi_oh: list, d, d_max: int, name: str,+ a_lo_max: int = 255) -> list:+ """The 12 bits of (A + d) mod 4096 (as add12, but each bit is one dim: a bit's node+ sums the units of the nibble values that have it, so 12 dims instead of 48)."""+ lo_hi = a_lo_max + d_max+ L = a_lo + d++ def nib_terms(x, div):+ terms = []+ for k in range(lo_hi // (16 * div) + 1):+ a = div * (16 * k + x)+ terms += self.t_in(L, a, a + div, 0, lo_hi)+ return terms++ def n2_terms(x):+ return (self.t_lt(L, 256, 0, lo_hi, lits=[a_hi_oh[x]])+ + self.t_ge(L, 256, 0, lo_hi, lits=[a_hi_oh[(x - 1) % 16]]))++ out = []+ for nib, terms_of in enumerate([lambda x: nib_terms(x, 1), lambda x: nib_terms(x, 16),+ n2_terms]):+ for j in range(4):+ terms = [t for x in range(16) if x >> j & 1 for t in terms_of(x)]+ out.append(self.node(terms, f"{name}.b{4 * nib + j}"))+ return out++ # --- heads ----------------------------------------------------------------++ def slot_head(self, name, keys: dict, queries: dict, data: dict, ref, a=0,+ outs=None, gate=None, tiny: float = 0.0) -> list:+ """A head whose rows are 0/1 data columns plus a ballast.++ keys: slot -> Chan or pre-activation; queries likewise; data: column (>= 2) -> Chan or+ Expr giving the *bit* (0/1) to store; ref: Chan or Expr, 1 where the row written carries+ the ballast. Returns one dim per data column, holding the read bit.+ """+ def chan(c, scale):+ if isinstance(c, Chan):+ return Chan(c.expr * scale, c.taps)+ return Chan(c * scale)++ def sel(c):+ """Keys and queries: K * (2c - 1), so c = 1 gives K and c <= 0 gives <= -K. A+ false entry is then exactly 0 after silu even with fp32 noise in c, which matters:+ a leak into another row would be rescaled to full size by the output norm."""+ if isinstance(c, Chan):+ return Chan(c.expr * (2 * K) - self.one * K, c.taps)+ return Chan(c * (2 * K) - self.one * K)++ v = {col: chan(c, A_BIT) for col, c in data.items()}+ for col in REF_COLS:+ v[col] = chan(ref, A_REF)+ gate = self.one if gate is None else gate+ cols = sorted(data) if outs is None else outs+ z = {col: K * gate for col in cols}+ return self.head(name, {s: sel(c) for s, c in queries.items()},+ {s: sel(c) for s, c in keys.items()}, v, a, z,+ [(col, 2.0**7) for col in cols], tiny=tiny)diff --git a/chipformer/q35/record.py b/chipformer/q35/record.pynew file mode 100644index 0000000..3127a35--- /dev/null+++ b/chipformer/q35/record.py@@ -0,0 +1,226 @@+"""Record a game played by a q35 checkpoint and render it to video at real-time speed.++ python -m chipformer.q35.record play MODEL tetris --seconds 60 --out run.npz (GPU box)+ python -m chipformer.q35.record render run.npz run.mp4 (needs ffmpeg)++``play`` runs the stock model with greedy decoding through ``fast.Runner``; the host only injects+TICKs every ``ipf`` instructions and key writes chosen by a simple bot from the screen. The trace+is verified record by record before it is saved. ``render`` replays the model's own writes and+shows one frame per TICK, i.e. at the speed the game would run on a real CHIP-8.+"""++import argparse+import json+import subprocess+import time+from pathlib import Path++import numpy as np++from chipformer import spec+from chipformer.ref.emu import apply_writes, display_pixels++ROOT = Path(__file__).resolve().parents[2]+++class TetrisBot:+ """Port of the browser demo's recorder bot: probe rotations, place by a heuristic score."""++ W0, COLS, ROWS, TOP = 27, 10, 31, 6++ def __init__(self, keys):+ self.k = keys+ self.was_top = False+ self.state, self.wait = "idle", 0+ self.stack = [[0] * 10 for _ in range(31)]++ def grid(self, px):+ return [[px[y][self.W0 + x] for x in range(self.COLS)] for y in range(self.ROWS)]++ @staticmethod+ def evaluate(st, cells, x):+ w = max(c[1] for c in cells) + 1+ h = max(c[0] for c in cells) + 1+ if x + w > 10:+ return None+ y = -h++ def fits(yy):+ return all(yy + dy < 31 and (yy + dy < 0 or not st[yy + dy][x + dx])+ for dy, dx in cells)++ while fits(y + 1):+ y += 1+ if any(y + dy < 0 for dy, _ in cells):+ return None+ g = [r[:] for r in st]+ for dy, dx in cells:+ g[y + dy][x + dx] = 1+ lines = 0+ r = 30+ while r >= 0:+ if all(g[r]):+ del g[r]+ g.insert(0, [0] * 10)+ lines += 1+ else:+ r -= 1+ hs = [next((31 - r for r in range(31) if g[r][c]), 0) for c in range(10)]+ holes = 0+ for c in range(10):+ seen = False+ for r in range(31):+ if g[r][c]:+ seen = True+ elif seen:+ holes += 1+ bump = sum(abs(hs[i] - hs[i + 1]) for i in range(9))+ return -0.51 * sum(hs) + 0.76 * lines - 0.72 * holes - 0.18 * bump++ def frame(self, px) -> set:+ """Keys to hold this frame (closed loop on the screen, like the browser bot)."""+ g = self.grid(px)+ top = [(y, x) for y in range(self.TOP) for x in range(10) if g[y][x]]+ spawned = bool(top) and not self.was_top+ self.was_top = bool(top)+ if spawned:+ self.stack = [r[:] if y >= self.TOP else [0] * 10 for y, r in enumerate(g)]+ self.state, self.wait = "probe", 0+ self.shapes = []+ if self.state == "idle":+ return set()+ piece = [(y, x) for y in range(31) for x in range(10)+ if g[y][x] and not self.stack[y][x] and y < 16]+ if self.wait > 0:+ self.wait -= 1+ return set()+ if self.state == "probe": # record each rotation's shape, then pick the best placement+ if len(piece) == 4:+ y0, x0 = min(c[0] for c in piece), min(c[1] for c in piece)+ self.shapes.append(sorted((y - y0, x - x0) for y, x in piece))+ if len(self.shapes) < 4:+ self.wait = 3+ return {self.k["a"]}+ best = None+ for r, cs in enumerate(self.shapes):+ for x in range(10):+ v = self.evaluate(self.stack, cs, x)+ if v is not None and (best is None or v > best[0]):+ best = (v, r, x)+ if best is None:+ self.state = "drop"+ return set()+ _, self.rot, self.target = best+ self.state = "rotate"+ return set()+ if self.state == "rotate":+ if self.rot > 0:+ self.rot -= 1+ self.wait = 3+ return {self.k["a"]}+ self.state = "move"+ if self.state == "move":+ if len(piece) != 4:+ return set()+ x = min(c[1] for c in piece)+ if x < self.target:+ return {self.k["right"]}+ if x > self.target:+ return {self.k["left"]}+ self.state = "drop"+ if self.state == "drop":+ return {self.k["down"]}+ return set()+++def play(model_dir: str, game: str, seconds: float, out: str, max_len: int = 65536) -> None:+ import torch+ from transformers import AutoModelForCausalLM, AutoModelForImageTextToText++ from chipformer.check import verify_trace+ from chipformer.q35.fast import Runner++ meta_path = Path(model_dir) / "chipformer.json"+ merged = meta_path.exists()+ cls = AutoModelForImageTextToText if merged else AutoModelForCausalLM+ model = cls.from_pretrained(model_dir, dtype=torch.bfloat16).cuda().eval()+ off = json.loads(meta_path.read_text())["chip_token_offset"] if merged else 0+ games = {g["id"]: g for g in json.loads((ROOT / "games" / "manifest.json").read_text(+ encoding="utf-8"))["games"]}+ g = games[game]+ runner = Runner(model, max_len=max_len, token_offset=off)+ s = runner.session((ROOT / "games" / g["file"]).read_bytes(), tuple(g["quirks"]))+ bot = TetrisBot(g["keys"]) if game == "tetris" else None+ held: set = set()+ frames = int(seconds * 60)+ t = time.time()+ for f in range(frames):+ want = bot.frame(s.pixels()) if bot else set()+ for k in held - want:+ s.set_key(k, False)+ for k in want - held:+ s.set_key(k, True)+ held = want+ s.tick()+ for _ in range(g["ipf"]):+ s.step()+ if f % 600 == 0:+ el = time.time() - t+ print(f"frame {f}/{frames} {len(s.tokens)} tokens {el:.0f}s", flush=True)+ n, err = verify_trace(s.tokens)+ print(f"verified {n} records, error: {err}, {time.time() - t:.0f}s")+ np.savez_compressed(out, tokens=np.array(s.tokens, dtype=np.uint32), game=game,+ model=str(model_dir), verified=err is None)+++def render(trace: str, out: str, scale: int = 12) -> None:+ from chipformer.trace import parse++ data = np.load(trace)+ tokens = data["tokens"].tolist()+ cells = [0] * spec.NUM_CELLS+ frames, prev = [], None+ for r in parse(tokens, allow_reboot=True):+ if r.kind == "boot":+ cells = [0] * spec.NUM_CELLS+ apply_writes(cells, r.writes)+ if r.kind == "tick":+ px = np.array(display_pixels(cells), dtype=np.uint8)+ shown = px if prev is None else np.maximum(px, prev) # soften CHIP-8 flicker+ frames.append(shown)+ prev = px+ h, w = 32 * scale, 64 * scale+ ink, lit = np.array([25, 34, 31], np.uint8), np.array([221, 245, 158], np.uint8)+ proc = subprocess.Popen(["ffmpeg", "-y", "-loglevel", "error", "-f", "rawvideo",+ "-pix_fmt", "rgb24", "-s", f"{w}x{h}", "-r", "60", "-i", "-",+ "-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "18", out],+ stdin=subprocess.PIPE)+ for px in frames:+ img = np.where(px[..., None].astype(bool), lit, ink)+ img = img.repeat(scale, 0).repeat(scale, 1)+ proc.stdin.write(img.tobytes())+ proc.stdin.close()+ proc.wait()+ print(f"{out}: {len(frames)} frames ({len(frames) / 60:.1f} s at 60 fps)")+++def main() -> None:+ ap = argparse.ArgumentParser()+ sub = ap.add_subparsers(dest="cmd", required=True)+ p = sub.add_parser("play")+ p.add_argument("model")+ p.add_argument("game")+ p.add_argument("--seconds", type=float, default=30)+ p.add_argument("--out", default="run.npz")+ r = sub.add_parser("render")+ r.add_argument("trace")+ r.add_argument("out")+ a = ap.parse_args()+ if a.cmd == "play":+ play(a.model, a.game, a.seconds, a.out)+ else:+ render(a.trace, a.out)+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/report.py b/chipformer/q35/report.pynew file mode 100644index 0000000..f14ee3e--- /dev/null+++ b/chipformer/q35/report.py@@ -0,0 +1,235 @@+"""Check a merged Qwen3.5 + CHIP-8 checkpoint against its base model.++ python -m chipformer.q35.report BASE MERGED [--device cuda] [--out report.json]++1. CHIP-8: teacher-forced predictions on game traces, pure and inside a chat (mixed context).+2. Leakage: the largest chip-token logit at text positions.+3. Text: greedy generations and logits vs the base model, next to the noise floor between two+ attention implementations of the base model itself.+"""++import argparse+import json+import time+from pathlib import Path++import numpy as np+import torch++from chipformer.check import run_machine++ROOT = Path(__file__).resolve().parents[2]+PROMPTS = [+ ("chat", "Explain why the sky is blue in two sentences."),+ ("raw", 'def fibonacci(n):\n """Return the n-th Fibonacci number."""\n'),+ ("chat", "What is 37 * 43? Think step by step."),+ ("chat", "Traduis en français : The weather is lovely today and I want to go for a walk."),+ ("chat", "写一首关于秋天的短诗。"),+ ("chat", "List three differences between TCP and UDP."),+ ("raw", "The CHIP-8 is a virtual machine from the 1970s that"),+ ("raw", "Q: If a train leaves at 3pm travelling 60 km/h, how far has it gone by 5:30pm?\nA:"),+ ("chat", "Write a haiku about a lighthouse."),+ ("chat", "Give me a Python one-liner that reverses the words in a sentence."),+ ("chat", "What causes the seasons on Earth?"),+ ("raw", "In 1969, Apollo 11"),+ ("chat", "Summarise the plot of Romeo and Juliet in three sentences."),+ ("chat", "Solve for x: 3x + 7 = 22."),+ ("raw", "SELECT name, COUNT(*) FROM orders"),+ ("chat", "¿Cuál es la capital de Australia y por qué no es Sídney?"),+]+++def game_trace(gid: str, steps: int, seed: int = 2):+ games = json.loads((ROOT / "games" / "manifest.json").read_text(encoding="utf-8"))["games"]+ g = next(g for g in games if g["id"] == gid)+ rom = (ROOT / "games" / g["file"]).read_bytes()+ return run_machine(rom, steps, ipf=g["ipf"], seed=seed, key_rate=0.3, quirks=tuple(g["quirks"]))+++def load(path, device, impl="sdpa"):+ from transformers import AutoModelForImageTextToText++ return AutoModelForImageTextToText.from_pretrained(+ path, dtype=torch.bfloat16, attn_implementation=impl).to(device).eval()+++def encode(tok, kind, text):+ if kind == "chat":+ return tok.apply_chat_template([{"role": "user", "content": text}],+ add_generation_prompt=True, return_tensors="pt",+ enable_thinking=False)["input_ids"][0].tolist()+ return tok(text).input_ids++[email protected]_grad()+def text_run(model, prompts, device, n_new):+ gens, logits = [], []+ for ids in prompts:+ x = torch.tensor([ids], device=device)+ g = model.generate(x, max_new_tokens=n_new, do_sample=False)+ gens.append(g[0, len(ids):].tolist())+ logits.append(model(input_ids=g).logits[0, :, :248320].float().cpu())+ return gens, logits+++def compare(a, b):+ ga, la = a+ gb, lb = b+ same = [x == y for x, y in zip(ga, gb)]+ first = [next((i for i, (p, q) in enumerate(zip(x, y)) if p != q), None)+ for x, y in zip(ga, gb)]+ dmax, kl, top1 = [], [], []+ for x, y, s in zip(la, lb, same):+ if not s:+ continue+ dmax.append(float((x - y).abs().max()))+ p, q = torch.log_softmax(x, -1), torch.log_softmax(y, -1)+ kl.append(float((p.exp() * (p - q)).sum(-1).mean()))+ top1.append(float((x.argmax(-1) == y.argmax(-1)).float().mean()))+ return {"identical_generations": f"{sum(same)}/{len(same)}", "first_divergence": first,+ "max_abs_dlogit": max(dmax, default=None), "mean_kl": float(np.mean(kl)) if kl else None,+ "top1_agreement": float(np.mean(top1)) if top1 else None}++[email protected]_grad()+def chip_check(model, off, tokens, generated, device, prefix=(), suffix=()):+ ids = list(prefix) + [t + off for t in tokens] + list(suffix)+ logits = model(input_ids=torch.tensor([ids], device=device)).logits[0].float().cpu()+ s = len(prefix)+ pred = logits[s : s + len(tokens)].argmax(-1).numpy() - off+ want = np.array(tokens[1:] + [-1])+ gen = np.array(generated[1:] + [False])+ text_pos = list(range(s)) + list(range(s + len(tokens), len(ids)))+ leak = float(logits[text_pos, off:off + 8708].max()) if text_pos else None+ return {"checked": int(gen.sum()), "mismatches": int((gen & (pred != want)).sum()),+ "min_predicted_logit": float(logits[s : s + len(tokens)].max(-1).values[gen].min()),+ "max_chip_logit_at_text": leak}++[email protected]_grad()+def post_trace(model, tok, off, device, lengths=(0, 2000, 10000)):+ """Does Qwen remember the conversation after a CHIP-8 run in the middle of it?++ Compares the merged model on [prompt, trace, question] with itself on [prompt, question]+ where the question keeps the positions it has after the trace, so only the trace's content+ differs. Reports KL on the answer's next-token distributions and both greedy answers.+ """+ pre = tok.apply_chat_template(+ [{"role": "user", "content": "Remember this: my locker code is 4817 and my cat is called "+ "Biscuit. I'll ask about it later. First, run this CHIP-8 program."},+ {"role": "assistant", "content": "Sure, running it now."}],+ return_tensors="pt", enable_thinking=False)["input_ids"][0].tolist()+ q = tok.apply_chat_template(+ [{"role": "user", "content": "What is my locker code and what is my cat called?"}],+ add_generation_prompt=True, return_tensors="pt", enable_thinking=False)["input_ids"][0]+ q = q.tolist()+ out = {}+ m = game_trace("tetris", 4000, seed=7)+ for n in lengths:+ trace = [t + off for t in m.tokens[:n]]+ if trace and m.tokens[n - 1] != 8705: # end on a STEP+ last = max(i for i in range(n) if m.tokens[i] == 8705)+ trace = trace[: last + 1]+ full = torch.tensor([pre + trace + q], device=device)+ gap = torch.tensor([pre + q], device=device)+ pos = torch.cat([torch.arange(len(pre)), torch.arange(len(q)) + len(pre) + len(trace)])+ a = model.generate(full, max_new_tokens=24, do_sample=False)[0, full.shape[1]:]+ b = model.generate(gap, position_ids=pos[None].to(device), max_new_tokens=24,+ do_sample=False)[0, gap.shape[1]:] if trace else a+ la = model(input_ids=torch.cat([full, a[None]], 1)).logits[0, -len(a) - 1:-1].float()+ pos_b = torch.cat([pos, torch.arange(len(a)) + pos[-1] + 1])[None].to(device)+ lb = model(input_ids=torch.cat([gap, a[None]], 1), position_ids=pos_b).logits[+ 0, -len(a) - 1:-1].float()+ p, r = torch.log_softmax(la[:, :248320], -1), torch.log_softmax(lb[:, :248320], -1)+ out[n] = {"trace_tokens": len(trace), "kl": float((p.exp() * (p - r)).sum(-1).mean()),+ "answer_with_trace": tok.decode(a, skip_special_tokens=True),+ "answer_without": tok.decode(b, skip_special_tokens=True)}+ return out+++PRINT_ROM = bytes([0xA2, 0x0A, 0x60, 4, 0x00, 0xFD, 0x12, 0x06, 0, 0, 4, 8, 1, 7]) # prints 4817++[email protected]_grad()+def bridge_demo(model, tok, meta, device, rom: bytes = PRINT_ROM, n_new: int = 120):+ """One generate(): a chat turn hands over to a CHIP-8 program, which runs, prints its result+ as text, halts, and Qwen continues. Checks the machine part against the reference."""+ from chipformer.q35 import bridge+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ off = meta["chip_token_offset"]+ text_ids = {int(k): v for k, v in meta["text_ids"].items()}+ to_id = {**text_ids}+ pre = tok.apply_chat_template(+ [{"role": "user", "content": "Run the program and tell me the number it prints, "+ "then that number plus one."}],+ add_generation_prompt=True, return_tensors="pt", enable_thinking=False)["input_ids"][0]+ pre = pre.tolist() + tok("Running it now.\n", add_special_tokens=False).input_ids+ boot = boot_tokens(Chip8(rom).cells)+ ids = pre + [t + off for t in boot]+ out = model.generate(torch.tensor([ids], device=device), max_new_tokens=n_new,+ do_sample=False)[0, len(ids):].tolist()+ # reference: the machine records and the print, as model token ids+ bm = bridge.BridgeMachine(rom)+ while not bm.halted:+ bm.step()+ want = [to_id[t] if bridge.is_text(t) else t + off for t in bm.tokens[len(boot):]]+ got_machine = out[:len(want)]+ return {"machine_and_print_exact": got_machine == want,+ "printed": tok.decode(out[:len(want)][-(len(want) - want.index(text_ids[bridge.NL])):]),+ "continuation": tok.decode(out[len(want):], skip_special_tokens=True),+ "n_machine_tokens": len(want)}+++def main() -> None:+ from transformers import AutoTokenizer++ ap = argparse.ArgumentParser()+ ap.add_argument("base")+ ap.add_argument("merged")+ ap.add_argument("--device", default="cuda" if torch.cuda.is_available() else "cpu")+ ap.add_argument("--new", type=int, default=64)+ ap.add_argument("--out", default=None)+ args = ap.parse_args()+ meta = json.loads((Path(args.merged) / "chipformer.json").read_text())+ off = meta["chip_token_offset"]+ tok = AutoTokenizer.from_pretrained(args.merged)+ prompts = [encode(tok, k, t) for k, t in PROMPTS]+ rep = {"meta": meta}++ t = time.time()+ model = load(args.merged, args.device)+ rep["params_B"] = sum(p.numel() for p in model.parameters()) / 1e9+ for gid in ("brix", "tetris", "space-invaders"):+ try:+ m = game_trace(gid, 400)+ except StopIteration:+ continue+ rep[f"chip_{gid}"] = chip_check(model, off, m.tokens, m.generated, args.device)+ m = game_trace("tetris", 200, seed=5)+ pre = encode(tok, "chat", "Run this CHIP-8 program for me.")+ post = tok("\nThe machine is running Tetris now. What should I do next?",+ add_special_tokens=False).input_ids+ rep["chip_mixed"] = chip_check(model, off, m.tokens, m.generated, args.device, pre, post)+ rep["post_trace"] = post_trace(model, tok, off, args.device)+ if meta.get("layout") == "v2":+ rep["bridge"] = bridge_demo(model, tok, meta, args.device)+ merged = text_run(model, prompts, args.device, args.new)+ rep["sample_merged"] = tok.decode(merged[0][2])+ del model+ base = load(args.base, args.device)+ ref = text_run(base, prompts, args.device, args.new)+ rep["sample_base"] = tok.decode(ref[0][2])+ del base+ eager = text_run(load(args.base, args.device, "eager"), prompts, args.device, args.new)+ rep["noise_floor_sdpa_vs_eager"] = compare(ref, eager)+ rep["base_vs_merged"] = compare(ref, merged)+ rep["seconds"] = time.time() - t+ print(json.dumps(rep, indent=2, ensure_ascii=False))+ if args.out:+ Path(args.out).write_text(json.dumps(rep, indent=2, ensure_ascii=False), encoding="utf-8")+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/spike.py b/chipformer/q35/spike.pynew file mode 100644index 0000000..91590a3--- /dev/null+++ b/chipformer/q35/spike.py@@ -0,0 +1,208 @@+"""Phase 0 spike: check that the real Qwen3.5 modules can hold exact machine state.++One Qwen3.5 decoder layer (Gated DeltaNet + MLP) with hand-set weights, fed `inputs_embeds`+directly. The residual stream layout below stands in for the chip dims of the merged model.++Every chip token carries a ballast of 2^40 in two dims. That pins the RMSNorm scale to exactly+2^-36, so a residual value x reaches the projections as x * 2^-36, and the weights carry the+2^36 back. Text tokens carry no ballast and leave the chip state untouched.++Heads (all 16 key heads, one value head each):+ 0..13 RAM: write the token's bits to (whead, wslot), read (rhead, rslot)+ 14 RAM whose state is cleared on RST tokens (display CLS)+ 15 counter: x = 2^-w, w = chip tokens since the last CNTW token+"""++import torch+from transformers import Qwen3_5TextConfig, Qwen3_5TextModel++D = 512+B0, B1 = 0, 1 # ballast+ISW, RST, CNTW = 2, 3, 4 # write flag, display clear, counter reset+WSLOT, WHEAD, BITS = 8, 136, 152+RSLOT, RHEAD = 160, 288+OUT_BITS, OUT_FOUND, OUT_CNT = 304, 312, 313+S_DIM, C_DIM, AND_DIM, NM = 320, 356, 392, 36 # counter decode stages, AND gadget+GARBAGE = 400 # dims 400.. stand in for Qwen's own dims++BALLAST = 2.0**40+FEAT = 2.0**36 # residual value 1 -> normed 2^-36; weights carry this factor back+ONE = 16.0 # the normed ballast+K = 128.0 # silu(K n) == K n for integers n >= 1, and -0 for n <= -1+A_REF = 2.0**21 # ballast columns of a value row+NH, HD = 16, 128+RAM_HEADS, CLS_HEAD, CNT_HEAD = range(14), 14, 15+++def config():+ return Qwen3_5TextConfig(+ vocab_size=16,+ hidden_size=D,+ intermediate_size=128,+ # HF's cache needs one attention layer to count positions; it has zero weights here+ num_hidden_layers=2,+ layer_types=["linear_attention", "full_attention"],+ num_attention_heads=2,+ num_key_value_heads=1,+ head_dim=256,+ linear_num_key_heads=NH,+ linear_num_value_heads=NH,+ linear_key_head_dim=HD,+ linear_value_head_dim=HD,+ tie_word_embeddings=False,+ )+++def build(dtype):+ model = Qwen3_5TextModel(config()).eval()+ for p in model.parameters():+ p.data.zero_()+ layer = model.layers[0]+ la = layer.linear_attn+ qkv = la.in_proj_qkv.weight.data # (3*NH*HD, D)+ z, b, a = la.in_proj_z.weight.data, la.in_proj_b.weight.data, la.in_proj_a.weight.data+ out = la.out_proj.weight.data # (D, NH*HD)+ q_off, k_off, v_off = 0, NH * HD, 2 * NH * HD++ def row(w, terms): # terms: {dim: coef}; the ballast dim carries constants+ for d, c in terms.items():+ w[d] += c / ONE if d == B0 else c * FEAT++ for h in range(NH):+ b[h, B0] = 128.0 / ONE # beta = sigmoid(128) = 1+ z[h * HD : (h + 1) * HD, B0] = K / ONE # silu(z) = 128 on chip tokens, 0 on text+ la.dt_bias.data.fill_(-110.0) # softplus(a - 110) = 0 at a = 0: decay exactly 1+ la.A_log.data.zero_()+ la.norm.weight.data.fill_(1.0)+ la.conv1d.weight.data[:, 0, -1] = 1.0 # current-token tap only++ for h in [*RAM_HEADS, CLS_HEAD]:+ for s in range(HD):+ row(qkv[q_off + h * HD + s], {RSLOT + s: K, RHEAD + h: K, B0: -K})+ row(qkv[k_off + h * HD + s], {WSLOT + s: K, WHEAD + h: K, ISW: K, B0: -2 * K})+ v = v_off + h * HD+ row(qkv[v + 0], {B0: A_REF})+ row(qkv[v + 1], {B0: A_REF})+ for i in range(8):+ row(qkv[v + 2 + i], {BITS + i: K})+ # normed bit column = 2^-11 (bf16 snapped); times silu(z) = 128 -> 2^-4+ out[OUT_BITS + i, h * HD + 2 + i] = 16.0+ out[OUT_FOUND, h * HD] = 2.0**-10 # normed ballast column 8, times 128+ row(a[CLS_HEAD], {RST: 238.0}) # softplus(128) = 128: decay exp(-128) = 0++ # counter: row 0 = count (written at CNTW), row 1 = ref (refreshed on every other token)+ h = CNT_HEAD+ row(qkv[q_off + h * HD + 0], {B0: K})+ row(qkv[q_off + h * HD + 1], {B0: K})+ row(qkv[k_off + h * HD + 0], {CNTW: K})+ row(qkv[k_off + h * HD + 1], {B0: K, CNTW: -K})+ v = v_off + h * HD+ row(qkv[v + 0], {CNTW: K})+ row(qkv[v + 1], {B0: 2.0**20, CNTW: -(2.0**20)})+ row(qkv[v + 2], {B0: 2.0**20, CNTW: -(2.0**20)})+ row(a[h], {B0: 110.0, CNTW: 128.0}) # softplus(0) = ln 2 per token, softplus(128) at reset+ # normed count column = 8 * 2^-w * 128 / 2^20 = 2^-(w+10), times 128 -> 2^-(w+3)+ out[OUT_CNT, h * HD] = 8.0++ # MLPs: silu(K s) / K is an exact relu(s) when K s is a bf16 integer multiple of K.+ # Layer 0: s_m = relu(5 - 6 * 2^m * x) and an AND of three bits. Layer 1: c_m = [w > m].+ m0, m1 = model.layers[0].mlp, model.layers[1].mlp+ for m in range(NM):+ row(m0.gate_proj.weight.data[m], {B0: 5 * K, OUT_CNT: -6 * 2.0**m * K})+ row(m0.up_proj.weight.data[m], {B0: 1 / K})+ m0.down_proj.weight.data[S_DIM + m, m] = 1.0+ for j, c in enumerate((0.0, -1.0)):+ u = 2 * m + j+ row(m1.gate_proj.weight.data[u], {S_DIM + m: K, B0: c * K})+ row(m1.up_proj.weight.data[u], {B0: (1 - 2 * j) / K})+ m1.down_proj.weight.data[C_DIM + m, u] = 1.0+ u = NM+ row(m0.gate_proj.weight.data[u], {OUT_BITS: K, OUT_BITS + 1: K, OUT_FOUND: K, B0: -2 * K})+ row(m0.up_proj.weight.data[u], {B0: 1 / K})+ m0.down_proj.weight.data[AND_DIM, u] = 1.0+ return model.to(dtype)+++def make_sequence(n, seed=0, text_frac=0.1):+ """Random tokens plus the expected chip outputs from a plain Python model of the heads."""+ g = torch.Generator().manual_seed(seed)+ r = lambda hi: int(torch.randint(hi, (1,), generator=g))+ x = torch.zeros(n, D, dtype=torch.float64)+ want = {} # token -> {dim: value}+ ram = {}+ cnt_w = None+ for t in range(n):+ if torch.rand(1, generator=g) < text_frac:+ x[t, GARBAGE:] = torch.randn(D - GARBAGE, generator=g, dtype=torch.float64) * 1e3+ continue+ x[t, B0] = x[t, B1] = BALLAST+ x[t, GARBAGE:] = torch.randn(D - GARBAGE, generator=g, dtype=torch.float64) * 1e3+ rst = r(20) == 0+ cntw = r(25) == 0+ if rst:+ x[t, RST] = 1+ for key in [k for k in ram if k[0] == CLS_HEAD]:+ del ram[key]+ if r(2):+ hd, s, val = r(15), r(HD), r(256)+ x[t, ISW] = 1+ x[t, WSLOT + s] = x[t, WHEAD + hd] = 1+ for i in range(8):+ x[t, BITS + i] = (val >> i) & 1+ ram[hd, s] = val+ exp = {}+ if r(3):+ hd, s = r(15), r(HD)+ x[t, RSLOT + s] = x[t, RHEAD + hd] = 1+ val = ram.get((hd, s))+ for i in range(8):+ exp[OUT_BITS + i] = 0.0 if val is None else float((val >> i) & 1)+ exp[OUT_FOUND] = 0.0 if val is None else 1.0+ exp[AND_DIM] = float(val is not None and val & 3 == 3)+ if cntw:+ x[t, CNTW] = 1+ cnt_w = 0+ elif cnt_w is not None:+ cnt_w += 1+ if cnt_w <= 100: # records are at most 82 tokens; 2^-w stays a normal bf16+ exp[OUT_CNT] = 2.0**-cnt_w+ for m in range(NM):+ exp[C_DIM + m] = float(cnt_w > m)+ else:+ exp[OUT_CNT] = 0.0+ want[t] = exp+ return x, want+++def run(model, x, mode, split=None):+ """Residual after the layer, for prefill (chunked kernel) or prefill+decode (recurrent)."""+ dtype = next(model.parameters()).dtype+ x = x.to(dtype)[None]+ outs = []+ hook = model.layers[-1].register_forward_hook(lambda m, i, o: outs.append(o[0] if isinstance(o, tuple) else o))+ try:+ with torch.no_grad():+ if mode == "prefill":+ model(inputs_embeds=x, use_cache=False)+ else:+ split = split or 1+ o = model(inputs_embeds=x[:, :split], use_cache=True)+ cache = o.past_key_values+ for t in range(split, x.shape[1]):+ o = model(inputs_embeds=x[:, t : t + 1], past_key_values=cache, use_cache=True)+ cache = o.past_key_values+ finally:+ hook.remove()+ return torch.cat([o[0] for o in outs], 0).to(torch.float64)+++def compare(res, want, x):+ """(max abs error on expected outputs, max abs change on text tokens' chip dims, count)."""+ err, n = 0.0, 0+ for t, exp in want.items():+ for d, v in exp.items():+ err = max(err, abs(float(res[t, d]) - v))+ n += 1+ text = [t for t in range(x.shape[0]) if t not in want]+ leak = float((res[text, :GARBAGE] - x[text, :GARBAGE]).abs().max()) if text else 0.0+ return err, leak, ndiff --git a/chipformer/q35/spikes_pos.py b/chipformer/q35/spikes_pos.pynew file mode 100644index 0000000..b12c7da--- /dev/null+++ b/chipformer/q35/spikes_pos.py@@ -0,0 +1,78 @@+"""Spike: does freezing RoPE positions on chip tokens stop Qwen repeating a line after an inline call?++ python -m chipformer.q35.spikes_pos MERGED --ids gsmhard-82 gsmhard-306 --max-new 1500++Decodes each problem twice with a manual greedy loop: positions advancing on every token (what+HF generate does), and positions frozen on chip tokens (Qwen's text sees contiguous positions).+Reports the most repeated line that holds an inline call.+"""++import argparse+import json+import re+from pathlib import Path++import torch++[email protected]_grad()+def decode(model, prompt, chip_lo, chip_hi, max_new, freeze, stop):+ from transformers import DynamicCache++ cache = DynamicCache(config=model.config)+ pos, p = [], 0+ for t in prompt:+ pos.append(p)+ if not (freeze and chip_lo <= t < chip_hi):+ p += 1+ out = model(input_ids=torch.tensor([prompt], device="cuda"),+ position_ids=torch.tensor([pos], device="cuda"), past_key_values=cache,+ use_cache=True)+ gen = []+ for _ in range(max_new):+ t = int(out.logits[0, -1].float().argmax())+ gen.append(t)+ if t in stop:+ break+ pos_t = p+ if not (freeze and chip_lo <= t < chip_hi):+ p += 1+ out = model(input_ids=torch.tensor([[t]], device="cuda"),+ position_ids=torch.tensor([[pos_t]], device="cuda"), past_key_values=cache,+ use_cache=True)+ return gen+++def main():+ from chipformer.q35 import evals+ from chipformer.q35.demo import load+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ ap = argparse.ArgumentParser()+ ap.add_argument("merged")+ ap.add_argument("--ids", nargs="+", default=["gsmhard-82"])+ ap.add_argument("--cond", default="merged_inline")+ ap.add_argument("--max-new", type=int, default=1500)+ args = ap.parse_args()+ meta = json.loads((Path(args.merged) / "chipformer.json").read_text())+ off, nchip = meta["chip_token_offset"], meta["chip_vocab"]+ bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ model, tok = load(args.merged)+ data = {r["id"]: r for r in evals.load_data("gsmhard", 200)}+ stop = {248046, 248044, 248045}+ for i in args.ids:+ prompt = evals.build_prompt(tok, data[i]["question"], args.cond, bios)+ for freeze in (False, True):+ gen = decode(model, prompt, off, off + nchip, args.max_new, freeze, stop)+ shown = re.sub(r"(<\|chip[^|]*\|>)+", "[m]", tok.decode(gen))+ lines = [ln for ln in shown.split("\n") if "[m]" in ln]+ rep = max((lines.count(ln) for ln in lines), default=0)+ print(json.dumps({"id": i, "freeze": freeze, "n": len(gen), "fires": shown.count("[m]"),+ "max_line_repeat": rep, "gold": data[i].get("gold"),+ "tail": shown[-240:]}, ensure_ascii=False), flush=True)+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/spikes_replay.py b/chipformer/q35/spikes_replay.pynew file mode 100644index 0000000..d7b9cc1--- /dev/null+++ b/chipformer/q35/spikes_replay.py@@ -0,0 +1,103 @@+"""Spike: replay a merged model's generation through the circuit IR and compare, token by token.++ python -m chipformer.q35.spikes_replay MERGED --data gsmhard --idx 28 --cond merged_inline++Generates greedily with HF generate (batch 1), maps the tokens to the circuit's alphabet (chip+tokens, text classes, OTHER), evaluates the IR on that stream, and reports every position the+circuit owns where the model's token differs from the IR's prediction, plus each inline fire.+"""++import argparse+import json+import re+from pathlib import Path++import numpy as np+import torch+++def class_map(tok, meta):+ from chipformer.q35 import bridge as B++ m = {int(i): int(cls) for cls, i in meta["text_ids"].items()}+ for cls, strings in B.EXTRA_TEXT.items():+ for s in strings:+ (i,) = tok.encode(s, add_special_tokens=False)+ m[i] = cls+ return m+++def main():+ from chipformer.q35 import bridge as B+ from chipformer.q35 import check as Q+ from chipformer.q35 import chip8_q35, evals+ from chipformer.q35.demo import load+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ ap = argparse.ArgumentParser()+ ap.add_argument("merged")+ ap.add_argument("--data", default="gsmhard")+ ap.add_argument("--idx", type=int, nargs="+", default=[28])+ ap.add_argument("--cond", default="merged_inline")+ ap.add_argument("--max-new", type=int, default=1500)+ ap.add_argument("--n", type=int, default=200)+ ap.add_argument("--ids", help="replay these saved prompt+generated ids (.npy) instead")+ ap.add_argument("--prompt-len", type=int, help="with --ids: the prompt is this many ids")+ args = ap.parse_args()+ meta = json.loads((Path(args.merged) / "chipformer.json").read_text())+ off, nchip = meta["chip_token_offset"], meta["chip_vocab"]+ bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ if args.ids:+ from transformers import AutoTokenizer++ model, tok = None, AutoTokenizer.from_pretrained(args.merged)+ else:+ model, tok = load(args.merged)+ cmap = class_map(tok, meta)+ circuit = chip8_q35.build()+ data = evals.load_data(args.data, args.n)+ for i in args.idx:+ prompt = evals.build_prompt(tok, data[i]["question"], args.cond, bios)+ if args.ids:+ saved = np.load(args.ids).tolist()+ if args.prompt_len:+ prompt = saved[:args.prompt_len]+ assert saved[:len(prompt)] == prompt, "saved ids do not start with this prompt"+ gen = saved[len(prompt):]+ else:+ x = torch.tensor([prompt], device="cuda")+ with torch.no_grad():+ out = model.generate(x, max_new_tokens=args.max_new, do_sample=False)+ gen = out[0, len(prompt):].tolist()+ ids = prompt + gen+ stream = [t - off if off <= t < off + nchip else cmap.get(t, B.OTHER) for t in ids]+ pred, m = Q.predictions(circuit, stream)+ from chipformer.q35 import ir++ owner = ir.ev(circuit.owner, m)+ bad = [p for p in range(len(prompt) - 1, len(ids) - 1)+ if owner[p] == 1 and pred[p] != stream[p + 1]]+ text = tok.decode(gen)+ shown = re.sub(r"(<\|chip[^|]*\|>)+", "[m]", text)+ fires = re.findall(r"([\d,]+)\s*(\\times|\*|×|\+|-)\s*([\d,]+)\s*=\[m\]\s*(\d*)", shown)+ wrong = []+ for a, op, b, r in fires:+ a_, b_ = int(a.replace(",", "")), int(b.replace(",", ""))+ want = {"+": a_ + b_, "-": a_ - b_}.get(op, a_ * b_)+ if r != str(want):+ wrong.append((a, op, b, r, want))+ print(json.dumps({"idx": i, "n_gen": len(gen), "ir_mismatch_at_owned": len(bad),+ "first": [(p - len(prompt), int(stream[p + 1]), int(pred[p]))+ for p in bad[:5]],+ "fires": len(fires), "wrong_fires": wrong}), flush=True)+ if bad:+ p = bad[0]+ print("context:", repr(tok.decode(ids[max(len(prompt), p - 40):p + 2])), flush=True)+ if not args.ids:+ np.save(Path(args.merged).parent / f"replay_{args.data}_{i}.npy", np.array(ids))+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/store.py b/chipformer/q35/store.pynew file mode 100644index 0000000..15e39a2--- /dev/null+++ b/chipformer/q35/store.py@@ -0,0 +1,441 @@+"""The object store (v1): a small filesystem of named texts inside the model.++Qwen uses it with native tool calls:++ put(name, text) store text under name (replaces it)+ append(name, text) add text at the end (creates the file if it is missing)+ get(name) the stored text, token for token+ list() the names, one per line+ delete(name) forget a name++The circuit captures the name and the text *as Qwen writes them* inside the tool call (no extra+trace tokens), keeps them in Gated DeltaNet state, and answers at ``</tool_call>`` with a few+chip records (register and device writes) and a ``<tool_response>`` it prints itself. Any Qwen+token can be stored and printed: every vocabulary row carries an 18-bit code of its own id+(``ID_BITS``), so the circuit reads a text token's id and can predict any token.++Layout (``Device``):++ content chunk g holds the text of one put / append call, up to CMAX tokens (rows of+ 8 tokens; Qwen3.5-4B's two value heads per key head hold 4 each)+ meta chunk g: (length, next chunk, has next), a file is a chain of chunks+ dir entry k: (name tokens, name length, alive), heads of its chain in dirH / dirT++The registers live in unused cells between the keys and the display, the ports there too: a+write to a port stores a row of a device table (the row is chosen by a register).++``StoreMachine`` is the reference: it extends ``bridge.BridgeMachine`` with Qwen's tool calls and+the circuit's answers, token for token.+"""++from dataclasses import dataclass, field++from chipformer import spec+from chipformer.q35 import bridge as B++ID_BITS = 18 # Qwen3.5's vocabulary (248320 ids) in 18 bits+QWEN_VOCAB = 248320+SPECIAL = 248044 # <|endoftext|> and up: never stored or printed by the store+NMAX = 6 # name tokens kept (a name's trailing newline counts)+# text tokens per call (one chunk); a longer text keeps its first CMAX. The circuit counts them+# with a row that halves per token (2^-n), and 2^-(n+14) must stay a normal float: n <= 112+CMAX = 104+CHUNK = 128 # address stride of a chunk+NFILES_MAX = 127++# registers and ports (cells 0x1040.. are unused by CHIP-8)+NEXTG, NFILES, SK, GCUR, TMP, TMPL = range(0x1040, 0x1046)+DIRW, DIRH, DIRT, METAW, METAN = range(0x1050, 0x1055)+REGS = {"NEXTG": NEXTG, "NFILES": NFILES, "SK": SK, "GCUR": GCUR, "TMP": TMP, "TMPL": TMPL}+REG_BITS = {NEXTG: 10, NFILES: 7, SK: 7, GCUR: 10, TMP: 10, TMPL: 7}+PORTS = {"DIRW": DIRW, "DIRH": DIRH, "DIRT": DIRT, "METAW": METAW, "METAN": METAN}++OPS = ("put", "append", "get", "list", "delete")++# Qwen3.5 token ids (tests check them against the tokenizer)+Q_NL, Q_EQ, Q_FUNCTION, Q_PARAMETER, Q_LTSLASH = 198, 28, 1628, 15704, 510+Q_LT, Q_GT = 27, 29+# the function name: "<function=put>" is "<" "function" "=" "put" ">", "=get" / "=list" are one token+OP_BARE = {"put": 611, "append": 4928, "delete": 4394, "get": 447, "list": 1551}+OP_JOINED = {"get": 27362, "list": 39619}+PARAM_BARE = {"name": 591, "text": 1272}+PARAM_JOINED = {"name": 37943, "text": 46696}+MSG = {"ok": [547], "not_found": [1856, 1669], "empty": [3092]} # "ok", "not" " found", "empty"+TOOLS = [+ {"type": "function", "function": {+ "name": "put", "description": "Store a text under a name in your memory (replaces it).",+ "parameters": {"type": "object", "properties": {+ "name": {"type": "string", "description": "a short name"},+ "text": {"type": "string", "description": f"the text (up to {CMAX} tokens)"}},+ "required": ["name", "text"]}}},+ {"type": "function", "function": {+ "name": "append", "description": "Add a text at the end of a stored text.",+ "parameters": {"type": "object", "properties": {+ "name": {"type": "string"}, "text": {"type": "string"}},+ "required": ["name", "text"]}}},+ {"type": "function", "function": {+ "name": "get", "description": "The text stored under a name, exactly.",+ "parameters": {"type": "object", "properties": {"name": {"type": "string"}},+ "required": ["name"]}}},+ {"type": "function", "function": {+ "name": "list", "description": "The stored names.",+ "parameters": {"type": "object", "properties": {}}}},+ {"type": "function", "function": {+ "name": "delete", "description": "Forget a stored name.",+ "parameters": {"type": "object", "properties": {"name": {"type": "string"}},+ "required": ["name"]}}},+]+++def qtok(qid: int, cls: int | None = None) -> int:+ """A Qwen text token as the IR sees it: its class below bit 16, its id above."""+ return (qid << 16) | (B.OTHER if cls is None else cls)+++def qid_of(t: int) -> int | None:+ return t >> 16 if t >> 16 else None+++def cls_of(t: int) -> int:+ return t & 0xFFFF+++# --- the vocabulary: per-token features of Qwen's rows -------------------------------------+++def default_tokenizer() -> str:+ """A Qwen3.5 tokenizer.json: $QWEN_TOKENIZER, or one of the usual checkpoint folders."""+ import os+ from pathlib import Path++ cands = [os.environ.get("QWEN_TOKENIZER", "")]+ for root in (Path(__file__).resolve().parents[2], Path("D:/weights"), Path.home() / "weights"):+ for sub in ("qwen3.5-0.8b-chip8", "q4-v15", "q4-v14", "q4-v13", "q35-tiny"):+ cands.append(str(root / "dist" / sub / "tokenizer.json"))+ for c in cands:+ if c and Path(c).exists():+ return c+ raise FileNotFoundError("no Qwen3.5 tokenizer.json found; set QWEN_TOKENIZER")+++class Vocab:+ """Qwen token strings, their text classes and the store's per-row features."""++ def __init__(self, tokenizer_json: str | None = None):+ from tokenizers import Tokenizer++ tok = Tokenizer.from_file(str(tokenizer_json or default_tokenizer()))+ self.tok = tok+ self.strings = {i: tok.decode([i]) for i in range(SPECIAL)}+ self.class_of: dict[int, int] = {}+ for cls, s in B.QWEN_TEXT.items():+ (i,) = tok.encode(s, add_special_tokens=False).ids+ self.class_of[i] = cls+ for cls, ss in B.EXTRA_TEXT.items():+ for s in ss:+ (i,) = tok.encode(s, add_special_tokens=False).ids+ self.class_of[i] = cls++ def has_nl(self, qid: int) -> bool:+ return "\n" in self.strings.get(qid, "")++ def cls(self, qid: int) -> int:+ return self.class_of.get(qid, B.OTHER)++ def tok_ids(self, text: str) -> list[int]:+ return self.tok.encode(text, add_special_tokens=False).ids++ def tokens(self, qids: list[int]) -> list[int]:+ return [qtok(q, self.cls(q)) for q in qids]+++def id_code(qid: int | None) -> list[int]:+ """The 18 id bits of a token row (0 for chip tokens and for the special tokens)."""+ if qid is None or qid >= SPECIAL:+ return [0] * ID_BITS+ return [qid >> i & 1 for i in range(ID_BITS)]+++# --- the reference ---------------------------------------------------------------------------+++@dataclass+class Entry:+ name: tuple+ length: int+ alive: int+++@dataclass+class Device:+ content: dict = field(default_factory=dict) # (chunk, index) -> qid+ meta: dict = field(default_factory=dict) # chunk -> (length, next, has_next)+ dir: dict = field(default_factory=dict) # entry -> Entry+ dir_h: dict = field(default_factory=dict)+ dir_t: dict = field(default_factory=dict)+++class StoreScan:+ """What the circuit captures from Qwen's tool calls (the reference for its capture).++ Capture happens only inside a tool call (from <tool_call> to </tool_call>), so the store's+ own prints, which may contain any text, never capture anything."""++ def __init__(self, vocab: Vocab, n_chunks: int):+ self.v = vocab+ self.n_chunks = n_chunks+ self.prev = self.prev2 = None+ self.in_call = False+ self.op = None+ self.pk = None # "name" / "text" while inside a parameter+ self.nlc = 0 # newline tokens since the parameter's name+ self.name: list[int] = [] # name value tokens (all of them; NMAX are kept)+ self.n_text = 0 # text value tokens++ def feed(self, t: int, nextg: int, dev: Device) -> bool:+ """Feed one token; returns True at a store call's </tool_call>."""+ q, c = qid_of(t), cls_of(t)+ prev, prev2 = self.prev, self.prev2+ self.prev, self.prev2 = q, prev+ if c == B.TC_OPEN:+ self.in_call = True+ self.op, self.pk, self.nlc, self.name, self.n_text = None, None, 0, [], 0+ return False+ if c == B.TC_CLOSE or t in (spec.STEP, spec.BOOT):+ was = self.in_call+ self.in_call = False+ return c == B.TC_CLOSE and was and self.op is not None+ if q is None or not self.in_call: # chip tokens, and text outside tool calls+ return False+ for op, i in OP_BARE.items():+ if q == i and prev == Q_EQ and prev2 == Q_FUNCTION:+ self.op = op+ for op, i in OP_JOINED.items():+ if q == i and prev == Q_FUNCTION:+ self.op = op+ for p, i in PARAM_BARE.items():+ if q == i and prev == Q_EQ and prev2 == Q_PARAMETER:+ self._param(p)+ return False+ for p, i in PARAM_JOINED.items():+ if q == i and prev == Q_PARAMETER:+ self._param(p)+ return False+ closer = q == Q_PARAMETER and prev == Q_LTSLASH+ nl = self.v.has_nl(q)+ if self.pk is not None and not closer:+ if nl:+ self.nlc += 1+ if self.nlc >= 2 or (self.nlc >= 1 and not nl):+ if self.pk == "name":+ self.name.append(q)+ else:+ if self.n_text < CMAX and nextg < self.n_chunks:+ dev.content[nextg, self.n_text] = q if q < SPECIAL else 0+ self.n_text += 1+ if closer:+ self.pk = None+ return False++ def _param(self, p):+ self.pk, self.nlc = p, 0+ if p == "name":+ self.name = []+ else:+ self.n_text = 0++ def name_key(self) -> tuple:+ """The name as the directory compares it: (length, the first NMAX tokens)."""+ n = max(min(len(self.name), NMAX + 2) - 1, 0) # the closer's "</" was counted+ toks = [q if q < SPECIAL else 0 for q in self.name[:NMAX]]+ return n, tuple(toks + [0] * (NMAX - len(toks)))++ def text_len(self) -> int:+ return max(min(self.n_text, CMAX + 1) - 1, 0)+++class StoreMachine(B.BridgeMachine):+ """Reference trace with the object store: Qwen's text and tool calls, the circuit's+ capture and answers."""++ def __init__(self, vocab: Vocab, rom=None, quirks=None, n_chunks: int = 640):+ from chipformer.q35.lib import bcd++ super().__init__(rom or bcd.ROM, quirks or bcd.QUIRKS)+ self.v = vocab+ self.dev = Device()+ self.n_chunks = n_chunks+ self.sscan = StoreScan(vocab, n_chunks)+ self.answers: list[tuple] = []+ for t in self.tokens:+ self.sscan.feed(t, 0, self.dev)++ @classmethod+ def resume(cls, prev: "StoreMachine") -> "StoreMachine":+ """A new session (its own boot record and text) whose store is ``prev``'s: what a+ restored snapshot gives (store_snapshot.py)."""+ import copy++ m = cls(prev.v, n_chunks=prev.n_chunks)+ m.dev = copy.deepcopy(prev.dev)+ for a in REGS.values():+ m.emu.cells[a] = prev.emu.cells[a]+ return m++ # --- the stream ------------------------------------------------------------------------++ def _emit(self, tokens: list[int], generated: bool) -> None:+ for t in tokens:+ fire = self.sscan.feed(t, self.emu.cells[NEXTG], self.dev)+ super()._emit([t], generated)+ if fire and not generated:+ self._respond()++ def qwen_text(self, qids: list[int]) -> None:+ """Qwen writes these tokens (no arithmetic is answered inside the store's tests)."""+ self._emit(self.v.tokens(qids), False)++ def call(self, op: str, name: list[int] | None = None, text: list[int] | None = None):+ """Qwen's native tool call, as Qwen3.5's template spells it; returns the answer."""+ t = [248058, Q_NL, Q_LT, Q_FUNCTION] # <tool_call>+ t += [OP_JOINED[op]] if op in OP_JOINED else [Q_EQ, OP_BARE[op]]+ t += [Q_GT, Q_NL]+ for p, val in (("name", name), ("text", text)):+ if val is None:+ continue+ t += [Q_LT, Q_PARAMETER, PARAM_JOINED[p], Q_GT, Q_NL, *val, Q_NL, Q_LTSLASH,+ Q_PARAMETER, Q_GT, Q_NL]+ t += [Q_LTSLASH, Q_FUNCTION, Q_GT, Q_NL, 248059] # </tool_call>+ n = len(self.answers)+ self._emit(self.v.tokens(t), False)+ assert len(self.answers) == n + 1, "the call did not reach the store"+ return self.answers[-1]++ # --- the circuit's answer --------------------------------------------------------------++ def _print_cls(self, cls: int) -> None:+ self._emit([qtok(full_qid(cls), cls)], True)++ def _print_qid(self, q: int) -> None:+ self._emit([qtok(q, self.v.cls(q))], True)++ def _write(self, a: int, v: int) -> None:+ cells, dev = self.emu.cells, self.dev+ self.emu.apply([(a, v)])+ if a == DIRW:+ n, toks = self.sscan.name_key()+ dev.dir[cells[SK]] = Entry(toks, n, v & 1)+ elif a == DIRH:+ dev.dir_h[cells[SK]] = v+ elif a == DIRT:+ dev.dir_t[cells[SK]] = v+ elif a == METAW:+ if cells[NEXTG] < self.n_chunks:+ dev.meta[cells[NEXTG]] = (v, 0, 0)+ elif a == METAN:+ if cells[TMP] < self.n_chunks:+ dev.meta[cells[TMP]] = (cells[TMPL], v, 1)+ self._emit([spec.addr_token(a), spec.val_token(v)], True)++ def _match(self, k: int) -> bool:+ e = self.dev.dir.get(k)+ n, toks = self.sscan.name_key()+ return e is not None and e.alive == 1 and e.name == toks and e.length == n++ def _respond(self) -> None:+ op, cells, dev = self.sscan.op, self.emu.cells, self.dev+ nf, g = cells[NFILES], cells[NEXTG]+ header = (B.NL, B.TR_OPEN, B.NL)++ def head():+ for cls in header:+ self._print_cls(cls)++ def msg(name):+ for q in MSG[name]:+ self._print_qid(q)+ self._print_cls(B.NL)++ def close():+ self._print_cls(B.TR_CLOSE)+ self._print_cls(B.NL)++ if op == "list":+ head()+ if nf == 0:+ msg("empty")+ else:+ self._write(SK, 0)+ while cells[SK] != nf:+ e = dev.dir.get(cells[SK])+ if e is not None and e.alive:+ for q in e.name[:min(e.length, NMAX)]:+ self._print_qid(q)+ self._write(SK, cells[SK] + 1)+ close()+ self.answers.append(("list",))+ return+ self._write(SK, 0)+ while not (self._match(cells[SK]) or cells[SK] == nf):+ self._write(SK, cells[SK] + 1)+ found = self._match(cells[SK])+ k = cells[SK]+ if op in ("put", "append") and not (op == "append" and found):+ if not found:+ self._write(NFILES, nf + 1)+ self._write(DIRW, 1)+ self._write(DIRH, g)+ self._write(DIRT, g)+ elif op == "append":+ self._write(TMP, dev.dir_t.get(k, 0))+ self._write(TMPL, dev.meta.get(cells[TMP], (0, 0, 0))[0])+ self._write(METAN, g)+ self._write(DIRT, g)+ if op in ("put", "append"):+ self._write(METAW, self.sscan.text_len())+ self._write(NEXTG, g + 1)+ head()+ msg("ok")+ elif op == "delete":+ if found:+ self._write(DIRW, 0)+ head()+ msg("ok" if found else "not_found")+ elif op == "get":+ head()+ if not found:+ msg("not_found")+ else:+ self._write(GCUR, dev.dir_h.get(k, 0))+ while True:+ n, nxt, has = dev.meta.get(cells[GCUR], (0, 0, 0))+ for i in range(n):+ self._print_qid(dev.content.get((cells[GCUR], i), 0))+ if not has:+ break+ self._write(GCUR, nxt)+ close()+ self.answers.append((op, found))++ def stored(self, name_key) -> list[int] | None:+ """The text of a file (for tests): its chunks' tokens in order."""+ for k, e in self.dev.dir.items():+ if e.alive and (e.length, e.name) == name_key:+ out, g = [], self.dev.dir_h[k]+ while True:+ n, nxt, has = self.dev.meta.get(g, (0, 0, 0))+ out += [self.dev.content.get((g, i), 0) for i in range(n)]+ if not has:+ return out+ g = nxt+ return None+++def full_qid(cls: int) -> int:+ """The Qwen id of a printable class (its one spelling)."""+ return QWEN_CLASS_IDS[cls]+++# the ids of the classes the store prints (Qwen3.5 tokenizer; checked by the tests)+QWEN_CLASS_IDS = {B.NL: Q_NL, B.TR_OPEN: 248066, B.TR_CLOSE: 248067, B.TC_OPEN: 248058,+ B.TC_CLOSE: 248059}diff --git a/chipformer/q35/store_check.py b/chipformer/q35/store_check.pynew file mode 100644index 0000000..4ef9bb6--- /dev/null+++ b/chipformer/q35/store_check.py@@ -0,0 +1,136 @@+"""Differential check of the store circuit against store.StoreMachine, token by token."""++import numpy as np++from chipformer.q35 import bridge as B+from chipformer.q35 import ir+from chipformer.q35 import store as S+from chipformer.q35.chip8_q35 import decode_code+++def predictions(c, vocab: S.Vocab, tokens: list[int], check: bool = True,+ chunk: int | None = None):+ """The circuit's next token at every position (any-prints as Qwen tokens), the owner bit,+ and the residual stream (of the last chunk, with ``chunk``: a long stream is evaluated+ that many tokens at a time)."""+ if chunk is not None and len(tokens) > chunk:+ carry, preds, owners = {}, [], []+ for i in range(0, len(tokens), chunk):+ p, o, m = _predict(c, vocab, tokens[i:i + chunk], check, carry)+ preds.append(p)+ owners.append(o)+ return np.concatenate(preds), np.concatenate(owners), m+ return _predict(c, vocab, tokens, check, None)+++def _predict(c, vocab, tokens, check, carry):+ m = ir.evaluate(c, tokens, check, carry)+ bits = np.stack([ir.ev(b, m) for b in c.next_bits], 1)+ assert np.isin(bits, [0.0, 1.0]).all(), "output bits must be 0/1"+ anyp = ir.ev(c.anyp, m)+ ybits = np.stack([ir.ev(b, m) for b in c.ybits], 1)+ assert np.isin(anyp, [0.0, 1.0]).all() and np.isin(ybits, [0.0, 1.0]).all()+ pred = []+ for row, a, y in zip(bits.astype(int).tolist(), anyp, ybits.astype(int).tolist()):+ if a:+ q = sum(b << i for i, b in enumerate(y))+ pred.append(S.qtok(q, vocab.cls(q)))+ else:+ t = decode_code(row)+ if B.is_text(t) and t in S.QWEN_CLASS_IDS:+ t = S.qtok(S.full_qid(t), t)+ pred.append(t)+ return np.array(pred, dtype=np.int64), ir.ev(c.owner, m), m+++def check(c, vocab, sm: S.StoreMachine, limit: int = 10, chunk: int | None = None):+ """(checked positions, first wrong predictions, first wrong owner bits)."""+ pred, owner, _ = predictions(c, vocab, sm.tokens, chunk=chunk)++ def norm(t): # a printed class token with or without its id is the same token+ t = int(t)+ return S.cls_of(t) if t >= 0 and S.cls_of(t) in S.QWEN_CLASS_IDS and B.is_text(t) else t++ pred = np.array([norm(t) for t in pred], dtype=np.int64)+ want = np.array([norm(t) for t in sm.tokens[1:]] + [-1], dtype=np.int64)+ gen = np.array(sm.generated[1:] + [False])+ bad = [(int(p), int(want[p]), int(pred[p])) for p in np.nonzero(gen & (pred != want))[0]]+ ow = B.owner_expected(sm)+ bad_owner = [(p, float(owner[p]), w) for p, w in enumerate(ow)+ if w is not None and owner[p] != w]+ return int(gen.sum()), bad[:limit], bad_owner[:limit]+++def hf_model(c, vocab_tokens: list[int], dtype=None):+ """A stock Qwen3_5ForCausalLM holding the circuit (with the store), whose token i is+ ``vocab_tokens[i]`` (chip tokens and Qwen text tokens as qtok ids): runs the store in real+ HF modules without a Qwen checkpoint."""+ import torch+ from transformers import Qwen3_5ForCausalLM, Qwen3_5TextConfig++ from chipformer.q35 import tiny+ from chipformer.q35.emit import allocate, check_bf16, embedding_rows, emit_layers+ from chipformer.q35.merge import ID_GAIN++ dtype = dtype or torch.bfloat16+ keep = set(c.keep)+ alloc = allocate(c, None, keep, c.target.intermediate)+ lay = tiny.pick_layout(alloc.n_slots + 1)+ lay.slots = alloc.slots+ n_chip = c.n_layers()+ n_vocab = -(-len(vocab_tokens) // 64) * 64+ tc = dict(tiny.BASE, hidden_size=lay.hidden, vocab_size=n_vocab, num_hidden_layers=n_chip + 1,+ layer_types=["linear_attention"] * n_chip + ["full_attention"],+ linear_num_value_heads=c.target.value_heads, intermediate_size=c.target.intermediate)+ sd = {k: v.to(dtype) for k, v in emit_layers(c, lay, tc, 0, "model.", alloc, keep).items()}+ emb = torch.zeros(n_vocab, lay.hidden)+ emb[: len(vocab_tokens)] = embedding_rows(c, lay, vocab_tokens, lay.hidden)+ sd["model.embed_tokens.weight"] = emb.to(dtype)+ norm = torch.full((lay.hidden,), -1.0)+ for e in c.code:+ norm[lay.dim(next(iter(e.terms)))] = tiny.code_gain(lay)+ for e in (c.store.yc if c.store is not None else []):+ norm[lay.dim(next(iter(e.terms)))] = ID_GAIN * tiny.code_gain(lay)+ check_bf16("final norm", norm)+ sd["model.norm.weight"] = norm.to(dtype)+ config = Qwen3_5TextConfig(**tc)+ old = torch.get_default_dtype()+ torch.set_default_dtype(dtype) # (no full-size fp32 copy of the model)+ try:+ model = Qwen3_5ForCausalLM(config)+ finally:+ torch.set_default_dtype(old)+ for p in model.parameters():+ p.data.zero_()+ missing, unexpected = model.load_state_dict(sd, strict=False)+ assert not unexpected, unexpected[:5]+ del sd+ model.tie_weights()+ model.layout = lay+ return model.to(dtype).eval()+++def hf_check(model, sm, vocab_tokens):+ """Teacher-forced argmax of the HF model at every generated position: the mismatches."""+ import torch++ index = {t: i for i, t in enumerate(vocab_tokens)}+ ids = torch.tensor([[index[t] for t in sm.tokens]])+ with torch.no_grad():+ logits = model(input_ids=ids).logits[0]+ pred = logits.argmax(-1).tolist()+ bad = [(p, sm.tokens[p + 1], vocab_tokens[pred[p]]) for p in range(len(sm.tokens) - 1)+ if sm.generated[p + 1] and vocab_tokens[pred[p]] != sm.tokens[p + 1]]+ return sum(sm.generated[1:]), bad+++def name(t: int) -> str:+ """A token, readable (for failure messages)."""+ from chipformer import spec++ q = S.qid_of(t)+ if q is not None:+ return f"q{q}"+ if B.is_text(t):+ return f"cls{t - B.TEXT_BASE}"+ return spec.token_name(t)diff --git a/chipformer/q35/store_eval.py b/chipformer/q35/store_eval.pynew file mode 100644index 0000000..259d6dc--- /dev/null+++ b/chipformer/q35/store_eval.py@@ -0,0 +1,397 @@+"""GPU evals of the object store on a merged checkpoint (stock HF modules, greedy decoding).++ python -m chipformer.q35.store_eval exact MERGED # teacher-forced: the circuit's+ # answers == the reference+ python -m chipformer.q35.store_eval recall MERGED BASE --n 50 --pad 10000 100000+ python -m chipformer.q35.store_eval session MERGED # snapshot -> fresh session++``recall``: Qwen stores n random 20-token strings with ``put`` (one user turn each), then a long+unrelated chat, then for each string a separate question; exact match of its final answer.+Against base Qwen told the same strings in its context, and base Qwen with the strings right+before the question (no padding).+"""++import argparse+import re+import copy+import json+import random+import time+from pathlib import Path++import torch++from chipformer.q35 import store as S++SYSTEM = ("You have a small memory store: put saves a text under a name, get returns it "+ "exactly, list shows the names, append adds to a text, delete removes a name. Use it "+ "whenever you are asked to save or recall something.")+IM_START, IM_END, EOT = 248045, 248046, 248044+++class Runner:+ """Greedy decoding with a stock DynamicCache (chunked prefill)."""++ def __init__(self, path, device="cuda"):+ from transformers import AutoModelForImageTextToText, AutoTokenizer++ self.tok = AutoTokenizer.from_pretrained(path)+ self.model = AutoModelForImageTextToText.from_pretrained(+ path, dtype=torch.bfloat16).to(device).eval()+ self.dev = device+ self.lm_head = self.model.lm_head+ meta = Path(path) / "chipformer.json"+ self.meta = json.loads(meta.read_text()) if meta.exists() else None++ def ids(self, text: str) -> list[int]:+ return self.tok.encode(text, add_special_tokens=False)++ @torch.no_grad()+ def feed(self, cache, ids: list[int], chunk: int = 2048):+ """Prefill ``ids`` into ``cache``; the logits of the last position."""+ from transformers import DynamicCache++ cache = cache if cache is not None else DynamicCache(config=self.model.config)+ out = None+ for i in range(0, len(ids), chunk):+ x = torch.tensor([ids[i:i + chunk]], device=self.dev)+ out = self.model(input_ids=x, past_key_values=cache, use_cache=True,+ logits_to_keep=1)+ return cache, out.logits[0, -1].float()++ @torch.no_grad()+ def generate(self, cache, ids: list[int], max_new: int, stop=(IM_END,)):+ """Greedy, with <|im_start|> and <|endoftext|> banned (Qwen ends a turn with <|im_end|>;+ evals' "nb" and "ne")."""+ cache, logits = self.feed(cache, ids)+ out = []+ for _ in range(max_new):+ logits[IM_START] = float("-inf")+ logits[EOT] = float("-inf")+ t = int(logits.argmax())+ out.append(t)+ cache, logits = self.feed(cache, [t]) # (the cache holds every token, the end too)+ if t in stop:+ break+ return cache, out++ def decode(self, ids: list[int]) -> str:+ lo = self.meta["chip_token_offset"] if self.meta else 1 << 30+ hi = lo + (self.meta["chip_vocab"] if self.meta else 0)+ return self.tok.decode([t for t in ids if not lo <= t < hi], skip_special_tokens=False)+++def turn(role: str, text: str) -> str:+ return f"<|im_start|>{role}\n{text}<|im_end|>\n"+++def opening(r: Runner, tools: bool) -> list[int]:+ msgs = [{"role": "system", "content": SYSTEM}] if tools else []+ s = r.tok.apply_chat_template(msgs + [{"role": "user", "content": "hi"}],+ tools=S.TOOLS if tools else None, tokenize=False,+ add_generation_prompt=False, enable_thinking=False)+ return r.ids(s[:s.rindex("<|im_start|>user")]) # the system part+++ASSISTANT = "<|im_start|>assistant\n<think>\n\n</think>\n\n"+++def strings(rng: random.Random, r: Runner, n: int) -> list[str]:+ """n random strings of about 20 tokens: hex ids, uuids, code lines, word salad."""+ words = ("alpha bravo kernel tensor lattice orbit cobalt maple vortex quartz ember delta "+ "harbor pixel signal falcon cedar nimbus").split()+ out = []+ while len(out) < n:+ kind = rng.randrange(4)+ if kind == 0:+ s = "".join(rng.choice("0123456789abcdef") for _ in range(rng.randint(24, 34)))+ elif kind == 1:+ h = "".join(rng.choice("0123456789abcdef") for _ in range(32))+ s = f"{h[:8]}-{h[8:12]}-{h[12:16]}-{h[16:20]}-{h[20:]}"+ elif kind == 2:+ v = rng.choice(["x", "acc", "w_2", "buf"])+ s = (f"{v} = np.{rng.choice(['dot', 'clip', 'roll'])}({v}[{rng.randint(1, 9)}:], "+ f"{rng.randint(10, 999)}) + {rng.random():.4f}")+ else:+ s = " ".join(rng.choice(words) for _ in range(rng.randint(8, 12))) + f" {rng.randint(100, 9999)}"+ if 16 <= len(r.ids(s)) <= 24:+ out.append(s)+ return out+++def fires_inline(r: Runner, ids: list[int]) -> bool:+ """Would the circuit answer an inline "A op B =" in this text? Prefilled history must not+ contain one: the prompt would lack the circuit's answer and leave it mid-call."""+ from chipformer.q35.bridge import InlineScan++ vocab = getattr(r, "_vocab", None)+ if vocab is None:+ vocab = r._vocab = S.Vocab(Path(r.tok.name_or_path) / "tokenizer.json")+ scan = InlineScan()+ return any(scan.feed(vocab.cls(q) if q < S.SPECIAL else 0) for q in ids)+++def padding(rng: random.Random, r: Runner, n_tokens: int) -> list[int]:+ """Unrelated chat turns (GSM8K questions and answers when available) of about n tokens, none+ of which the circuit's inline arithmetic would answer (see fires_inline)."""+ pairs = []+ try:+ from datasets import load_dataset++ ds = load_dataset("openai/gsm8k", "main", split="train")+ pairs = [(x["question"], x["answer"]) for x in ds]+ except Exception:+ words = "the a of to and in is it for on with as was at by an be this that from".split()+ for _ in range(5000):+ q = " ".join(rng.choice(words) for _ in range(30)) + "?"+ pairs.append((q, " ".join(rng.choice(words) for _ in range(60)) + "."))+ rng.shuffle(pairs)+ out = []+ for q, a in pairs:+ a = re.sub(r"<<[^>]*>>", "", a) # GSM8K's calculator annotations+ ids = r.ids(turn("user", q) + turn("assistant", a))+ if fires_inline(r, ids):+ continue+ out += ids+ if len(out) >= n_tokens:+ break+ return out+++def is_exact(name: str, ans: str, want: str) -> bool:+ """The answer is the stored text (allowing quotes, backticks, a final period and a leading+ "name:")."""+ a = ans.strip()+ if a.startswith(name + ":"):+ a = a[len(name) + 1:].strip()+ return a.strip("`\"' ") == want or a.strip("`\"' .") == want+++def final_answer(r: Runner, gen: list[int]) -> str:+ text = r.decode(gen)+ if "</tool_response>" in text:+ text = text[text.rindex("</tool_response>") + len("</tool_response>"):]+ text = re.sub(r"<think>.*?</think>", "", text, flags=re.S) # (Qwen opens a thinking block)+ return text.replace("<|im_end|>", "").replace("<|endoftext|>", "").strip()+++def tool_text(r: Runner, gen: list[int]) -> str | None:+ text = r.decode(gen)+ if "<tool_response>" not in text:+ return None+ a = text.index("<tool_response>") + len("<tool_response>\n")+ return text[a:text.index("</tool_response>", a)].rstrip("\n")+++def recall(args):+ rng = random.Random(args.seed)+ merged = Runner(args.merged)+ items = strings(rng, merged, args.n)+ names = [f"s{i + 1}" for i in range(args.n)]+ res = {"n": args.n, "strings": items, "conditions": {}}+ t0 = time.time()+ # --- merged: Qwen stores them itself+ ctx = opening(merged, True)+ cache, _ = merged.feed(None, ctx)+ puts = []+ for name, s in zip(names, items):+ q = merged.ids(turn("user", f"Save this exactly under the name {name}: {s}") + ASSISTANT)+ cache, gen = merged.generate(cache, q, 400)+ ctx += q + gen + merged.ids("\n")+ cache, _ = merged.feed(cache, merged.ids("\n"))+ puts.append({"name": name, "said": merged.decode(gen)[-300:],+ "stored_ok": tool_text(merged, gen) == "ok"})+ res["puts"] = puts+ print(f"stored {sum(p['stored_ok'] for p in puts)}/{args.n} ({time.time() - t0:.0f}s)",+ flush=True)+ asks = {"plain": "What is stored under the name {name}? Look it up and reply with exactly "+ "that text, nothing else.",+ "tool": "Use the get tool to look up the name {name}, then reply with exactly the "+ "stored text, nothing else."}+ if args.fresh: # a new session: only the store's heads carry over (store_snapshot.py)+ from chipformer.q35 import store_snapshot as snap++ state = snap.snapshot(cache, merged.meta["store"]["persistent_heads"])+ del cache+ torch.cuda.empty_cache()+ for pad in args.pad:+ pad_ids = padding(random.Random(args.seed + pad), merged, pad)+ if args.fresh:+ c2, _ = merged.feed(snap.restore(merged.model, state), opening(merged, True) + pad_ids)+ else:+ c2, _ = merged.feed(copy.deepcopy(cache), pad_ids)+ for ask in args.ask:+ rows = []+ for name, s in zip(names, items):+ q = merged.ids(turn("user", asks[ask].format(name=name)) + ASSISTANT)+ _, gen = merged.generate(copy.deepcopy(c2), q, 300)+ ans, tt = final_answer(merged, gen), tool_text(merged, gen)+ rows.append({"name": name, "want": s, "answer": ans, "tool": tt,+ "exact": is_exact(name, ans, s), "tool_exact": tt == s})+ key = (("fresh" if args.fresh else "merged") + f"_pad{pad}"+ + ("" if ask == "plain" else f"_{ask}"))+ res["conditions"][key] = rows+ print(f"{key}: exact {sum(x['exact'] for x in rows)}/{len(rows)}, "+ f"get called {sum(x['tool'] is not None for x in rows)}, "+ f"printed exact {sum(x['tool_exact'] for x in rows)} ({time.time() - t0:.0f}s)",+ flush=True)+ del c2+ torch.cuda.empty_cache()+ if args.merged_only or args.fresh:+ Path(args.out).write_text(json.dumps(res, indent=2, ensure_ascii=False))+ return+ del merged, cache+ torch.cuda.empty_cache()+ # --- base: the strings in its context+ base = Runner(args.base)+ ctx = opening(base, False)+ for name, s in zip(names, items):+ ctx += base.ids(turn("user", f"Remember this exactly; its name is {name}: {s}")+ + turn("assistant", "Noted."))+ cache, _ = base.feed(None, ctx)+ for pad in args.pad:+ pad_ids = padding(random.Random(args.seed + pad), base, pad)+ c2, _ = base.feed(copy.deepcopy(cache), pad_ids)+ rows = []+ for name, s in zip(names, items):+ q = base.ids(turn("user", f"What exactly was {name}? Reply with exactly that text, "+ "nothing else.") + ASSISTANT)+ _, gen = base.generate(copy.deepcopy(c2), q, 120)+ ans = final_answer(base, gen)+ rows.append({"name": name, "want": s, "answer": ans,+ "exact": is_exact(name, ans, s)})+ res["conditions"][f"base_pad{pad}"] = rows+ print(f"base pad {pad}: exact {sum(x['exact'] for x in rows)}/{len(rows)}", flush=True)+ rows = [] # the string right before its question+ for name, s in zip(names, items):+ q = opening(base, False) + base.ids(turn("user", f"{name}: {s}\nWhat exactly is {name}? "+ "Reply with exactly that text, nothing else.")+ + ASSISTANT)+ _, gen = base.generate(None, q, 120)+ ans = final_answer(base, gen)+ rows.append({"name": name, "want": s, "answer": ans,+ "exact": is_exact(name, ans, s)})+ res["conditions"]["base_in_context"] = rows+ print(f"base in context: exact {sum(x['exact'] for x in rows)}/{len(rows)}", flush=True)+ Path(args.out).write_text(json.dumps(res, indent=2, ensure_ascii=False))+++def session(args):+ """Session 1 saves notes; its store is snapshotted; session 2 starts fresh and recalls."""+ from chipformer.q35 import store_snapshot as snap++ r = Runner(args.merged)+ heads = r.meta["store"]["persistent_heads"]+ notes = [("release", "Ship v2.1 on Friday 14:00 UTC; Priya signs off; freeze main at noon."),+ ("wifi", "SSID garden-5G, password 7Tq!pw-93xz"),+ ("todo", "renew the TLS cert, email Marco the Q3 numbers")]+ ctx = opening(r, True)+ cache, _ = r.feed(None, ctx)+ log = []+ for name, text in notes:+ q = r.ids(turn("user", f"Please save this under the name {name}: {text}") + ASSISTANT)+ cache, gen = r.generate(cache, q, 400)+ log.append(("user", f"save {name}"), )+ log.append(("assistant", r.decode(gen)))+ cache, _ = r.feed(cache, r.ids("\n"))+ state = snap.snapshot(cache, heads)+ path = Path(args.out).with_suffix(".snapshot.pt")+ snap.save(state, path)+ size = path.stat().st_size+ del cache+ # session 2: a new conversation, nothing of session 1 in Qwen's context+ cache = snap.restore(r.model, snap.load(path))+ q = opening(r, True) + r.ids(turn("user", "In an earlier session I asked you to save some "+ "notes. Which notes do you have? Then tell me the "+ "wifi password and when the release ships.")+ + ASSISTANT)+ gen_all = []+ cache, gen = r.generate(cache, q, 600)+ gen_all += gen+ out = {"session1": log, "snapshot_bytes": size, "heads": len(heads),+ "session2": r.decode(gen_all)}+ Path(args.out).write_text(json.dumps(out, indent=2, ensure_ascii=False))+ print(json.dumps(out, indent=2, ensure_ascii=False))+++def exact(args):+ """Teacher-forced: at every position the reference says the circuit predicts, the merged+ model's argmax is that token (store calls with random tokens, long text in between)."""+ from chipformer.q35 import bridge as B++ r = Runner(args.merged)+ v = S.Vocab(Path(args.merged) / "tokenizer.json")+ rng = random.Random(args.seed)+ sm = S.StoreMachine(v) # (the trace starts with the BIOS boot record)+ lo = r.meta["chip_token_offset"]++ def rnd(k):+ return [rng.randrange(S.SPECIAL) for _ in range(k)]++ def plain(k): # text that starts nothing (no digits, operators or tags)+ return [q for q in rnd(4 * k) if v.cls(q) == B.OTHER][:k]++ names = [rnd(2) for _ in range(5)]+ sm.qwen_text(plain(30))+ for i in range(args.calls):+ op = rng.choice(["put", "append", "get", "get", "list", "delete"])+ nm = rng.choice(names)+ if op in ("put", "append"):+ sm.call(op, nm, rnd(rng.choice([3, 20, 60, S.CMAX + 4])))+ elif op == "list":+ sm.call("list")+ else:+ sm.call(op, nm)+ sm.qwen_text(plain(rng.randint(5, args.gap)))+ from chipformer import spec++ # chip tokens -> their merged ids; text tokens -> their Qwen id+ ids = [S.qid_of(t) if S.qid_of(t) is not None else+ (t + lo if t < spec.VOCAB_SIZE else S.full_qid(t)) for t in sm.tokens]+ pred = []+ with torch.no_grad():+ from transformers import DynamicCache++ cache = DynamicCache(config=r.model.config)+ for i in range(0, len(ids), 1024):+ x = torch.tensor([ids[i:i + 1024]], device=r.dev)+ lg = r.model(input_ids=x, past_key_values=cache, use_cache=True).logits[0]+ lg[:, IM_START] = float("-inf")+ pred += lg.argmax(-1).tolist()+ bad = [p for p in range(len(ids) - 1) if sm.generated[p + 1] and pred[p] != ids[p + 1]]+ n = sum(sm.generated[1:])+ print(f"{len(ids)} tokens, {n} predicted by the circuit, {len(bad)} wrong", flush=True)+ Path(args.out).write_text(json.dumps({"tokens": len(ids), "checked": n, "wrong": len(bad),+ "first_wrong": bad[:20]}))+++def main():+ ap = argparse.ArgumentParser()+ sub = ap.add_subparsers(dest="cmd", required=True)+ a = sub.add_parser("recall")+ a.add_argument("merged")+ a.add_argument("base")+ a.add_argument("--n", type=int, default=50)+ a.add_argument("--pad", type=int, nargs="+", default=[10000, 100000])+ a.add_argument("--seed", type=int, default=0)+ a.add_argument("--ask", nargs="+", default=["plain", "tool"], choices=["plain", "tool"])+ a.add_argument("--merged-only", action="store_true")+ a.add_argument("--fresh", action="store_true",+ help="ask in a new session restored from the store's snapshot (implies "+ "--merged-only: base Qwen has nothing to recall from)")+ a.add_argument("--out", default="store_recall.json")+ b = sub.add_parser("session")+ b.add_argument("merged")+ b.add_argument("--out", default="store_session.json")+ e = sub.add_parser("exact")+ e.add_argument("merged")+ e.add_argument("--calls", type=int, default=40)+ e.add_argument("--gap", type=int, default=400)+ e.add_argument("--seed", type=int, default=0)+ e.add_argument("--out", default="store_exact.json")+ args = ap.parse_args()+ {"recall": recall, "session": session, "exact": exact}[args.cmd](args)+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/store_q35.py b/chipformer/q35/store_q35.pynew file mode 100644index 0000000..d4ffb23--- /dev/null+++ b/chipformer/q35/store_q35.py@@ -0,0 +1,592 @@+"""The object store (store.py) as part of the q35 circuit.++Built in two parts from ``chip8_q35.build``:++``capture`` token features of Qwen's rows (their id bits), what is inside a store tool call+ (the function, the name and text parameters), the name buffer, the content heads'+ writes (as Qwen writes the text), and ``S``: 1 from a store call's </tool_call> to+ the end of its answer. The rest of the circuit is gated off while S is 1.+``respond`` the answer: chip records (register and device-port writes) and the printed+ <tool_response>. Its cases join the circuit's output cases; prints of arbitrary+ tokens ("any-prints") go out through the id code instead of the class code.++All tables are Gated DeltaNet heads with one-hot keys (exact overwrites):++ content rows of 8 tokens: key head = 16 rows per chunk, its two value heads (Qwen3.5-4B)+ hold 4 tokens each, written token by token with conv taps (lags 0..3) so a row's+ last write carries its whole group; 4 tokens per row with one value head+ meta chunk -> (length, next, has next)+ dir entry -> (name tokens, name length, alive), dht: entry -> (first chunk, last chunk)++Every register and table address is a set of 2-bit one-hot groups: a key or query channel is+the sum of its groups' literals minus (n - 1), exactly 1 on its own row and <= 0 elsewhere.+"""++from fractions import Fraction++from chipformer import spec+from chipformer.dsl.graph import Expr+from chipformer.q35 import bridge as B+from chipformer.q35 import store as S+from chipformer.q35.ir import A_BIT, A_REF, REF_COLS, Chan, K+from chipformer.q35.prims import from_bits, onehot_bits++CODE_AMP = 128 # the id code in a row's output dims: CODE_AMP * (2 * bit - 1)+++def sel(c, e):+ """Key / query pre-activation K * (2e - 1) for a 0/1 (or <= 0 when false) form e."""+ if isinstance(e, Chan):+ return Chan(e.expr * (2 * K) - c.one * K, e.taps)+ return Chan(e * (2 * K) - c.one * K)+++def chan(e, scale):+ if isinstance(e, Chan):+ return Chan(e.expr * scale, e.taps)+ return Chan(e * scale)+++def thead(c, name, keys, queries, vheads, tiny=0.0):+ """A head of 0/1 tables: one dict per value head with data {column: 0/1 form or Chan},+ ref (1 where a written row carries the ballast), optional gate (0/1: its outputs), beta+ (write gate) and into (dims to add the outputs onto). Returns each value head's outputs."""+ specs = []+ for x in vheads:+ v = {col: chan(e, A_BIT) for col, e in x["data"].items()}+ for col in REF_COLS:+ v[col] = chan(x["ref"], A_REF)+ gate = c.one if x.get("gate") is None else x["gate"]+ cols = sorted(x["data"])+ specs.append({"v": v, "a": x.get("a", 0), "z": {col: K * gate for col in cols},+ "outs": [(col, 2.0**7) for col in cols], "beta": x.get("beta"),+ "into": x.get("into")})+ first, rest = specs[0], specs[1:]+ outs = c.head(name, {s: sel(c, e) for s, e in queries.items()},+ {s: sel(c, e) for s, e in keys.items()}, first["v"], first["a"], first["z"],+ first["outs"], tiny=tiny, beta=first["beta"], into=first["into"], vx=rest)+ res, i = [], 0+ for x in specs:+ n = len(x["outs"])+ res.append(outs[i:i + n])+ i += n+ return res+++def groups(c, bits, name):+ """2-bit one-hot groups of an integer's bits: groups[k][v] = [bits 2k..2k+1 == v]."""+ out = []+ for k in range(0, len(bits), 2):+ bb = bits[k:k + 2]+ oh = []+ for v in range(1 << len(bb)):+ lits = [b if v >> i & 1 else c.one - b for i, b in enumerate(bb)]+ oh.append(c.and_(lits, f"{name}.g{k // 2}=={v}"))+ out.append(oh)+ return out+++def addr_lits(gr, value):+ """The group literals of one address (groups of 2 bits, the last may have 1)."""+ nbits = 2 * len(gr) - (len(gr[-1]) == 2)+ assert value >> nbits == 0, (value, nbits)+ return [g[value >> 2 * k & (len(g) - 1)] for k, g in enumerate(gr)]+++def match_form(lits):+ """1 when all literals are 1, <= 0 otherwise (a key / query channel or AND-form)."""+ return sum(lits, Expr()) - (len(lits) - 1)+++class Store:+ """Handles between the two parts."""+++def capture(c, vocab: "S.Vocab", in_call, tco, tcc, n_chunks: int | None = None) -> Store:+ one = c.one+ st = Store()+ st.vocab = vocab+ r_v = c.target.value_heads // c.target.key_heads+ st.tpr = 8 if r_v >= 2 else 4 # tokens per content row+ st.n_chunks = n_chunks or (640 if r_v >= 2 else 256)+ f = c.token_feature+ qid = S.qid_of++ # --- features of Qwen's rows: the id bits, and the tokens the capture looks for+ def fq(name, pred):+ return f(name, lambda t: int(qid(t) is not None and pred(qid(t))), full=True)++ st.xb = [f(f"id.b{i}", lambda t, i=i: S.id_code(qid(t))[i], full=True)+ for i in range(S.ID_BITS)]+ st.yc = [f(f"id.code{i}", lambda t, i=i: 0 if qid(t) is None or qid(t) >= S.SPECIAL+ else CODE_AMP * (2 * S.id_code(qid(t))[i] - 1), full=True)+ for i in range(S.ID_BITS)]+ nl = fq("st.nl", vocab.has_nl)+ f_eq, f_fn = fq("st.eq", lambda q: q == S.Q_EQ), fq("st.function", lambda q: q == S.Q_FUNCTION)+ f_par = fq("st.parameter", lambda q: q == S.Q_PARAMETER)+ f_lts = fq("st.lt_slash", lambda q: q == S.Q_LTSLASH)+ op_bare = {op: fq(f"st.op.{op}", lambda q, i=i: q == i) for op, i in S.OP_BARE.items()}+ op_join = {op: fq(f"st.op={op}", lambda q, i=i: q == i) for op, i in S.OP_JOINED.items()}+ p_bare = {p: fq(f"st.p.{p}", lambda q, i=i: q == i) for p, i in S.PARAM_BARE.items()}+ p_join = {p: fq(f"st.p={p}", lambda q, i=i: q == i) for p, i in S.PARAM_JOINED.items()}+ cells = list(S.REGS.values()) + list(S.PORTS.values())+ st.at = {a: f(f"st.at@{a:x}", lambda t, a=a: int(t == spec.addr_token(a))) for a in cells}+ st_addr = sum(st.at.values(), Expr())++ # --- previous tokens (conv taps), one head+ T = c.text+ prev_cols = {2: Chan(f_eq, (0, 1, 0, 0)), 3: Chan(f_fn, (0, 1, 0, 0)),+ 4: Chan(f_fn, (0, 0, 1, 0)), 5: Chan(f_par, (0, 1, 0, 0)),+ 6: Chan(f_par, (0, 0, 1, 0)), 7: Chan(f_lts, (0, 1, 0, 0)),+ 8: Chan(T[B.TR_CLOSE], (0, 1, 0, 0))}+ for i, a in enumerate(cells):+ prev_cols[9 + i] = Chan(match_form(c.addr_lits(a)), (0, 1, 0, 0))+ outs = c.slot_head("st.prev", {0: one}, {0: one}, prev_cols, one)+ prev_eq, prev_fn, prev2_fn, prev_par, prev2_par, prev_lts, prev_trc = outs[:7]+ st.pv = dict(zip(cells, outs[7:])) # at a VAL token: its write's cell is a++ # --- inside a tool call: the function, the parameters and their values+ op_t = {}+ for op in S.OPS:+ terms = c.t_and([op_bare[op], prev_eq, prev2_fn, in_call])+ if op in op_join:+ terms += c.t_and([op_join[op], prev_fn, in_call])+ op_t[op] = c.node(terms, f"st.op_tok.{op}")+ op_any = sum(op_t.values(), Expr())+ pn, pt = (c.node(c.t_and([p_join[p], prev_par, in_call])+ + c.t_and([p_bare[p], prev_eq, prev2_par, in_call]), f"st.param.{p}")+ for p in ("name", "text"))+ closer = c.and_([f_par, prev_lts, in_call], "st.closer")+ ops = c.slot_head("st.op", {0: tco + op_any}, {0: one},+ {2 + i: op_t[op] for i, op in enumerate(S.OPS)}, tco + op_any)+ st.op = dict(zip(S.OPS, ops))+ pk_name, pk_text = c.slot_head("st.pk", {0: tco + pn + pt + closer}, {0: one},+ {2: pn, 3: pt}, tco + pn + pt + closer)+ x_nl = counter(c, "st.nlc", tco + pn + pt, nl)+ n1, n2 = (c.node(ge_terms(c, x_nl, tco + pn + pt, m), f"st.nlc>={m}") for m in (1, 2))++ def value(pk, name):+ return c.node(c.t_and([pk, n2, in_call]) + c.t_and([pk, n1, one - n2, one - nl, in_call]),+ name)++ st.val_name, st.val_text = value(pk_name, "st.val.name"), value(pk_text, "st.val.text")++ # name: up to NMAX tokens in a buffer (each in its own column block, all read at once), and+ # its length (counted with the closer's "</")+ nreset = tco + pn+ x_ni = counter(c, "st.ni", nreset, st.val_name)+ ni = [c.node(ge_terms(c, x_ni, nreset, n + 1) + neg(ge_terms(c, x_ni, nreset, n + 2)),+ f"st.ni=={n}") for n in range(S.NMAX)] # [this is name token n]+ st.nlen = count_bits(c, x_ni, nreset, S.NMAX + 2, 3, "st.nlen")+ data = {}+ for n in range(S.NMAX):+ for b in range(S.ID_BITS):+ data[2 + S.ID_BITS * n + b] = match_form([st.xb[b], ni[n]])+ keys = {n: match_form([st.val_name, ni[n]]) for n in range(S.NMAX)}+ keys[127] = nreset+ # rows of name tokens carry no ballast; the reset writes row 127 with it, so a read of all+ # rows has a constant norm. The reset also clears the buffer (decay 0).+ st.nbuf = thead(c, "st.nbuf", keys, {s: one for s in keys},+ [{"data": data, "ref": Chan(nreset), "a": 238 * nreset}])[0]+ nbuf_tok = [st.nbuf[S.ID_BITS * n:S.ID_BITS * (n + 1)] for n in range(S.NMAX)]+ st.nbuf_tok = nbuf_tok++ # text: index of each value token, its length+ treset = tco + pt+ x_ci = counter(c, "st.ci", treset, st.val_text)+ st.ibits = count_bits(c, x_ci, treset, S.CMAX + 1, 7, "st.i") # max(count - 1, 0)+ st.tlen = st.ibits # after the text: its length (the closer's "</" was counted)++ # --- registers (cells between the keys and the display; never written by a boot). The+ # ballast row is rewritten at every text token (and BOOT), so the store needs no BIOS.+ bal = c.is_boot + c.is_text+ keys, rdata, col, blocks = {127: Chan(bal)}, {}, 2, {}+ for slot, (rname, cell) in enumerate(S.REGS.items()):+ keys[slot] = c.write_key(cell)+ lits = c.addr_lits(cell)+ nb = S.REG_BITS[cell]+ blocks[rname] = list(range(col, col + nb))+ for i in range(nb):+ rdata[col] = Chan(c.vb[i] + sum(lits, Expr()) - one * Fraction(len(lits), 2),+ (1, 1, 0, 0))+ col += 1+ out = c.slot_head("st.regs", keys, {s: one for s in keys}, rdata, Chan(bal))+ by_col = dict(zip(sorted(rdata), out))+ st.reg = {rname: [by_col[k] for k in cols] for rname, cols in blocks.items()}++ # --- S: from a store call's </tool_call> to the end of its answer (the NL after its+ # </tool_response>)+ fin = c.and_([T[B.NL], prev_trc], "st.final_nl")+ op_set = sum(st.op.values(), Expr())+ store_tcc = c.and_([tcc, op_set], "st.call_end")+ (st.S,) = c.slot_head("st.S", {0: tcc + tco + fin}, {0: one}, {2: store_tcc},+ tcc + tco + fin)++ # --- the content heads: written at each text token, chunk NEXTG, index i < CMAX (the+ # index saturates beyond, so a longer text keeps its first CMAX tokens)+ content = c.node(c.t_and([st.val_text]) + neg(c.t_and([st.val_text], a=c.node(+ ge_terms(c, x_ci, treset, S.CMAX + 1), "st.ci>CMAX"))), "st.content")+ g_bits = st.reg["NEXTG"]+ gw = groups(c, g_bits, "st.nextg")+ tpr = st.tpr+ lo = tpr.bit_length() - 1 # index bits inside a row: 3 (8 per row) or 2+ iw = groups(c, st.ibits[lo:], "st.irow")+ slot_oh = [c.and_([b if s >> i & 1 else one - b for i, b in enumerate(st.ibits[:2])],+ f"st.islot=={s}") for s in range(4)]+ e = [[c.and_([st.xb[b], slot_oh[s], content], f"st.e{s}.{b}") for b in range(S.ID_BITS)]+ for s in range(4)]+ if tpr == 8:+ beta = [c.and_([content, one - st.ibits[2]], "st.beta.a"),+ c.and_([content, st.ibits[2]], "st.beta.b")]+ st.content_w = dict(content=content, gw=gw, iw=iw, e=e, beta=beta if tpr == 8 else None)+ return st+++def neg(terms):+ return [(-coef, a, b) for coef, a, b in terms]+++XS = 26 # counters come out as x = 2^(XS - n): large n stays a normal float in the residual and+# the thresholds' gate weights 2^(m + 2 - XS) * K / (norm scale ~2^-36) <= 2^124 stay inside bf16+# (m <= 106), while x^2 <= 2^52 vanishes next to the ballast's square (~2^84) of every RMSNorm+++def counter(c, name, reset, count, tiny=2.0**-118):+ """x = 2^(XS - n), n = tokens with count = 1 since the last reset (0 at the reset itself).++ The row written at a reset decays by half at each counted token; a ballast row is+ rewritten at every other token, so a read is normed by a constant. n <= 112 keeps the+ normed 2^-(n + 14) a normal float."""+ one = c.one+ q = {0: one - reset, 1: one - reset}+ k = {0: reset, 1: one - reset}+ v = {2: Chan(reset * A_BIT)}+ for col in REF_COLS:+ v[col] = Chan((one - reset) * A_REF)+ (x,) = c.head(name, {s: sel(c, e) for s, e in q.items()}, {s: sel(c, e) for s, e in k.items()},+ v, 110 * count + 238 * reset, {2: K * one}, [(2, 2.0**(7 + XS))], tiny=tiny)+ c.big.add(next(iter(x.terms))) # up to 2^XS: see emit.allocate+ return x+++def ge_terms(c, x, reset, m):+ """Unit terms of [n >= m] (m >= 1) from a counter's x = 2^(XS - n) (0 at reset tokens)."""+ one = c.one+ k = 8 * Fraction(2) ** (m - 1 - XS)+ return [(1, one - reset, 7 * one - k * x), (-1, one - reset, 6 * one - k * x)]+++def count_bits(c, x, reset, nmax, nbits, name):+ """Bits of max(min(n, nmax) - 1, 0), straight from x (no thermometer dims)."""+ out = []+ for b in range(nbits):+ terms = []+ for v in range(2, nmax + 1): # [n >= v] adds bit b of (v - 1) minus that of (v - 2)+ d = ((v - 1) >> b & 1) - ((v - 2) >> b & 1)+ if d:+ terms += [(d * co, a_, b_) for co, a_, b_ in ge_terms(c, x, reset, v)]+ out.append(c.node(terms, f"{name}.b{b}"))+ return out+++def inc_bits(c, bits, name):+ """Bits of (v + 1) mod 2^n."""+ one = c.one+ out = [one - bits[0]]+ for i in range(1, len(bits)):+ out.append(c.node(c.t_and([bits[i]]) + c.t_and(bits[:i]) + c.t_and([bits[i], *bits[:i]], -2),+ f"{name}.b{i}"))+ return out+++def respond(c, st: Store, case_list, cnt, jeq, tcc):+ """The answer's cases (appended to the circuit's), and the any-print outputs."""+ one = c.one+ T = c.text+ Sx = st.S+ op = st.op+ reg = st.reg+ pv, at = st.pv, st.at+ is_text, is_addr, is_val = c.is_text, c.is_addr, c.is_val+ trc = T[B.TR_CLOSE]++ def case(lits, kind, payload):+ case_list.append(([Sx, *lits], kind, payload))++ def cls_code(cls):+ from chipformer.q35.chip8_q35 import TEXT_CODE+ return TEXT_CODE + cls - B.TEXT_BASE++ # flags since the call's end: a GCUR write (the get's chain started), an SK write in a+ # list (its body started), a DIRW write (delete found its entry)+ def flag(name, at_tok, data=None):+ (x,) = c.slot_head(name, {0: tcc + at_tok}, {0: one}, {2: at_tok if data is None else data},+ tcc + at_tok)+ return x++ chain = flag("st.chain", pv[S.GCUR])+ lst = flag("st.listing", pv[S.SK], match_form([pv[S.SK], op["list"]])) # (a search writes SK too)+ wrote = flag("st.wrote", pv[S.DIRW])++ # H: text tokens since the call's end or the last store ADDR token+ st_addr = sum(at.values(), Expr())+ hreset = tcc + st_addr+ x_h = counter(c, "st.H", hreset, is_text - tcc)+ hbits = count_bits_n(c, x_h, hreset, S.CMAX + 2, 7, "st.H")+ hv = from_bits(hbits)++ def heq(v):+ return c.node(c.t_eq(hv, v, 0, 127), f"st.H=={v}")++ hs = {v: heq(v) for v in range(8)}++ # --- tables read during the answer+ sk_g = groups(c, reg["SK"], "st.sk")+ # dir: entry SK -> name, length, alive; written at DIRW's VAL token+ dir_data = {}+ for n in range(S.NMAX):+ for b in range(S.ID_BITS):+ dir_data[2 + S.ID_BITS * n + b] = st.nbuf_tok[n][b]+ for b in range(3):+ dir_data[2 + S.ID_BITS * S.NMAX + b] = st.nlen[b]+ dir_data[2 + S.ID_BITS * S.NMAX + 3] = c.vb[0]+ rows = range(S.NFILES_MAX + 1)+ dkeys = {k: match_form([pv[S.DIRW], *addr_lits(sk_g, k)]) for k in rows}+ dq = {k: match_form([Sx, *addr_lits(sk_g, k)]) for k in rows}+ d_out = thead(c, "st.dir", dkeys, dq, [{"data": dir_data, "ref": one}])[0]+ d_name = [d_out[S.ID_BITS * n:S.ID_BITS * (n + 1)] for n in range(S.NMAX)]+ d_len = d_out[S.ID_BITS * S.NMAX:S.ID_BITS * S.NMAX + 3]+ d_alive = d_out[S.ID_BITS * S.NMAX + 3]+ # dirH / dirT (first and last chunk of the entry)+ vb10 = c.vb[:10]+ if st.tpr == 8:+ hkeys = {k: match_form([pv[S.DIRH] + pv[S.DIRT], *addr_lits(sk_g, k)]) for k in rows}+ dh, dt = thead(c, "st.dht", hkeys, dq,+ [{"data": {2 + i: b for i, b in enumerate(vb10)}, "ref": one,+ "beta": pv[S.DIRH]},+ {"data": {2 + i: b for i, b in enumerate(vb10)}, "ref": one,+ "beta": pv[S.DIRT]}])+ else:+ dh = thead(c, "st.dirH", {k: match_form([pv[S.DIRH], *addr_lits(sk_g, k)]) for k in rows},+ dq, [{"data": {2 + i: b for i, b in enumerate(vb10)}, "ref": one}])[0]+ dt = thead(c, "st.dirT", {k: match_form([pv[S.DIRT], *addr_lits(sk_g, k)]) for k in rows},+ dq, [{"data": {2 + i: b for i, b in enumerate(vb10)}, "ref": one}])[0]++ # meta: chunk -> length, next, has next. Written at METAW (chunk NEXTG: length from the VAL+ # token) and METAN (chunk TMP: length TMPL, next from the VAL token); read at TMPL's ADDR+ # token (chunk TMP) and during a get's print (chunk GCUR)+ wm, wn = pv[S.METAW], pv[S.METAN]+ ws = [c.node(c.t_and([g_, one - wn]) + c.t_and([t_, wn]), f"st.meta.w.b{i}")+ for i, (g_, t_) in enumerate(zip(reg["NEXTG"], reg["TMP"]))]+ at_tmpl = at[S.TMPL]+ qsb = [c.node(c.t_and([g_, one - at_tmpl]) + c.t_and([t_, at_tmpl]), f"st.meta.q.b{i}")+ for i, (g_, t_) in enumerate(zip(reg["GCUR"], reg["TMP"]))]+ get_print = c.and_([Sx, chain, one - is_addr], "st.get_print")+ q_act = c.and_([Sx, chain], "st.in_chain") + at_tmpl # (the chain's ADDR(GCUR) reads next)+ wsg, qsg = groups(c, ws[:7], "st.meta.w"), groups(c, qsb[:7], "st.meta.q")+ whi, qhi = groups(c, ws[7:], "st.meta.wh"), groups(c, qsb[7:], "st.meta.qh")+ m_len = [c.node(c.t_and([c.vb[i], wm]) + c.t_and([reg["TMPL"][i], wn]), f"st.meta.len.b{i}")+ for i in range(7)]+ m_data = {2 + i: b for i, b in enumerate(m_len)}+ for i in range(10):+ m_data[9 + i] = c.and_([c.vb[i], wn], f"st.meta.next.b{i}")+ m_data[19] = wn+ m_rows = None+ n_meta = -(-st.n_chunks // 128)+ for h in range(n_meta):+ keys = {r: match_form([wm + wn, *addr_lits(wsg, r), *addr_lits(whi, h)]) for r in range(128)}+ qs = {r: match_form([q_act, *addr_lits(qsg, r), *addr_lits(qhi, h)]) for r in range(128)}+ o = thead(c, f"st.meta{h}", keys, qs, [{"data": m_data, "ref": one, "into": m_rows}])[0]+ m_rows = m_rows or o+ mr_len, mr_next, mr_has = m_rows[0:7], m_rows[7:17], m_rows[17]++ # content: written in capture's terms; read at chunk GCUR, index H+ cw = st.content_w+ tpr = st.tpr+ lo = tpr.bit_length() - 1+ rpc = S.CHUNK // tpr # rows per chunk+ cph = 128 // rpc # chunks per head+ n_heads = -(-st.n_chunks // cph)+ gr = groups(c, reg["GCUR"], "st.gcur")+ hr = groups(c, hbits[lo:], "st.hrow")+ hslot = [c.and_([b if s_ >> i & 1 else one - b for i, b in enumerate(hbits[:2])],+ f"st.hslot=={s_}") for s_ in range(4)]+ cdata = {2 + S.ID_BITS * s_ + b: Chan(cw["e"][s_][b], (1, 1, 1, 1))+ for s_ in range(4) for b in range(S.ID_BITS)}+ c_rows = None+ for h in range(n_heads):+ keys, qs = {}, {}+ for r in range(128):+ g = h * cph + r // rpc+ if g >= st.n_chunks:+ continue+ ir = r % rpc+ keys[r] = match_form([cw["content"], *addr_lits(cw["gw"], g), *addr_lits(cw["iw"], ir)])+ qs[r] = match_form([get_print, *addr_lits(gr, g), *addr_lits(hr, ir)])+ if tpr == 8: # the second value head (tokens 4..7 of a row) adds onto the same dims+ vh = [{"data": cdata, "ref": one, "beta": cw["beta"][0], "gate": one - hbits[2],+ "into": c_rows},+ {"data": cdata, "ref": one, "beta": cw["beta"][1], "gate": hbits[2],+ "into": c_rows or "main"}]+ o = thead(c, f"st.content{h}", keys, qs, vh)+ c_rows = c_rows or o[0]+ else:+ o = thead(c, f"st.content{h}", keys, qs, [{"data": cdata, "ref": one, "into": c_rows}])+ c_rows = c_rows or o[0]+ c_tok = [c_rows[S.ID_BITS * s_:S.ID_BITS * (s_ + 1)] for s_ in range(4)]++ # --- comparisons+ mism = []+ for n in range(S.NMAX):+ for b in range(S.ID_BITS):+ a_, b_ = d_name[n][b], st.nbuf_tok[n][b]+ mism += [(1, one, a_ - b_), (1, one, b_ - a_)]+ for b in range(3):+ mism += [(1, one, d_len[b] - st.nlen[b]), (1, one, st.nlen[b] - d_len[b])]+ n_mism = c.node(mism, "st.mismatch")+ match = c.node(c.t_eq(n_mism, 0, 0, 3 * S.NMAX * S.ID_BITS, lits=[d_alive]), "st.match")+ sk, nf = from_bits(reg["SK"]), from_bits(reg["NFILES"])+ end = c.node(c.t_eq(sk - nf, 0, -127, 127), "st.end")+ nf0 = c.node(c.t_eq(nf, 0, 0, 127), "st.nf==0")+ in_chunk = c.node(c.t_lt(hv - from_bits(mr_len), 0, -127, 127), "st.H<len")+ nlen_d = from_bits(d_len)+ h_lt_n = c.node(c.t_lt(hv, S.NMAX, 0, 127), "st.H<NMAX")+ # a list prints an entry's name: alive, H < its length, H < NMAX+ in_name = c.node(c.t_lt(hv - nlen_d, 0, -127, 127, lits=[d_alive, h_lt_n]), "st.name_tok")+ sk1, nf1, ng1 = (inc_bits(c, reg[r_], f"st.{r_}+1") for r_ in ("SK", "NFILES", "NEXTG"))++ # --- the answer+ p_put, p_app, p_get, p_lst, p_del = (op[o_] for o_ in S.OPS)+ putapp = p_put + p_app+ nonlist = one - p_lst+ NL, TRO = cls_code(B.NL), cls_code(B.TR_OPEN)+ TRC = cls_code(B.TR_CLOSE)+ # the call's end+ case([tcc, nonlist], 1, S.SK)+ case([tcc, p_lst], 2, NL)+ # ADDR(SK): 0 first (right after </tool_call>, or after the list's header), else SK + 1+ first = c.node(c.t_and([jeq(0), nonlist]) + c.t_and([jeq(3), p_lst]), "st.sk.first")+ case([at[S.SK], first], 0, 0)+ case([at[S.SK], one - first], 0, sk1)+ # the search: VAL(SK)+ sv = [pv[S.SK], nonlist]+ case([*sv, match, p_put + p_del], 1, S.DIRW)+ case([*sv, match, p_app], 1, S.TMP)+ case([*sv, match, p_get], 2, NL)+ case([*sv, one - match, end, putapp], 1, S.NFILES)+ case([*sv, one - match, end, p_get + p_del], 2, NL)+ case([*sv, one - match, one - end], 1, S.SK)+ # writes+ case([*c.addr_lits(S.NFILES)], 0, nf1)+ case([pv[S.NFILES]], 1, S.DIRW)+ case([*c.addr_lits(S.DIRW), putapp], 0, 1)+ case([*c.addr_lits(S.DIRW), p_del], 0, 0)+ case([pv[S.DIRW], putapp], 1, S.DIRH)+ case([pv[S.DIRW], p_del], 2, NL)+ case([*c.addr_lits(S.DIRH)], 0, reg["NEXTG"])+ case([pv[S.DIRH]], 1, S.DIRT)+ case([*c.addr_lits(S.DIRT)], 0, reg["NEXTG"])+ case([pv[S.DIRT]], 1, S.METAW)+ case([*c.addr_lits(S.METAW)], 0, st.tlen)+ case([pv[S.METAW]], 1, S.NEXTG)+ case([*c.addr_lits(S.NEXTG)], 0, ng1)+ case([pv[S.NEXTG]], 2, NL)+ case([*c.addr_lits(S.TMP)], 0, dt)+ case([pv[S.TMP]], 1, S.TMPL)+ case([*c.addr_lits(S.TMPL)], 0, mr_len)+ case([pv[S.TMPL]], 1, S.METAN)+ case([*c.addr_lits(S.METAN)], 0, reg["NEXTG"])+ case([pv[S.METAN]], 1, S.DIRT)+ case([*c.addr_lits(S.GCUR), one - chain], 0, dh)+ case([*c.addr_lits(S.GCUR), chain], 0, mr_next)+ # the header, then the body+ hdr = [is_text, one - chain, one - lst, one - trc]+ case([*hdr, hs[1]], 2, TRO)+ case([*hdr, hs[2]], 2, NL)+ m_ok = c.node(c.t_and([putapp]) + c.t_and([p_del, wrote]), "st.msg.ok")+ m_nf = c.node(c.t_and([p_del, one - wrote]) + c.t_and([p_get, one - match]), "st.msg.nf")+ m_em = c.and_([p_lst, nf0], "st.msg.empty")+ case([*hdr, hs[3], p_get, match], 1, S.GCUR)+ case([*hdr, hs[3], p_lst, one - nf0], 1, S.SK)+ anyp_cases = [] # (literals, 18 bits: ints or forms)++ def anyc(lits, bits):+ anyp_cases.append(([Sx, *lits], bits))++ def qbits(q):+ return S.id_code(q)++ msgs = [(m_ok, S.MSG["ok"]), (m_nf, S.MSG["not_found"]), (m_em, S.MSG["empty"])]+ for m_, toks in msgs:+ for i, q in enumerate(toks):+ anyc([*hdr, hs[3 + i], m_], qbits(q))+ case([*hdr, hs[3 + len(toks)], m_], 2, NL)+ case([*hdr, hs[4 + len(toks)], m_], 2, TRC)+ case([trc, is_text], 2, NL)+ # a get's chunks: content[GCUR][H] while H < length, then the next chunk or the end+ gp = [chain, one - is_addr, one - trc]+ for s_ in range(4):+ anyc([*gp, in_chunk, hslot[s_]], c_tok[s_])+ case([*gp, one - in_chunk, mr_has], 1, S.GCUR)+ case([*gp, one - in_chunk, one - mr_has], 2, TRC)+ # a list: after VAL(SK) and each printed name token+ lp = [p_lst, lst, one - is_addr, one - trc]+ case([*lp, end], 2, TRC)+ for n in range(S.NMAX):+ anyc([*lp, one - end, in_name, hs[n] if n < 8 else heq(n)], d_name[n])+ case([*lp, one - end, one - in_name], 1, S.SK)++ # any-prints: their condition and the 18 id bits of the token+ anyp = c.node([t for lits, _ in anyp_cases for t in c.t_and(lits)], "st.anyprint")+ ybits = []+ for b in range(S.ID_BITS):+ terms = []+ for lits, bits in anyp_cases:+ v = bits[b]+ if isinstance(v, int):+ if v:+ terms += c.t_and(lits)+ else:+ terms += c.t_and([*lits, v])+ ybits.append(c.node(terms, f"st.ybit{b}"))+ owner = c.t_and([is_text, Sx])+ st.debug = dict(match=match, end=end, H=hbits, chain=chain, lst=lst, wrote=wrote,+ in_chunk=in_chunk, in_name=in_name, mr_len=mr_len, d_alive=d_alive)+ return owner, anyp, ybits+++PERSISTENT = ("st.regs", "st.dir", "st.dht", "st.dirH", "st.dirT", "st.meta", "st.content")+++def is_persistent(name: str) -> bool:+ """Heads that hold the store's files and registers (a snapshot keeps exactly these)."""+ return any(name == p or (p in ("st.meta", "st.content") and name.startswith(p)+ and name[len(p):].isdigit()) for p in PERSISTENT)+++def persistent_heads(c) -> list[list[int]]:+ """[chip layer, value head] of every value head that holds store state, in the emitter's+ numbering (key head j of a layer owns value heads j * r .. j * r + r - 1)."""+ from chipformer.q35.ir import Head++ r = c.target.value_heads // c.target.key_heads+ out = []+ for layer in range(c.n_layers()):+ heads = [n for n in c.nodes if isinstance(n, Head) and n.level == 2 * layer + 1]+ for j, n in enumerate(heads):+ if is_persistent(n.name):+ out += [[layer, j * r + x] for x in range(len(n.vheads()))]+ return out+++def count_bits_n(c, x, reset, nmax, nbits, name):+ """Bits of min(n, nmax) from x = 2^-n."""+ out = []+ for b in range(nbits):+ terms = []+ for v in range(1, nmax + 1):+ d = (v >> b & 1) - ((v - 1) >> b & 1)+ if d:+ terms += [(d * co, a_, b_) for co, a_, b_ in ge_terms(c, x, reset, v)]+ out.append(c.node(terms, f"{name}.b{b}"))+ return outdiff --git a/chipformer/q35/store_snapshot.py b/chipformer/q35/store_snapshot.pynew file mode 100644index 0000000..94e03cc--- /dev/null+++ b/chipformer/q35/store_snapshot.py@@ -0,0 +1,115 @@+"""Durability for the object store: carry its files from one session to the next.++Two ways:++``snapshot`` / ``restore`` the store lives in Gated DeltaNet state of the chip layers, so a+ snapshot is just those heads' recurrent states (a few MB however much is stored). ``restore``+ returns a fresh stock ``DynamicCache`` in which only these heads are filled; prefill a new+ conversation with it (``model(..., past_key_values=cache)`` or ``generate``) and Qwen starts+ from an empty context while its computer still has the files.++``disk_image`` a token record that rebuilds the store when it is prefilled at the start of a+ session: one ``put`` (and ``append``s) per live file with the circuit's answers.++The store's persistent heads are listed in the checkpoint's chipformer.json+(``store.persistent_heads``: [chip layer, value head]).+"""++import json+from pathlib import Path++import torch++from chipformer.q35 import store as S+++def persistent(model_dir) -> list[list[int]]:+ meta = json.loads((Path(model_dir) / "chipformer.json").read_text())+ return meta["store"]["persistent_heads"]+++def snapshot(cache, heads: list[list[int]], row: int = 0) -> dict:+ """The store's state in ``cache`` (batch row ``row``): {"heads", "states"}."""+ states = [cache.layers[layer].recurrent_states[0][row, vh].detach().clone().cpu()+ for layer, vh in heads]+ return {"heads": heads, "states": torch.stack(states)}+++def restore(model, snap: dict, batch: int = 1):+ """A fresh DynamicCache holding only the snapshot's heads (every other state at 0)."""+ from transformers import DynamicCache++ cfg = model.config.get_text_config(decoder=True)+ cache = DynamicCache(config=model.config)+ dev = next(model.parameters()).device+ nk, nv = cfg.linear_num_key_heads, cfg.linear_num_value_heads+ dk, dv, kern = cfg.linear_key_head_dim, cfg.linear_value_head_dim, cfg.linear_conv_kernel_dim+ conv_dim = 2 * nk * dk + nv * dv+ by_layer: dict[int, list] = {}+ for (layer, vh), s in zip(snap["heads"], snap["states"]):+ by_layer.setdefault(layer, []).append((vh, s))+ for layer, items in by_layer.items():+ L = cache.layers[layer]+ rec = torch.zeros(batch, nv, dk, dv, dtype=items[0][1].dtype, device=dev)+ for vh, s in items:+ rec[:, vh] = s.to(dev)+ L.recurrent_states[0] = rec+ L.conv_states[0] = torch.zeros(batch, conv_dim, kern, dtype=model.dtype, device=dev)+ L.conv_kernel_size[0] = kern+ L.is_conv_states_initialized[0] = True+ L.is_recurrent_states_initialized[0] = True+ L.has_previous_state[0] = True+ L.dtype, L.device = model.dtype, dev+ return cache+++def save(snap: dict, path) -> None:+ torch.save(snap, path)+++def load(path) -> dict:+ return torch.load(path, weights_only=True)+++# --- the IR view of the same thing (tests) -----------------------------------------------+++def ir_snapshot(c, carry: dict) -> dict:+ """The persistent heads' states from an IR evaluation's ``carry`` (store_check)."""+ from chipformer.q35.store_q35 import is_persistent++ return {k: v for k, v in carry.items()+ if k[1] == "S" and is_persistent(c.nodes[k[0]].name)}+++# --- the disk image ----------------------------------------------------------------------+++def disk_image(sm: S.StoreMachine) -> list[tuple]:+ """The calls that rebuild ``sm``'s live files, in directory order: (op, name, text)."""+ calls = []+ for k in sorted(sm.dev.dir):+ e = sm.dev.dir[k]+ if not e.alive:+ continue+ name = list(e.name[:min(e.length, S.NMAX)])+ g, first = sm.dev.dir_h[k], True+ while True:+ n, nxt, has = sm.dev.meta.get(g, (0, 0, 0))+ body = [sm.dev.content.get((g, i), 0) for i in range(n)]+ # the stored chunk ends with the template's newline (its call adds it back)+ if body and body[-1] == S.Q_NL:+ body = body[:-1]+ calls.append(("put" if first else "append", name[:-1] if name and name[-1] == S.Q_NL+ else name, body))+ first = False+ if not has:+ break+ g = nxt+ return calls+++def replay(sm: S.StoreMachine, calls: list[tuple]) -> None:+ """Prefill a disk image: the calls and the circuit's answers become part of the trace."""+ for op, name, text in calls:+ sm.call(op, name, text)diff --git a/chipformer/q35/text_eval.py b/chipformer/q35/text_eval.pynew file mode 100644index 0000000..3047d4b--- /dev/null+++ b/chipformer/q35/text_eval.py@@ -0,0 +1,625 @@+"""Text benchmarks: is a merged Qwen3.5 + CHIP-8 checkpoint as capable as its base on pure text?++ python -m chipformer.q35.text_eval run MODEL --name NAME [--attn sdpa|eager] [--tokenizer BASE]+ python -m chipformer.q35.text_eval table [dist/evals/text] [--ref base]++``run`` evaluates one checkpoint and writes ``OUT/NAME/TASK.json`` (per-item records; a task whose+file exists is skipped, so a killed run resumes). ``table`` compares every run against ``--ref``.+All runs share one tokenizer (pass the base's) and identical batches, so items line up.++Tasks (all prompts in Qwen's chat template with thinking off, except the ARC/HellaSwag contexts):+ mmlu cais/mmlu, 1000 seeded test questions, 0-shot; prediction = argmax over A/B/C/D+ logits at the first answer token+ arc ARC-Challenge test (1172), continuation log-likelihood, acc and acc_norm+ hellaswag 1000 seeded validation items, continuation log-likelihood, acc_norm+ gsm8k 200 seeded test problems, greedy, no tools, final answer in \\boxed{}+ ifeval google/IFEval (541), greedy, scored by lm-eval's checker (strict and loose)+ niah needle (a 7-digit "magic number") in a wikitext-103 haystack at 8k/16k/32k tokens,+ chunked prefill; greedy answer and the teacher-forced log-prob of the needle+ ppl 8 disjoint 32k-token wikitext-103 windows: per-token NLL by position and greedy+ next-token agreement (a dense, long-context fidelity measure)++The noise floor is the base model under another attention implementation (eager vs sdpa), which+changes nothing but the rounding inside Qwen's full-attention layers.+"""++import argparse+import itertools+import json+import math+import random+import re+import sys+import time+from pathlib import Path++import numpy as np+import torch++TASKS = ("mmlu", "arc", "hellaswag", "gsm8k", "ifeval", "niah", "ppl")+TEXT_VOCAB = 248320 # Qwen's own vocabulary; the merged checkpoint's chip tokens come after it+TOKEN_BUDGET = 4096 # tokens per log-likelihood batch+GEN_BATCH = 16+CHUNK = 1024 # prefill chunk for the long-context check, same for every model+++# --- model ------------------------------------------------------------------------------------+++def load(path, attn, mem_gb):+ from transformers import AutoModelForImageTextToText++ if mem_gb:+ total = torch.cuda.get_device_properties(0).total_memory / 2**30+ torch.cuda.set_per_process_memory_fraction(min(1.0, mem_gb / total))+ model = AutoModelForImageTextToText.from_pretrained(+ path, dtype=torch.bfloat16, attn_implementation=attn)+ del model.model.visual # text only; frees ~0.7 GB+ return model.to("cuda").eval()+++def chat(tok, text):+ return tok.apply_chat_template([{"role": "user", "content": text}], add_generation_prompt=True,+ return_tensors="pt", enable_thinking=False)["input_ids"][0].tolist()+++def batches_by_budget(lengths, budget=TOKEN_BUDGET, max_b=64):+ """Indices sorted by length, grouped so that batch * max_len <= budget."""+ order = sorted(range(len(lengths)), key=lambda i: -lengths[i])+ out, cur = [], []+ for i in order:+ if cur and (len(cur) + 1) * lengths[cur[0]] > budget or len(cur) == max_b:+ out.append(cur)+ cur = []+ cur.append(i)+ return out + [cur] if cur else out++[email protected]_grad()+def target_logprobs(model, seqs, queries, pad):+ """For each sequence, log-probs (full vocabulary) of chosen tokens at chosen positions.++ ``queries[i]`` is a list of (position, [token ids]); returns, per sequence, a list of+ (logprobs of those ids, argmax id) per query. Right padding, so real positions are exact.+ """+ out = [None] * len(seqs)+ for b in batches_by_budget([len(s) for s in seqs]):+ n = max(len(seqs[i]) for i in b)+ ids = torch.full((len(b), n), pad, dtype=torch.long)+ mask = torch.zeros((len(b), n), dtype=torch.long)+ for r, i in enumerate(b):+ ids[r, :len(seqs[i])] = torch.tensor(seqs[i])+ mask[r, :len(seqs[i])] = 1+ h = model.model(input_ids=ids.cuda(), attention_mask=mask.cuda(),+ use_cache=False).last_hidden_state+ rows = [(r, p, toks) for r, i in enumerate(b) for p, toks in queries[i]]+ sel = h[torch.tensor([r for r, _, _ in rows]), torch.tensor([p for _, p, _ in rows])]+ vals, tops = [], []+ for c in range(0, len(rows), 256): # bounded memory for the vocabulary-wide softmax+ lp = torch.log_softmax(model.lm_head(sel[c:c + 256]).float(), -1)+ tops += lp.argmax(-1).tolist()+ rr = [k - c for k in range(c, min(c + 256, len(rows))) for _ in rows[k][2]]+ tt = [t for k in range(c, min(c + 256, len(rows))) for t in rows[k][2]]+ vals += lp[rr, tt].tolist()+ k = v = 0+ for i in b:+ res = []+ for _, toks in queries[i]:+ res.append((vals[v:v + len(toks)], tops[k]))+ k, v = k + 1, v + len(toks)+ out[i] = res+ return out++[email protected]_grad()+def generate(model, prompts, max_new, eos, pad, batch=GEN_BATCH):+ """Greedy, batched with left padding; batches of similar prompt length, same for every+ model. Returns the new tokens of each prompt, cut after the first EOS."""+ order = sorted(range(len(prompts)), key=lambda i: -len(prompts[i]))+ out = [None] * len(prompts)+ for s in range(0, len(order), batch):+ b = order[s:s + batch]+ n = max(len(prompts[i]) for i in b)+ ids = torch.full((len(b), n), pad, dtype=torch.long)+ mask = torch.zeros((len(b), n), dtype=torch.long)+ for r, i in enumerate(b):+ ids[r, n - len(prompts[i]):] = torch.tensor(prompts[i])+ mask[r, n - len(prompts[i]):] = 1+ model.model.rope_deltas = None+ g = model.generate(input_ids=ids.cuda(), attention_mask=mask.cuda(), max_new_tokens=max_new,+ do_sample=False, temperature=None, top_p=None, top_k=None,+ eos_token_id=eos, pad_token_id=pad)[:, n:].tolist()+ for r, i in enumerate(b):+ toks = g[r]+ cut = next((j for j, t in enumerate(toks) if t in eos), None)+ out[i] = toks if cut is None else toks[:cut + 1]+ print(f" gen {s + len(b)}/{len(prompts)}", flush=True)+ return out+++def gen_record(tok, toks, eos):+ text = tok.decode([t for t in toks if t < TEXT_VOCAB], skip_special_tokens=True)+ return {"text": text, "tokens": toks, "n": len(toks), "finished": bool(toks) and toks[-1] in eos,+ "chip_tokens": sum(t >= TEXT_VOCAB for t in toks)}+++# --- tasks ------------------------------------------------------------------------------------+++def task_mmlu(model, tok, ctx, n=1000, seed=0):+ from datasets import load_dataset++ ds = load_dataset("cais/mmlu", "all", split="test")+ idx = sorted(random.Random(seed).sample(range(len(ds)), n))+ letters = [tok.encode(c, add_special_tokens=False) for c in "ABCD"]+ assert all(len(x) == 1 for x in letters)+ letters = [x[0] for x in letters]+ seqs, items = [], []+ for i in idx:+ d = ds[i]+ q = (f"The following is a multiple choice question about {d['subject'].replace('_', ' ')}."+ f"\n\n{d['question'].strip()}\n"+ + "".join(f"{c}. {t}\n" for c, t in zip("ABCD", d["choices"]))+ + "\nAnswer with the letter of the correct option only.")+ seqs.append(chat(tok, q))+ items.append({"idx": i, "subject": d["subject"], "gold": d["answer"]})+ res = target_logprobs(model, seqs, [[(len(s) - 1, letters)] for s in seqs], ctx["pad"])+ for it, ((lp, top),) in zip(items, res):+ it.update(lp=lp, pred=int(np.argmax(lp)), top_is_letter=top in letters)+ it["correct"] = it["pred"] == it["gold"]+ return items+++def choice_items(model, tok, ctx, docs):+ """docs: (context string, [continuation strings], gold). Continuation log-likelihoods."""+ seqs, queries, meta = [], [], []+ for c, conts, gold in docs:+ cids = tok.encode(c, add_special_tokens=False)+ meta.append((gold, [len(x) for x in conts]))+ for x in conts:+ xids = tok.encode(x, add_special_tokens=False)+ seqs.append(cids + xids)+ queries.append([(len(cids) - 1 + j, [t]) for j, t in enumerate(xids)])+ res = target_logprobs(model, seqs, queries, ctx["pad"])+ items, k = [], 0+ for gold, chars in meta:+ lp, greedy = [], []+ for _ in chars:+ lp.append(sum(r[0][0] for r in res[k]))+ greedy.append(all(r[1] == q[1][0] for r, q in zip(res[k], queries[k])))+ k += 1+ norm = [v / n for v, n in zip(lp, chars)]+ items.append({"gold": gold, "lp": lp, "pred": int(np.argmax(lp)),+ "pred_norm": int(np.argmax(norm)), "greedy": greedy})+ items[-1]["correct"] = items[-1]["pred"] == gold+ items[-1]["correct_norm"] = items[-1]["pred_norm"] == gold+ return items+++def task_arc(model, tok, ctx):+ from datasets import load_dataset++ ds = load_dataset("allenai/ai2_arc", "ARC-Challenge", split="test")+ docs = [(f"Question: {d['question']}\nAnswer:", [" " + t for t in d["choices"]["text"]],+ d["choices"]["label"].index(d["answerKey"])) for d in ds]+ items = choice_items(model, tok, ctx, docs)+ for d, it in zip(ds, items):+ it["id"] = d["id"]+ return items+++def _hs(text):+ text = text.strip().replace(" [title]", ". ")+ return re.sub(r"\[.*?\]", "", text).replace(" ", " ")+++def task_hellaswag(model, tok, ctx, n=1000, seed=0):+ from datasets import load_dataset++ ds = load_dataset("Rowan/hellaswag", split="validation")+ idx = sorted(random.Random(seed).sample(range(len(ds)), n))+ docs = []+ for i in idx:+ d = ds[i]+ c = _hs(d["activity_label"] + ": " + d["ctx_a"] + " " + d["ctx_b"].capitalize())+ docs.append((c, [" " + _hs(e) for e in d["endings"]], int(d["label"])))+ items = choice_items(model, tok, ctx, docs)+ for i, it in zip(idx, items):+ it["idx"] = i+ return items+++def gsm_answer(text):+ boxed = re.findall(r"\\boxed\{((?:[^{}]|\{[^{}]*\})*)\}", text)+ s = boxed[-1] if boxed else text+ nums = re.findall(r"-?\d[\d,]*(?:\.\d+)?", s.replace("$", ""))+ return nums[-1].replace(",", "").rstrip(".") if nums else None+++def same_number(a, b):+ try:+ return a is not None and abs(float(a) - float(b)) < 1e-6+ except ValueError:+ return False+++def task_gsm8k(model, tok, ctx, n=200, seed=0, max_new=1024):+ from datasets import load_dataset++ ds = load_dataset("openai/gsm8k", "main", split="test")+ idx = sorted(random.Random(seed).sample(range(len(ds)), n))+ prompts = [chat(tok, ds[i]["question"] + "\nPlease reason step by step, and put your final "+ "answer within \\boxed{}.") for i in idx]+ gens = generate(model, prompts, max_new, ctx["eos"], ctx["pad"])+ items = []+ for i, g in zip(idx, gens):+ gold = ds[i]["answer"].split("####")[-1].strip().replace(",", "")+ r = gen_record(tok, g, ctx["eos"])+ r.update(idx=i, gold=gold, pred=gsm_answer(r["text"]))+ r["correct"] = same_number(r["pred"], gold)+ items.append(r)+ return items+++def ifeval_utils():+ """lm-eval's IFEval checker, imported without lm_eval/__init__ (which pulls in the whole+ harness). Needs langdetect, immutabledict, nltk (punkt_tab)."""+ import importlib.util+ import types++ root = Path(importlib.util.find_spec("lm_eval").submodule_search_locations[0])+ for name, p in (("lm_eval", root), ("lm_eval.tasks", root / "tasks")):+ if name not in sys.modules:+ m = types.ModuleType(name)+ m.__path__ = [str(p)]+ sys.modules[name] = m+ from lm_eval.tasks.ifeval import utils++ return utils+++def task_ifeval(model, tok, ctx, max_new=1280):+ from datasets import load_dataset++ utils = ifeval_utils()+ ds = load_dataset("google/IFEval", split="train")+ gens = generate(model, [chat(tok, d["prompt"]) for d in ds], max_new, ctx["eos"], ctx["pad"])+ items = []+ for d, g in zip(ds, gens):+ r = gen_record(tok, g, ctx["eos"])+ s = utils.process_results(d, [r["text"]])+ r.update(key=d["key"], strict=s["prompt_level_strict_acc"],+ loose=s["prompt_level_loose_acc"], inst_strict=s["inst_level_strict_acc"],+ inst_loose=s["inst_level_loose_acc"])+ r["correct"] = r["strict"]+ items.append(r)+ return items+++CITIES = ["Lisbon", "Nairobi", "Osaka", "Quito", "Tbilisi", "Winnipeg", "Hanoi", "Bergen",+ "Cusco", "Dakar", "Perth", "Tallinn"]+++def haystack_ids(tok):+ from datasets import load_dataset++ ds = load_dataset("Salesforce/wikitext", "wikitext-103-raw-v1", split="test")+ text = "".join(t for t in ds["text"] if t.strip() and not t.strip().startswith("="))+ return tok.encode(text, add_special_tokens=False)++[email protected]_grad()+def prefill(model, ids, keep=1):+ """Chunked prefill with a fresh cache; logits (text vocab) of the last ``keep`` positions."""+ from transformers import DynamicCache++ model.model.rope_deltas = None # stale after a batched generate(); text-only deltas are 0+ cache = DynamicCache(config=model.config)+ n = len(ids) - keep+ cuts = list(range(0, n, CHUNK)) + [n, len(ids)]+ for a, b in itertools.pairwise(cuts):+ x = torch.tensor([ids[a:b]], device="cuda")+ out = model(input_ids=x, past_key_values=cache, use_cache=True, logits_to_keep=keep)+ return out.logits[0, :, :TEXT_VOCAB].float(), cache++[email protected]_grad()+def greedy_from(model, logits, cache, n, eos):+ toks = []+ t = int(logits[-1].argmax())+ for _ in range(n):+ toks.append(t)+ if t in eos:+ break+ out = model(input_ids=torch.tensor([[t]], device="cuda"), past_key_values=cache,+ use_cache=True)+ t = int(out.logits[0, -1].argmax()) # full vocabulary: a chip token would show up+ return toks+++def task_niah(model, tok, ctx, lengths=(8192, 16384, 32768), depths=(0.1, 0.3, 0.5, 0.7, 0.9),+ seeds=4):+ hay = haystack_ids(tok)+ items = []+ for L in lengths:+ for depth in depths:+ for s in range(seeds):+ rng = random.Random(L * 100 + int(depth * 10) * 10 + s)+ city, num = rng.choice(CITIES), str(rng.randrange(10**6, 10**7))+ question = (f"What is the special magic number for {city} mentioned in the text? "+ "Answer with the number only.")+ head = "Read the text below and answer the question that follows.\n\n"+ over = len(chat(tok, head + "\n\n" + question)) + 20+ off = rng.randrange(0, len(hay) - L)+ body = hay[off:off + L - over]+ p = int(depth * len(body))+ left, right = tok.decode(body[:p]), tok.decode(body[p:])+ cut = left.rfind(". ")+ if cut > len(left) - 2000: # put the needle at a sentence boundary+ left, right = left[:cut + 2], left[cut + 2:] + right+ needle = f"The special magic number for {city} is {num}. "+ ids = chat(tok, head + left + needle + right + "\n\n" + question)+ ans = tok.encode(num, add_special_tokens=False)+ logits, _ = prefill(model, ids + ans, keep=len(ans) + 1)+ lp = torch.log_softmax(logits, -1)+ gold_lp = float(sum(lp[j, t] for j, t in enumerate(ans)))+ logits, cache = prefill(model, ids)+ g = greedy_from(model, logits, cache, 16, ctx["eos"])+ del cache+ r = gen_record(tok, g, ctx["eos"])+ r.update(length=len(ids), target=L, depth=depth, seed=s, city=city, gold=num,+ gold_logprob=gold_lp, correct=num in r["text"])+ items.append(r)+ print(f" niah L={len(ids)} d={depth} s={s} ok={r['correct']} "+ f"lp={gold_lp:.3f} {r['text']!r}", flush=True)+ return items++[email protected]_grad()+def task_ppl(model, tok, ctx, length=32768, docs=8):+ """Long-context language modelling: per-token NLL and greedy next token over ``docs``+ disjoint ``length``-token windows of wikitext-103 test, chunked prefill."""+ from transformers import DynamicCache++ hay = haystack_ids(tok)+ items = []+ for d in range(docs):+ ids = hay[d * length:(d + 1) * length]+ model.model.rope_deltas = None+ cache = DynamicCache(config=model.config)+ nll, top = [], []+ for a in range(0, length, CHUNK):+ x = torch.tensor([ids[a:a + CHUNK]], device="cuda")+ h = model.model(input_ids=x, past_key_values=cache, use_cache=True).last_hidden_state[0]+ for c in range(0, h.shape[0], 256):+ lp = torch.log_softmax(model.lm_head(h[c:c + 256]).float(), -1)+ p0 = a + c+ tgt = torch.tensor(ids[p0 + 1:p0 + 1 + lp.shape[0]], device="cuda")+ nll += (-lp[torch.arange(len(tgt)), tgt]).tolist()+ top += lp.argmax(-1)[:len(tgt)].tolist()+ del cache+ items.append({"offset": d * length, "nll": [round(v, 5) for v in nll], "top1": top,+ "target": ids[1:], "correct": float(np.mean(np.array(top) == ids[1:]))})+ print(f" ppl doc {d}: ppl {math.exp(np.mean(nll)):.3f} top1 {items[-1]['correct']:.4f}",+ flush=True)+ return items+++PPL_BUCKETS = ((0, 2048), (2048, 8192), (8192, 16384), (16384, 32768))+++def ppl_table(data, names, ref, summary):+ """Rows for the long-context perplexity task: PPL per position bucket, signed mean dNLL vs+ ``ref`` with a bootstrap CI over 1024-token blocks, |dNLL|, greedy next-token agreement."""+ lines = []+ for lo, hi in PPL_BUCKETS:+ cells = []+ for n in names:+ if "ppl" not in data[n]:+ cells.append("-")+ continue+ v = np.concatenate([np.array(it["nll"][lo:hi]) for it in data[n]["ppl"]])+ s = {"ppl": float(np.exp(v.mean())), "tokens": len(v)}+ cell = f"{s['ppl']:.3f}"+ if n != ref:+ r = [np.array(it["nll"][lo:hi]) for it in data[ref]["ppl"]]+ m = [np.array(it["nll"][lo:hi]) for it in data[n]["ppl"]]+ blocks = np.array([(y[k:k + 1024] - x[k:k + 1024]).mean() for x, y in zip(r, m)+ for k in range(0, len(x), 1024)])+ boot = np.random.default_rng(0).choice(blocks, (10000, len(blocks))).mean(1)+ dl, dh = np.percentile(boot, [2.5, 97.5])+ d = np.concatenate([y - x for x, y in zip(r, m)])+ agree = np.concatenate([np.array(a["top1"][lo:hi]) == np.array(b["top1"][lo:hi])+ for a, b in zip(data[ref]["ppl"], data[n]["ppl"])])+ s.update(dnll=float(d.mean()), dnll_ci95=[float(dl), float(dh)],+ abs_dnll=float(np.abs(d).mean()), top1_agreement=float(agree.mean()))+ cell += (f"<br>dNLL {d.mean():+.2e} [{dl:+.1e}, {dh:+.1e}], mean abs dNLL "+ f"{np.abs(d).mean():.2e}, top-1 agree {100 * agree.mean():.2f}%")+ summary.setdefault(n, {}).setdefault("ppl", {})[f"{lo}-{hi}"] = s+ cells.append(cell)+ lines.append(f"| ppl | positions {lo}-{hi} | " + " | ".join(cells) + " |")+ return lines+++def stop_tokens(tok):+ """End of turn and end of text stop generation (as in the merged generation_config.json);+ the base snapshot ships no generation config."""+ eos = [tok.convert_tokens_to_ids("<|im_end|>"), tok.convert_tokens_to_ids("<|endoftext|>")]+ return {"eos": eos, "pad": eos[1]}+++# --- run --------------------------------------------------------------------------------------+++def run(args):+ from transformers import AutoTokenizer++ tok = AutoTokenizer.from_pretrained(args.tokenizer or args.model)+ ctx = stop_tokens(tok)+ out = Path(args.out) / args.name+ out.mkdir(parents=True, exist_ok=True)+ model = None+ for task in args.tasks.split(","):+ f = out / f"{task}.json"+ if f.exists():+ print(f"{args.name}/{task}: done already", flush=True)+ continue+ if model is None:+ model = load(args.model, args.attn, args.mem_gb)+ print(f"{args.name}/{task} ...", flush=True)+ t = time.time()+ torch.cuda.reset_peak_memory_stats()+ items = globals()[f"task_{task}"](model, tok, ctx)+ rec = {"model": args.model, "attn": args.attn, "task": task, "seconds": time.time() - t,+ "peak_gb": torch.cuda.max_memory_allocated() / 2**30, "items": items}+ f.write_text(json.dumps(rec, ensure_ascii=False))+ acc = np.mean([it["correct"] for it in items])+ print(f"{args.name}/{task}: acc {acc:.4f} n={len(items)} {rec['seconds']:.0f}s "+ f"peak {rec['peak_gb']:.1f} GB", flush=True)+++# --- statistics -------------------------------------------------------------------------------+++def wilson(k, n, z=1.96):+ if n == 0:+ return (float("nan"),) * 2+ p = k / n+ c = (p + z * z / (2 * n)) / (1 + z * z / n)+ h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)+ return c - h, c + h+++def paired_diff(a, b, n_boot=10000, seed=0):+ """Mean of a - b with a paired bootstrap 95% CI, and the exact McNemar p-value."""+ a, b = np.asarray(a, float), np.asarray(b, float)+ rng = np.random.default_rng(seed)+ idx = rng.integers(0, len(a), (n_boot, len(a)))+ d = (a[idx] - b[idx]).mean(1)+ n01, n10 = int(((a == 1) & (b == 0)).sum()), int(((a == 0) & (b == 1)).sum())+ m = n01 + n10+ p = min(1.0, 2 * sum(math.comb(m, k) for k in range(min(n01, n10) + 1)) / 2**m) if m else 1.0+ lo, hi = np.percentile(d, [2.5, 97.5])+ return float((a - b).mean()), float(lo), float(hi), p+++def choice_kl(p_lp, q_lp):+ """KL(p || q) between the normalised choice distributions of two runs, per item."""+ p = torch.log_softmax(torch.tensor(p_lp, dtype=torch.float64), -1)+ q = torch.log_softmax(torch.tensor(q_lp, dtype=torch.float64), -1)+ return float((p.exp() * (p - q)).sum())+++def first_divergence(x, y):+ return next((i for i, (p, q) in enumerate(zip(x, y)) if p != q),+ None if len(x) == len(y) else min(len(x), len(y)))+++METRICS = { # task -> (metric name, item key) list; the first is the headline+ "mmlu": [("acc", "correct")],+ "arc": [("acc_norm", "correct_norm"), ("acc", "correct")],+ "hellaswag": [("acc_norm", "correct_norm"), ("acc", "correct")],+ "gsm8k": [("acc", "correct")],+ "ifeval": [("prompt_strict", "strict"), ("prompt_loose", "loose")],+ "niah": [("acc", "correct")],+}+++def compare_items(task, ref, run):+ """Fine-grained agreement of ``run`` with ``ref`` beyond accuracy."""+ out = {}+ if task in ("mmlu", "arc", "hellaswag"):+ out["same_pred"] = [a["pred"] == b["pred"] for a, b in zip(ref, run)]+ out["mean_choice_kl"] = float(np.mean([choice_kl(a["lp"], b["lp"])+ for a, b in zip(ref, run)]))+ out["mean_abs_dlogprob"] = float(np.mean([abs(x - y) for a, b in zip(ref, run)+ for x, y in zip(a["lp"], b["lp"])]))+ elif task == "niah":+ out["same_pred"] = [a["correct"] == b["correct"] for a, b in zip(ref, run)]+ out["mean_abs_dlogprob"] = float(np.mean([abs(a["gold_logprob"] - b["gold_logprob"])+ for a, b in zip(ref, run)]))+ out["identical_text"] = float(np.mean([a["tokens"] == b["tokens"] for a, b in zip(ref, run)]))+ else:+ key = "pred" if task == "gsm8k" else "strict"+ out["same_pred"] = [a[key] == b[key] for a, b in zip(ref, run)]+ out["identical_text"] = float(np.mean([a["tokens"] == b["tokens"] for a, b in zip(ref, run)]))+ fd = [first_divergence(a["tokens"], b["tokens"]) for a, b in zip(ref, run)]+ fd = [x for x in fd if x is not None]+ out["median_first_divergence"] = float(np.median(fd)) if fd else None+ out["unfinished"] = int(sum(not b["finished"] for b in run))+ out["chip_tokens"] = int(sum(b["chip_tokens"] for b in run))+ return out+++def table(args):+ root = Path(args.dir)+ names = [args.ref] + [p.name for p in sorted(root.iterdir())+ if p.is_dir() and p.name != args.ref]+ data = {n: {f.stem: json.loads(f.read_text(encoding="utf-8"))["items"]+ for f in (root / n).glob("*.json")} for n in names}+ summary, lines = {}, []+ lines.append("| task | metric | " + " | ".join(names) + " |")+ lines.append("|---|---|" + "---|" * len(names))+ for task in TASKS:+ if task not in data[args.ref] or task == "ppl":+ continue+ ref = data[args.ref][task]+ for metric, key in METRICS[task]:+ cells = []+ for n in names:+ if task not in data[n]:+ cells.append("-")+ continue+ v = [bool(it[key]) for it in data[n][task]]+ lo, hi = wilson(sum(v), len(v))+ s = {"acc": float(np.mean(v)), "ci95": [lo, hi], "n": len(v)}+ cell = f"{100 * s['acc']:.1f} [{100 * lo:.1f}, {100 * hi:.1f}]"+ if n != args.ref:+ d, dlo, dhi, p = paired_diff(v, [bool(it[key]) for it in ref])+ s.update(diff=d, diff_ci95=[dlo, dhi], mcnemar_p=p)+ cell += f"<br>diff {100 * d:+.1f} [{100 * dlo:+.1f}, {100 * dhi:+.1f}] p={p:.2f}"+ summary.setdefault(n, {}).setdefault(task, {})[metric] = s+ cells.append(cell)+ lines.append(f"| {task} | {metric} (n={len(ref)}) | " + " | ".join(cells) + " |")+ cells = []+ for n in names:+ if n == args.ref or task not in data[n]:+ cells.append("ref" if n == args.ref else "-")+ continue+ c = compare_items(task, ref, data[n][task])+ same = c.pop("same_pred")+ lo, hi = wilson(sum(same), len(same))+ c["agreement"] = {"frac": float(np.mean(same)), "ci95": [lo, hi]}+ summary[n][task]["vs_ref"] = c+ extra = ", ".join(f"{k} {v:.3g}" if isinstance(v, float) else f"{k} {v}"+ for k, v in c.items() if k != "agreement")+ cells.append(f"{100 * np.mean(same):.1f}% [{100 * lo:.1f}, {100 * hi:.1f}]<br>{extra}")+ lines.append(f"| {task} | agreement with {args.ref} | " + " | ".join(cells) + " |")+ if "ppl" in data[args.ref]:+ lines += ppl_table(data, names, args.ref, summary)+ md = "\n".join(lines)+ print(md)+ (root / "summary.json").write_text(json.dumps(summary, indent=2))+ (root / "table.md").write_text(md + "\n", encoding="utf-8")+++def main() -> None:+ ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])+ sub = ap.add_subparsers(dest="cmd", required=True)+ r = sub.add_parser("run")+ r.add_argument("model")+ r.add_argument("--name", required=True)+ r.add_argument("--attn", default="sdpa")+ r.add_argument("--tokenizer", default=None, help="shared tokenizer (the base checkpoint)")+ r.add_argument("--tasks", default=",".join(TASKS))+ r.add_argument("--out", default="dist/evals/text")+ r.add_argument("--mem-gb", type=float, default=20.0, help="per-process GPU memory cap")+ t = sub.add_parser("table")+ t.add_argument("dir", nargs="?", default="dist/evals/text")+ t.add_argument("--ref", default="base")+ args = ap.parse_args()+ run(args) if args.cmd == "run" else table(args)+++if __name__ == "__main__":+ main()diff --git a/chipformer/q35/textonly.py b/chipformer/q35/textonly.pynew file mode 100644index 0000000..dfbc003--- /dev/null+++ b/chipformer/q35/textonly.py@@ -0,0 +1,59 @@+"""Export a merged checkpoint as a text-only stock ``Qwen3_5ForCausalLM`` (no vision tower).++Qwen3.5 ships as ``Qwen3_5ForConditionalGeneration`` (text plus a vision tower); the computer+lives in the text model, so a text-only export drops the vision weights (0.34B params at 4B)+and loads with ``AutoModelForCausalLM`` / ``Qwen3_5ForCausalLM``. Text in, text out is+unchanged: the language model's weights are copied as they are.++ python -m chipformer.q35.textonly MERGED OUT+"""++import json+import shutil+import sys+from pathlib import Path++PREFIX = "model.language_model."+++def export(src, dst) -> None:+ from safetensors import safe_open+ from safetensors.torch import save_file++ src, dst = Path(src), Path(dst)+ dst.mkdir(parents=True, exist_ok=True)+ cfg = json.loads((src / "config.json").read_text())+ tc = dict(cfg["text_config"])+ tc["architectures"] = ["Qwen3_5ForCausalLM"]+ tc["model_type"] = "qwen3_5_text"+ tc["tie_word_embeddings"] = cfg.get("tie_word_embeddings", tc.get("tie_word_embeddings", True))+ for k in ("torch_dtype", "dtype", "transformers_version"):+ if k in cfg and k not in tc:+ tc[k] = cfg[k]+ (dst / "config.json").write_text(json.dumps(tc, indent=2))+ index = json.loads((src / "model.safetensors.index.json").read_text())["weight_map"]+ keep = {k: f for k, f in index.items() if k.startswith(PREFIX)}+ files = sorted(set(keep.values()))+ weight_map, total = {}, 0+ for i, f in enumerate(files):+ out = {}+ with safe_open(src / f, "pt") as s:+ for k in s.keys(): # noqa: SIM118 - a safetensors handle, not a dict+ if k in keep:+ out["model." + k[len(PREFIX):]] = s.get_tensor(k)+ name = f"model-{i + 1:05d}-of-{len(files):05d}.safetensors"+ save_file(out, str(dst / name), metadata={"format": "pt"})+ for k, t in out.items():+ weight_map[k] = name+ total += t.numel() * t.element_size()+ (dst / "model.safetensors.index.json").write_text(+ json.dumps({"metadata": {"total_size": total}, "weight_map": weight_map}, indent=2))+ for p in src.iterdir(): # tokenizer, generation config, chat template, chipformer.json+ if p.is_file() and not p.name.endswith(".safetensors") and p.name not in (+ "config.json", "model.safetensors.index.json", "preprocessor_config.json",+ "video_preprocessor_config.json"):+ shutil.copy(p, dst / p.name)+++if __name__ == "__main__":+ export(sys.argv[1], sys.argv[2])diff --git a/chipformer/q35/tiny.py b/chipformer/q35/tiny.pynew file mode 100644index 0000000..91eba01--- /dev/null+++ b/chipformer/q35/tiny.py@@ -0,0 +1,132 @@+"""A standalone Qwen3.5 model that holds only the chip circuit.++Same architecture class and DeltaNet head shapes as Qwen3.5-0.8B, so the circuit that runs here+merges into the real model unchanged. HF's cache needs one full-attention layer to count+positions; it is appended last with zero weights, so it adds nothing.+"""+++import torch++from chipformer.q35.emit import Layout, allocate, check_bf16, embedding_rows, emit_layers+from chipformer.q35.ir import Graph++BASE = { # Qwen3.5-0.8B text config, minus the sizes we pick+ "hidden_act": "silu",+ "intermediate_size": 3584,+ "linear_conv_kernel_dim": 4,+ "linear_key_head_dim": 128,+ "linear_value_head_dim": 128,+ "linear_num_key_heads": 16,+ "linear_num_value_heads": 16,+ "num_attention_heads": 8,+ "num_key_value_heads": 2,+ "head_dim": 256,+ "rms_norm_eps": 1e-6,+ "attn_output_gate": True,+ "rope_parameters": {"rope_type": "default", "rope_theta": 10000000,+ "partial_rotary_factor": 0.25, "mrope_section": [11, 11, 10],+ "mrope_interleaved": True},+ "max_position_embeddings": 262144,+ "tie_word_embeddings": True,+}+++def pick_layout(n_dims: int) -> Layout:+ """Smallest hidden = m * 4^k (a multiple of 64) that fits m ballast dims and n_dims."""+ best = None+ for m in range(3, 16): # >= 3 ballast dims: constants are split over them+ k = 0+ while m * 4**k < m + n_dims or (m * 4**k) % 64:+ k += 1+ h = m * 4**k+ if best is None or h < best.hidden:+ best = Layout(hidden=h, m=m)+ return best+++CODE_GAIN = 2.0**20 # final-norm gain on the output code (the exit leaves KAPPA * +-1 there)+++def code_gain(lay: Layout | None = None) -> float:+ return CODE_GAIN+++def build(g: Graph, vocab: list[int], code_dims: list[int], dtype=torch.bfloat16,+ clearable: set[int] | None = None, share: bool = False):+ """Qwen3_5ForCausalLM whose token i is chip token vocab[i]; lm_head reads ``code_dims``.++ With ``clearable`` (dims whose slots may be handed over), dims share residual slots.+ """+ from transformers import Qwen3_5ForCausalLM, Qwen3_5TextConfig++ alloc = None+ keep = set(getattr(g, "keep", code_dims))+ if share:+ alloc = allocate(g, clearable, keep, BASE["intermediate_size"])+ lay = pick_layout(alloc.n_slots + 1)+ lay.slots = alloc.slots+ else:+ lay = pick_layout(len(g.dims))+ n_chip = g.n_layers()+ n_vocab = -(-len(vocab) // 64) * 64+ tc = dict(BASE, hidden_size=lay.hidden, vocab_size=n_vocab, num_hidden_layers=n_chip + 1,+ layer_types=["linear_attention"] * n_chip + ["full_attention"])+ sd = emit_layers(g, lay, tc, 0, "model.", alloc, keep if hasattr(g, "keep") else None)+ emb = torch.zeros(n_vocab, lay.hidden)+ emb[: len(vocab)] = embedding_rows(g, lay, vocab, lay.hidden)+ sd["model.embed_tokens.weight"] = emb+ norm = torch.full((lay.hidden,), -1.0) # (1 + w) = 0: lm_head sees only the code dims+ for d in code_dims:+ norm[lay.dim(d)] = code_gain(lay) # gain 1 + w, close enough to a power of 2+ check_bf16("final norm", norm)+ sd["model.norm.weight"] = norm+ config = Qwen3_5TextConfig(**tc)+ model = Qwen3_5ForCausalLM(config)+ for p in model.parameters():+ p.data.zero_()+ missing, unexpected = model.load_state_dict(sd, strict=False)+ assert not unexpected, unexpected[:5]+ assert all(k.startswith(f"model.layers.{n_chip}.") or k == "lm_head.weight" for k in missing), \+ [k for k in missing if not k.startswith(f"model.layers.{n_chip}.")][:5]+ model.tie_weights()+ model.layout = lay+ return model.to(dtype).eval()+++def sample_traces() -> list[list[int]]:+ """Short traces covering every opcode and quirk, for deciding which dims are clearable."""+ from chipformer.check import run_machine+ from chipformer.fuzz import random_program, random_quirks++ out = []+ for seed in range(6):+ m = run_machine(random_program(seed), 300, ipf=7, seed=seed, key_rate=0.3,+ quirks=random_quirks(seed))+ out.append(m.tokens)+ return out+++def build_chip8(dtype=torch.bfloat16, share_slots: bool = True):+ from chipformer.q35 import bridge, chip8_q35++ c = chip8_q35.build()+ vocab = list(range(bridge.TEXT_BASE + bridge.N_TEXT))+ model = build(c, vocab, [next(iter(e.terms)) for e in c.code], dtype=dtype,+ share=share_slots)+ return c, model+++def main() -> None:+ import argparse++ ap = argparse.ArgumentParser(description="Save CHIP-8 as a standalone Qwen3.5 checkpoint")+ ap.add_argument("--out", default="dist/q35-tiny")+ args = ap.parse_args()+ c, model = build_chip8()+ model.save_pretrained(args.out)+ print(f"{args.out}: {c.stats()}, hidden {model.layout.hidden}")+++if __name__ == "__main__":+ main()diff --git a/chipformer/ref/__init__.py b/chipformer/ref/__init__.pynew file mode 100644index 0000000..ffe231d--- /dev/null+++ b/chipformer/ref/__init__.py@@ -0,0 +1 @@+"""Reference (non-neural) CHIP-8 implementation used as ground truth."""diff --git a/chipformer/ref/emu.py b/chipformer/ref/emu.pynew file mode 100644index 0000000..f7375e4--- /dev/null+++ b/chipformer/ref/emu.py@@ -0,0 +1,288 @@+"""Reference CHIP-8 interpreter operating directly on the flat cell array.++Every state change is expressed as a list of ``(addr, value)`` writes computed+from the state *before* the instruction, then applied. The trace format is+therefore exactly the emulator's semantics: replaying the writes reproduces+the state.++Quirks are machine state: six flag cells (spec.QUIRKS) with the meaning of+Octo's flags of the same name, set when the ROM is loaded (docs/quirks.md):+ - shift: 8XY6/8XYE shift VX in place (off: VX = VY shifted)+ - loadstore: FX55/FX65 leave I unchanged (off: I += X + 1)+ - clip: sprites clip at the screen edges (off: they wrap around)+ - logic: 8XY1/2/3 reset VF to 0+ - jump: BXNN jumps to XNN + VX (off: BNNN jumps to NNN + V0)+ - vforder: for 8XY4-E the result is written after VF (wins when X = F)+Fixed behaviour: sprites start at (VX % 64, VY % 32); FX0A completes as soon+as any key is held (lowest key wins); the stack pointer wraps modulo 16.+"""++from chipformer import spec+from chipformer.spec import (+ CLS,+ DISPLAY,+ DT,+ KEYS,+ NULL,+ PC,+ Q_CLIP,+ Q_JUMP,+ Q_LOADSTORE,+ Q_LOGIC,+ Q_SHIFT,+ Q_VFORDER,+ RNG,+ SP,+ ST,+ STACK,+ I,+ V,+)++Writes = list[tuple[int, int]]+++def fetch(cells: list[int]) -> int:+ pc = cells[PC]+ return (cells[pc] << 8) | cells[(pc + 1) & 0xFFF]+++def opcode_class(op: int) -> str:+ """Human-readable opcode family, e.g. '8XY4' or 'DXYN'."""+ hi = op >> 12+ if op == 0x00E0:+ return "00E0"+ if op == 0x00EE:+ return "00EE"+ if hi == 0x0:+ return "0NNN"+ if hi == 0x8:+ return f"8XY{op & 0xF:X}"+ if hi in (0xE, 0xF):+ return f"{hi:X}X{op & 0xFF:02X}"+ return {1: "1NNN", 2: "2NNN", 3: "3XNN", 4: "4XNN", 5: "5XY0", 6: "6XNN", 7: "7XNN",+ 9: "9XY0", 0xA: "ANNN", 0xB: "BNNN", 0xC: "CXNN", 0xD: "DXYN"}[hi]+++class Chip8:+ def __init__(self, rom: bytes, quirks: "str | tuple[str, ...] | None" = None):+ if len(rom) > spec.RAM_SIZE - spec.PROGRAM_START:+ raise ValueError(f"ROM too large: {len(rom)} bytes")+ self.cells = [0] * spec.NUM_CELLS+ self.cells[spec.FONT_BASE:spec.FONT_BASE + len(spec.FONT)] = spec.FONT+ self.cells[spec.PROGRAM_START:spec.PROGRAM_START + len(rom)] = rom+ for at, data in getattr(rom, "segments", ()): # code below PROGRAM_START (q35's BIOS)+ assert spec.FONT_BASE + len(spec.FONT) <= at and at + len(data) <= spec.PROGRAM_START+ self.cells[at:at + len(data)] = data+ self.cells[PC] = spec.PROGRAM_START+ self.cells[RNG] = spec.RNG_SEED+ for a, v in spec.quirk_cells(quirks).items():+ self.cells[a] = v++ # --- state transitions --------------------------------------------------++ def apply(self, writes: Writes) -> None:+ apply_writes(self.cells, writes)++ def step(self) -> Writes:+ w = instruction_writes(self.cells)+ self.apply(w)+ return w++ def tick(self) -> Writes:+ w = tick_writes(self.cells)+ self.apply(w)+ return w++ def set_key(self, key: int, down: bool) -> Writes:+ w = [(KEYS + key, int(down))]+ self.apply(w)+ return w++ # --- views --------------------------------------------------------------++ def state_writes(self) -> Writes:+ return state_writes(self.cells)++ def pixels(self) -> list[list[int]]:+ return display_pixels(self.cells)+++def apply_writes(cells: list[int], writes: Writes) -> None:+ for a, v in writes:+ assert 0 <= v <= spec.MAX_VALUE, (a, v)+ if a == CLS:+ cells[DISPLAY:DISPLAY + spec.DISPLAY_H * spec.DISPLAY_ROW_BYTES] = (+ [0] * (spec.DISPLAY_H * spec.DISPLAY_ROW_BYTES))+ cells[a] = v+++def state_writes(cells: list[int]) -> Writes:+ """A state dump: replaying it on a zeroed state reproduces ``cells``.++ Only nonzero cells are written (reads of never-written cells return 0).+ CLS always comes first, so a "latest CLS" exists and the display bytes+ written after it survive.+ """+ return [(CLS, cells[CLS])] + [+ (a, cells[a]) for a in range(spec.NUM_CELLS) if a not in (CLS, NULL) and cells[a]]+++def display_pixels(cells: list[int]) -> list[list[int]]:+ rows = []+ for y in range(spec.DISPLAY_H):+ row = []+ for b in range(spec.DISPLAY_ROW_BYTES):+ byte = cells[DISPLAY + y * spec.DISPLAY_ROW_BYTES + b]+ row.extend((byte >> (7 - i)) & 1 for i in range(8))+ rows.append(row)+ return rows+++def tick_writes(cells: list[int]) -> Writes:+ return [(DT, max(cells[DT] - 1, 0)), (ST, max(cells[ST] - 1, 0))]+++def instruction_writes(c: list[int]) -> Writes:+ """Writes for one instruction, computed purely from the current state.++ The PC write is always last, so every earlier write sees the old PC.+ """+ pc = c[PC]+ op = fetch(c)+ nxt = (pc + 2) & 0xFFF+ skip = (pc + 4) & 0xFFF+ x, y, n = (op >> 8) & 0xF, (op >> 4) & 0xF, op & 0xF+ nn, nnn = op & 0xFF, op & 0xFFF+ vx, vy = c[V + x], c[V + y]+ hi = op >> 12+ w: Writes = []++ if op == 0x00E0:+ w = [(CLS, 0)]+ elif op == 0x00EE:+ sp = (c[SP] - 1) & 0xF+ w = [(SP, sp)]+ nxt = c[STACK + sp]+ elif hi == 0x0:+ pass # 0NNN: machine-code routine, ignored+ elif hi == 0x1:+ nxt = nnn+ elif hi == 0x2:+ sp = c[SP]+ w = [(STACK + sp, nxt), (SP, (sp + 1) & 0xF)]+ nxt = nnn+ elif hi == 0x3:+ nxt = skip if vx == nn else nxt+ elif hi == 0x4:+ nxt = skip if vx != nn else nxt+ elif hi == 0x5 and n == 0:+ nxt = skip if vx == vy else nxt+ elif hi == 0x9 and n == 0:+ nxt = skip if vx != vy else nxt+ elif hi == 0x6:+ w = [(V + x, nn)]+ elif hi == 0x7:+ w = [(V + x, (vx + nn) & 0xFF)]+ elif hi == 0x8:+ w = alu_writes(c, x, n, vx, vy)+ elif hi == 0xA:+ w = [(I, nnn)]+ elif hi == 0xB:+ nxt = (nnn + (vx if c[Q_JUMP] else c[V])) & 0xFFF+ elif hi == 0xC:+ s = spec.lfsr_step(c[RNG])+ w = [(V + x, s & nn & 0xFF), (RNG, s)]+ elif hi == 0xD:+ w = draw_writes(c, vx, vy, n)+ elif hi == 0xE and nn == 0x9E:+ nxt = skip if c[KEYS + (vx & 0xF)] else nxt+ elif hi == 0xE and nn == 0xA1:+ nxt = skip if not c[KEYS + (vx & 0xF)] else nxt+ elif hi == 0xF:+ ix = c[I]+ if nn == 0x07:+ w = [(V + x, c[DT])]+ elif nn == 0x0A:+ pressed = [k for k in range(16) if c[KEYS + k]]+ if pressed:+ w = [(V + x, pressed[0])]+ else:+ w = [(V + x, vx)]+ nxt = pc+ elif nn == 0x15:+ w = [(DT, vx)]+ elif nn == 0x18:+ w = [(ST, vx)]+ elif nn == 0x1E:+ w = [(I, (ix + vx) & 0xFFF)]+ elif nn == 0x29:+ w = [(I, spec.FONT_BASE + 5 * (vx & 0xF))]+ elif nn == 0x33:+ w = [((ix + j) & 0xFFF, d) for j, d in enumerate((vx // 100, vx // 10 % 10, vx % 10))]+ elif nn == 0x55:+ w = [((ix + j) & 0xFFF, c[V + j]) for j in range(x + 1)]+ elif nn == 0x65:+ w = [(V + j, c[(ix + j) & 0xFFF]) for j in range(x + 1)]+ if nn in (0x55, 0x65) and not c[Q_LOADSTORE]:+ w.append((I, (ix + x + 1) & 0xFFF))++ w.append((PC, nxt))+ return w+++def alu_writes(c: list[int], x: int, n: int, vx: int, vy: int) -> Writes:+ vf = V + 0xF+ if n in (0x1, 0x2, 0x3):+ result = {0x1: vx | vy, 0x2: vx & vy, 0x3: vx ^ vy}[n]+ return [(V + x, result)] + ([(vf, 0)] if c[Q_LOGIC] else [])+ src = vx if c[Q_SHIFT] else vy+ if n == 0x0:+ return [(V + x, vy)]+ if n == 0x4:+ s = vx + vy+ result, flag = s & 0xFF, s >> 8+ elif n == 0x5:+ result, flag = (vx - vy) & 0xFF, int(vx >= vy)+ elif n == 0x6:+ result, flag = src >> 1, src & 1+ elif n == 0x7:+ result, flag = (vy - vx) & 0xFF, int(vy >= vx)+ elif n == 0xE:+ result, flag = (src << 1) & 0xFF, src >> 7+ else:+ return []+ if c[Q_VFORDER]:+ return [(vf, flag), (V + x, result)]+ return [(V + x, result), (vf, flag)]+++def draw_writes(c: list[int], vx: int, vy: int, n: int) -> Writes:+ """DXYN as a fixed-length record: two display writes per sprite row+ (left and right byte the row can touch), then VF.++ With clipping, writes that fall off-screen go to NULL so the record length+ depends only on N, which keeps the transformer's micro-step bookkeeping+ trivial. Without it, rows and bytes wrap around the screen.+ """+ clip = c[Q_CLIP]+ x0, y0 = vx % spec.DISPLAY_W, vy % spec.DISPLAY_H+ b, o = x0 >> 3, x0 & 7+ w: Writes = []+ collision = 0+ for i in range(n):+ s = c[(c[I] + i) & 0xFFF]+ parts = ((b, s >> o), (b + 1, (s << (8 - o)) & 0xFF))+ row = y0 + i+ for col, bits in parts:+ if not clip:+ row, col = row % spec.DISPLAY_H, col % spec.DISPLAY_ROW_BYTES+ if row < spec.DISPLAY_H and col < spec.DISPLAY_ROW_BYTES:+ a = DISPLAY + row * spec.DISPLAY_ROW_BYTES + col+ old = c[a]+ w.append((a, old ^ bits))+ collision |= old & bits+ else:+ w.append((NULL, 0))+ w.append((V + 0xF, int(collision != 0)))+ return wdiff --git a/chipformer/ref/play.py b/chipformer/ref/play.pynew file mode 100644index 0000000..43fddfb--- /dev/null+++ b/chipformer/ref/play.py@@ -0,0 +1,72 @@+"""Play a ROM on the reference emulator.++ uv run --extra play python -m chipformer.ref.play roms/pong.ch8++Keypad layout (CHIP-8 -> keyboard):+ 1 2 3 C 1 2 3 4+ 4 5 6 D -> Q W E R+ 7 8 9 E A S D F+ A 0 B F Z X C V+"""++import argparse+import sys+from pathlib import Path++from chipformer import spec+from chipformer.trace import Machine, replay++KEYMAP = {+ "1": 0x1, "2": 0x2, "3": 0x3, "4": 0xC,+ "q": 0x4, "w": 0x5, "e": 0x6, "r": 0xD,+ "a": 0x7, "s": 0x8, "d": 0x9, "f": 0xE,+ "z": 0xA, "x": 0x0, "c": 0xB, "v": 0xF,+}+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("rom", type=Path)+ ap.add_argument("--ipf", type=int, default=10, help="instructions per 60 Hz frame")+ ap.add_argument("--scale", type=int, default=12)+ ap.add_argument("--check", action="store_true", help="replay the trace on exit and verify")+ args = ap.parse_args()++ import pygame++ m = Machine(args.rom.read_bytes())+ pygame.init()+ screen = pygame.display.set_mode((spec.DISPLAY_W * args.scale, spec.DISPLAY_H * args.scale))+ pygame.display.set_caption(f"chipformer reference - {args.rom.name}")+ clock = pygame.time.Clock()+ keycodes = {pygame.key.key_code(k): v for k, v in KEYMAP.items()}++ running = True+ while running:+ for ev in pygame.event.get():+ if ev.type == pygame.QUIT or (ev.type == pygame.KEYDOWN and ev.key == pygame.K_ESCAPE):+ running = False+ elif ev.type in (pygame.KEYDOWN, pygame.KEYUP) and ev.key in keycodes:+ m.set_key(keycodes[ev.key], ev.type == pygame.KEYDOWN)+ m.tick()+ for _ in range(args.ipf):+ m.step()+ screen.fill((16, 16, 24))+ for y, row in enumerate(m.emu.pixels()):+ for x, p in enumerate(row):+ if p:+ screen.fill((230, 230, 210),+ (x * args.scale, y * args.scale, args.scale, args.scale))+ pygame.display.flip()+ clock.tick(60)+ pygame.quit()++ print(f"trace: {len(m.tokens):,} tokens")+ if args.check:+ ok = replay(m.tokens) == m.emu.cells+ print("replay check:", "OK" if ok else "MISMATCH")+ sys.exit(0 if ok else 1)+++if __name__ == "__main__":+ main()diff --git a/chipformer/ref/profile.py b/chipformer/ref/profile.pynew file mode 100644index 0000000..cc46146--- /dev/null+++ b/chipformer/ref/profile.py@@ -0,0 +1,50 @@+"""Opcode histogram and trace statistics for a ROM under random input.++ uv run python -m chipformer.ref.profile roms/pong.ch8 --steps 200000+"""++import argparse+import random+from collections import Counter+from pathlib import Path++from chipformer.ref.emu import fetch, opcode_class+from chipformer.trace import Machine+++def profile(rom: bytes, steps: int, ipf: int = 10, seed: int = 0):+ rng = random.Random(seed)+ m = Machine(rom)+ boot_len = len(m.tokens)+ ops: Counter[str] = Counter()+ for i in range(steps):+ if i % ipf == 0:+ m.tick()+ if rng.random() < 0.1:+ m.set_key(rng.randrange(16), rng.random() < 0.5)+ ops[opcode_class(fetch(m.emu.cells))] += 1+ m.step()+ return m, ops, boot_len+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("rom", type=Path)+ ap.add_argument("--steps", type=int, default=100_000)+ ap.add_argument("--ipf", type=int, default=10)+ args = ap.parse_args()++ m, ops, boot_len = profile(args.rom.read_bytes(), args.steps, args.ipf)+ total = sum(ops.values())+ print(f"{'opcode':8} {'count':>9} {'share':>7}")+ for op, cnt in ops.most_common():+ print(f"{op:8} {cnt:9,} {cnt / total:7.2%}")+ gen = len(m.tokens) - boot_len+ print(f"\nboot prompt: {boot_len:,} tokens")+ print(f"generated: {gen:,} tokens over {total:,} instructions "+ f"({gen / total:.2f} tokens/instruction incl. ticks and input)")+ print(f"at {args.ipf} instr/frame, 60 Hz needs ~{gen / total * args.ipf * 60:,.0f} tok/s")+++if __name__ == "__main__":+ main()diff --git a/chipformer/session.py b/chipformer/session.pynew file mode 100644index 0000000..b1c9d95--- /dev/null+++ b/chipformer/session.py@@ -0,0 +1,72 @@+"""Host loop: drive any next-token model as a CHIP-8 machine.++The host only ever (1) feeds the boot prompt, (2) injects TICK tokens and key+writes right after a STEP, and (3) keeps a mirror of the cells built from the+model's own writes, for the screen. It never computes machine semantics.+"""++from collections.abc import Callable++from chipformer import spec+from chipformer.ref.emu import Chip8, apply_writes, display_pixels+from chipformer.trace import boot_tokens, encode_writes++MAX_RECORD = 2 * 40 + 2 # longer than any valid record+++class Session:+ def __init__(self, feed: Callable[[int], int], rom: bytes, quirks=None):+ """``feed(token)`` processes one position and returns the predicted next token."""+ self.feed = feed+ self.cells = [0] * spec.NUM_CELLS+ self.tokens: list[int] = []+ self.pred = -1+ boot = Chip8(rom, quirks)+ apply_writes(self.cells, boot.state_writes())+ for t in boot_tokens(boot.cells):+ self._push(t)++ def _push(self, tok: int) -> None:+ self.tokens.append(tok)+ self.pred = self.feed(tok)++ def _inject(self, toks: list[int]) -> None:+ for t in toks:+ self._push(t)++ def _generate_record(self) -> list[tuple[int, int]]:+ """Let the model write until it emits STEP; apply its writes to the mirror."""+ writes = []+ for _ in range(MAX_RECORD):+ tok = self.pred+ if tok == spec.STEP:+ self._push(tok)+ apply_writes(self.cells, writes)+ return writes+ if not spec.is_addr(tok):+ raise RuntimeError(f"model emitted {spec.token_name(tok)} where an ADDR or STEP "+ f"was expected (position {len(self.tokens)})")+ self._push(tok)+ val = self.pred+ if not spec.is_val(val):+ raise RuntimeError(f"model emitted {spec.token_name(val)} after an ADDR")+ self._push(val)+ writes.append((tok - spec.ADDR_BASE, val - spec.VAL_BASE))+ raise RuntimeError("record never ended")++ def step(self) -> list[tuple[int, int]]:+ return self._generate_record()++ def tick(self) -> list[tuple[int, int]]:+ self._inject([spec.TICK])+ return self._generate_record()++ def set_key(self, key: int, down: bool) -> None:+ if bool(self.cells[spec.KEYS + key]) == down:+ return+ w = [(spec.KEYS + key, int(down))]+ self._inject(encode_writes(w) + [spec.STEP])+ apply_writes(self.cells, w)++ def pixels(self) -> list[list[int]]:+ return display_pixels(self.cells)diff --git a/chipformer/spec.py b/chipformer/spec.pynew file mode 100644index 0000000..76c03af--- /dev/null+++ b/chipformer/spec.py@@ -0,0 +1,121 @@+"""Machine layout and token vocabulary shared by every component.++All machine state lives in one flat array of cells. A cell holds an integer in+0..MAX_VALUE. The trace the transformer reads and writes is a sequence of+write events ``[ADDR a][VAL v]`` grouped into records terminated by ``STEP``.+See docs/trace_format.md for the full specification.+"""++# --- cell address map -------------------------------------------------------+RAM = 0x0000+RAM_SIZE = 0x1000+V = 0x1000 # V0..VF+I = 0x1010 # 12-bit index register+PC = 0x1011 # 12-bit program counter+SP = 0x1012 # stack pointer, 0..15+DT = 0x1013 # delay timer+ST = 0x1014 # sound timer+RNG = 0x1015 # 12-bit LFSR state+CLS = 0x1016 # writing this cell clears the display+QUIRKS = 0x1017 # 6 quirk flags (0/1), meaning as Octo's flags of the same name; see docs/quirks.md+QUIRK_NAMES = ("shift", "loadstore", "clip", "logic", "jump", "vforder")+Q_SHIFT, Q_LOADSTORE, Q_CLIP, Q_LOGIC, Q_JUMP, Q_VFORDER = (QUIRKS + i for i in range(6))+STACK = 0x1020 # 16 return addresses+KEYS = 0x1030 # 16 key states, 0 or 1+DISPLAY = 0x1100 # 32 rows x 8 bytes, MSB is the leftmost pixel+NULL = 0x1200 # sink for writes that must exist but have no effect+NUM_CELLS = 0x1201++DISPLAY_W = 64+DISPLAY_H = 32+DISPLAY_ROW_BYTES = DISPLAY_W // 8++# --- values -----------------------------------------------------------------+NUM_VALUES = 0x1000+MAX_VALUE = NUM_VALUES - 1++# --- token vocabulary -------------------------------------------------------+VAL_BASE = 0 # VAL v -> v+ADDR_BASE = NUM_VALUES # ADDR a -> ADDR_BASE + a+STEP = ADDR_BASE + NUM_CELLS+TICK = STEP + 1+BOOT = STEP + 2+VOCAB_SIZE = BOOT + 1++# --- machine constants ------------------------------------------------------+PROGRAM_START = 0x200+FONT_BASE = 0x050+RNG_SEED = 0xACE+RNG_MASK = 0x829 # Galois taps for x^12 + x^6 + x^4 + x + 1 (period 4095)++# Named quirk profiles: the set of enabled flags. All flags off = Octo's defaults.+PROFILES = {+ "octo": (),+ "chip48": ("shift", "loadstore", "clip"), # HP48 CHIP-48 / SCHIP-era games, e.g. Pong+ "cosmac": ("logic", "clip"), # original COSMAC VIP interpreter+}+++def quirk_cells(quirks: "str | tuple[str, ...] | list[str] | None") -> dict[int, int]:+ """Cell values for a profile name or a list of enabled flag names."""+ if quirks is None:+ quirks = ()+ elif isinstance(quirks, str):+ quirks = PROFILES[quirks] if quirks in PROFILES else tuple(q for q in quirks.split(",") if q)+ unknown = set(quirks) - set(QUIRK_NAMES)+ if unknown:+ raise ValueError(f"unknown quirks {sorted(unknown)}; known: {QUIRK_NAMES}")+ return {QUIRKS + i: int(name in quirks) for i, name in enumerate(QUIRK_NAMES)}++FONT = bytes([+ 0xF0, 0x90, 0x90, 0x90, 0xF0, # 0+ 0x20, 0x60, 0x20, 0x20, 0x70, # 1+ 0xF0, 0x10, 0xF0, 0x80, 0xF0, # 2+ 0xF0, 0x10, 0xF0, 0x10, 0xF0, # 3+ 0x90, 0x90, 0xF0, 0x10, 0x10, # 4+ 0xF0, 0x80, 0xF0, 0x10, 0xF0, # 5+ 0xF0, 0x80, 0xF0, 0x90, 0xF0, # 6+ 0xF0, 0x10, 0x20, 0x40, 0x40, # 7+ 0xF0, 0x90, 0xF0, 0x90, 0xF0, # 8+ 0xF0, 0x90, 0xF0, 0x10, 0xF0, # 9+ 0xF0, 0x90, 0xF0, 0x90, 0x90, # A+ 0xE0, 0x90, 0xE0, 0x90, 0xE0, # B+ 0xF0, 0x80, 0x80, 0x80, 0xF0, # C+ 0xE0, 0x90, 0x90, 0x90, 0xE0, # D+ 0xF0, 0x80, 0xF0, 0x80, 0xF0, # E+ 0xF0, 0x80, 0xF0, 0x80, 0x80, # F+])+++def lfsr_step(s: int) -> int:+ lsb = s & 1+ s >>= 1+ if lsb:+ s ^= RNG_MASK+ return s+++def val_token(v: int) -> int:+ assert 0 <= v <= MAX_VALUE, v+ return VAL_BASE + v+++def addr_token(a: int) -> int:+ assert 0 <= a < NUM_CELLS, a+ return ADDR_BASE + a+++def is_val(tok: int) -> bool:+ return VAL_BASE <= tok < ADDR_BASE+++def is_addr(tok: int) -> bool:+ return ADDR_BASE <= tok < STEP+++def token_name(tok: int) -> str:+ if is_val(tok):+ return f"={tok - VAL_BASE:#x}"+ if is_addr(tok):+ return f"@{tok - ADDR_BASE:#06x}"+ return {STEP: "STEP", TICK: "TICK", BOOT: "BOOT"}.get(tok, f"?{tok}")diff --git a/chipformer/trace.py b/chipformer/trace.pynew file mode 100644index 0000000..4026634--- /dev/null+++ b/chipformer/trace.py@@ -0,0 +1,118 @@+"""Canonical token traces: encoding, parsing and replay.++A trace is::++ BOOT <state dump writes> STEP+ <record> STEP <record> STEP ...++where each record is one of+ instruction: <writes> (generated by the model)+ timer tick: TICK <DT write> <ST write> (TICK injected by the host)+ input: <KEY cell write> (injected whole by the host)++and every write is the token pair ``ADDR a, VAL v``.+"""++from dataclasses import dataclass, field++from chipformer import spec+from chipformer.ref.emu import Chip8, Writes, apply_writes+from chipformer.spec import BOOT, STEP, TICK+++def encode_writes(writes: Writes) -> list[int]:+ out = []+ for a, v in writes:+ out += [spec.addr_token(a), spec.val_token(v)]+ return out+++def boot_tokens(cells: list[int]) -> list[int]:+ from chipformer.ref.emu import state_writes++ return [BOOT] + encode_writes(state_writes(cells)) + [STEP]+++@dataclass+class Record:+ kind: str # "boot" | "instr" | "tick"+ writes: Writes = field(default_factory=list)+++def parse(tokens: list[int], allow_reboot: bool = False) -> list[Record]:+ """Split a trace into records, validating its grammar.++ ``allow_reboot`` accepts later BOOT records (context rollover in a dumped session).+ """+ records: list[Record] = []+ cur: Record | None = None+ i = 0+ while i < len(tokens):+ t = tokens[i]+ if t == STEP:+ if cur is None:+ raise ValueError(f"empty record before STEP at {i}")+ records.append(cur)+ cur = None+ i += 1+ elif t in (BOOT, TICK):+ if cur is not None:+ raise ValueError(f"{spec.token_name(t)} inside a record at {i}")+ if t == BOOT and records and not allow_reboot:+ raise ValueError(f"BOOT after start at {i}")+ cur = Record("boot" if t == BOOT else "tick")+ i += 1+ elif spec.is_addr(t):+ if i + 1 >= len(tokens) or not spec.is_val(tokens[i + 1]):+ raise ValueError(f"ADDR not followed by VAL at {i}")+ if cur is None:+ cur = Record("instr")+ cur.writes.append((t - spec.ADDR_BASE, tokens[i + 1] - spec.VAL_BASE))+ i += 2+ else:+ raise ValueError(f"unexpected token {spec.token_name(t)} at {i}")+ if cur is not None:+ raise ValueError("trace ends inside a record")+ if not records or records[0].kind != "boot":+ raise ValueError("trace must start with BOOT")+ return records+++def replay(tokens: list[int]) -> list[int]:+ """Rebuild the machine state from a trace alone."""+ cells = [0] * spec.NUM_CELLS+ for r in parse(tokens):+ apply_writes(cells, r.writes)+ return cells+++class Machine:+ """Drives the reference emulator and records the canonical trace.++ This is the ground truth the transformer must reproduce token for token.+ """++ def __init__(self, rom: bytes, quirks: "str | tuple[str, ...] | None" = None):+ self.emu = Chip8(rom, quirks)+ self.tokens = boot_tokens(self.emu.cells)+ self.generated = [False] * len(self.tokens) # tokens the model must predict++ def _emit(self, tokens: list[int], generated: bool) -> None:+ self.tokens += tokens+ self.generated += [generated] * len(tokens)++ def step(self) -> Writes:+ w = self.emu.step()+ self._emit(encode_writes(w) + [STEP], True)+ return w++ def tick(self) -> Writes:+ w = self.emu.tick()+ self._emit([TICK], False)+ self._emit(encode_writes(w) + [STEP], True)+ return w++ def set_key(self, key: int, down: bool) -> None:+ if bool(self.emu.cells[spec.KEYS + key]) == down:+ return+ self._emit(encode_writes(self.emu.set_key(key, down)) + [STEP], False)diff --git a/chipformer/web.py b/chipformer/web.pynew file mode 100644index 0000000..6d0a81b--- /dev/null+++ b/chipformer/web.py@@ -0,0 +1,197 @@+"""Publish the demo assets into the website, or assemble the Hugging Face Space.++ uv run python -m chipformer.web build --blog D:/Blog+ uv run python -m chipformer.web space --out dist/space [--build-wasm]++`build` builds the WebAssembly engine, then writes:+ <blog>/public/chipformer-assets/ engine.wasm, model.chipformer, roms/, thumbs/, ATTRIBUTION.md+ <blog>/components/chipformer/data.ts generated game list, machine constants, model facts+`space` writes a self-contained static site: the sources in space/ plus the same assets and a+generated data.js. It reuses the built engine.wasm unless --build-wasm is given.+The weights repo stays the source of truth; the blog and the Space only receive build outputs.+"""++import argparse+import hashlib+import json+import os+import shutil+import subprocess+from pathlib import Path++import numpy as np++from chipformer.games import GAMES, load_manifest, write_attribution++ROOT = Path(__file__).resolve().parent.parent+WASM = ROOT / "engine" / "target" / "wasm32-unknown-unknown" / "release" / "chipformer_engine.wasm"+PHOSPHOR, SCREEN_BG = (221, 245, 158), (11, 16, 13)+THUMB_INSTR = {"blinky": 20000, "super-pong": 800, "tetris": 120000} # a more telling frame+++def build_wasm() -> None:+ subprocess.run(["cargo", "build", "--release", "--lib", "--target", "wasm32-unknown-unknown",+ "--no-default-features"], cwd=ROOT / "engine", check=True)+++def thumbnail(game: dict, out: Path, instr: int = 2500) -> None:+ """Render a frame of the game from a verified engine run, in the site's phosphor palette."""+ from PIL import Image++ from chipformer import spec+ from chipformer.check import verify_trace+ from chipformer.ref.emu import apply_writes, display_pixels+ from chipformer.trace import parse++ exe = ROOT / "engine" / "target" / "release" / ("engine.exe" if os.name == "nt" else "engine")+ dump = out.with_suffix(".u32")+ subprocess.run([str(exe), "run", str(ROOT / "model"), str(GAMES / game["file"]),+ "--instr", str(instr), "--ipf", str(game["ipf"]), "--quirks", ",".join(game["quirks"]),+ "--seed", "11", "--dump", str(dump)], check=True, capture_output=True)+ tokens = np.fromfile(dump, dtype="<u4").tolist()+ dump.unlink()+ _, err = verify_trace(tokens)+ if err:+ raise RuntimeError(f"{game['id']}: trace does not verify: {err}")+ cells = [0] * spec.NUM_CELLS+ for rec in parse(tokens, allow_reboot=True):+ apply_writes(cells, rec.writes)+ img = Image.new("RGB", (spec.DISPLAY_W, spec.DISPLAY_H))+ img.putdata([PHOSPHOR if p else SCREEN_BG for row in display_pixels(cells) for p in row])+ img.resize((spec.DISPLAY_W * 8, spec.DISPLAY_H * 8), Image.NEAREST).save(out)+++def publish_assets(assets: Path) -> dict:+ """Copy the engine, model, featured ROMs, thumbnails and credits into `assets`; return the+ generated site data (games, layers, model facts, machine constants)."""+ from chipformer.compile import load_sparse++ for sub in ("roms", "thumbs"):+ (assets / sub).mkdir(parents=True, exist_ok=True)+ shutil.copyfile(WASM, assets / "engine.wasm")+ shutil.copyfile(ROOT / "model" / "model.chipformer", assets / "model.chipformer")+ _, config = load_sparse(ROOT / "model" / "model.chipformer")++ manifest = load_manifest()+ featured = sorted((g for g in manifest if "featured" in g), key=lambda g: g["featured"])+ games = []+ for g in featured:+ shutil.copyfile(GAMES / g["file"], assets / "roms" / f"{g['id']}.ch8")+ thumbnail(g, assets / "thumbs" / f"{g['id']}.png", THUMB_INSTR.get(g["id"], 2500))+ games.append({k: g.get(k) for k in ("id", "title", "authors", "year", "description", "license",+ "source", "quirks", "ipf", "keys", "controls", "tap")} | {"gameOver": g.get("game_over")})+ write_attribution(featured, assets / "ATTRIBUTION.md")++ digest = hashlib.sha1((assets / "engine.wasm").read_bytes()+ + (assets / "model.chipformer").read_bytes()).hexdigest()[:10]+ m = config["machine"]+ facts = {+ "layers": config["n_layers"], "dModel": config["d_model"],+ "heads": sum(lay["heads"] for lay in config["layers"]),+ "ffnUnits": sum(lay["ffn_units"] for lay in config["layers"]),+ "params": config["params"], "nonzero": config["nonzero_params"],+ "vocab": config["vocab_size"], "modelBytes": (assets / "model.chipformer").stat().st_size,+ "wasmBytes": (assets / "engine.wasm").stat().st_size,+ }+ return {+ "digest": digest,+ "machine": {k: m[k] for k in ("VAL_BASE", "ADDR_BASE", "STEP", "TICK", "BOOT", "NULL", "CLS", "PC",+ "RNG", "KEYS", "DISPLAY", "QUIRKS", "QUIRK_NAMES")},+ "facts": facts,+ "library": {"total": len(manifest), "verified": sum(g.get("status") == "ok" for g in manifest)},+ "layers": [{"heads": lay["head_names"], "units": lay["ffn_units"]} for lay in config["layers"]],+ "games": games,+ }+++def build(blog: Path) -> None:+ build_wasm()+ assets = blog / "public" / "chipformer-assets"+ d = publish_assets(assets)+ digest, machine, facts, library, layers, games = (+ d[k] for k in ("digest", "machine", "facts", "library", "layers", "games"))+ ts = "\n".join([+ "// Generated by chipformer/web.py in the chipformer repo. Do not edit by hand.",+ "",+ 'export const ASSET_BASE = "/chipformer-assets";',+ f'export const ASSET_VERSION = "{digest}";',+ "",+ f"export const MACHINE = {json.dumps(machine, indent=2)} as const;",+ "",+ f"export const MODEL_FACTS = {json.dumps(facts, indent=2)} as const;",+ "",+ f"export const LIBRARY = {json.dumps(library)} as const;",+ "",+ "/** Per layer: the names of its attention heads (in model order) and its FFN unit count. */",+ f"export const LAYERS: {{ heads: string[]; units: number }}[] = {json.dumps(layers)};",+ "",+ "export type KeyRole = \"up\" | \"down\" | \"left\" | \"right\" | \"a\" | \"b\" | \"player2Up\" | \"player2Down\";",+ "",+ "export type Game = {",+ " id: string;",+ " title: string;",+ " authors: string[];",+ " year: string | null;",+ " description: string;",+ " license: string;",+ " source: string;",+ " quirks: string[];",+ " ipf: number;",+ " keys: Partial<Record<KeyRole, number>>;",+ " controls: string;",+ " tap: KeyRole[];",+ " gameOver: { rows: number[]; cols: number[]; seconds: number; message: string } | null;",+ "};",+ "",+ f"export const GAMES: Game[] = {json.dumps(games, indent=2, ensure_ascii=False)};",+ "",+ ])+ (blog / "components" / "chipformer").mkdir(parents=True, exist_ok=True)+ (blog / "components" / "chipformer" / "data.ts").write_text(ts, encoding="utf-8")+ print(f"published {len(games)} games, engine {facts['wasmBytes'] / 1e3:.0f} kB, "+ f"model {facts['modelBytes'] / 1e6:.2f} MB -> {assets}")+++def space(out: Path, rebuild_wasm: bool = False) -> None:+ """Assemble the static Hugging Face Space: space/ sources + assets + a generated data.js."""+ if rebuild_wasm or not WASM.exists():+ build_wasm()+ if out.exists():+ shutil.rmtree(out)+ shutil.copytree(ROOT / "space", out)+ d = publish_assets(out / "assets")+ js = "\n".join([+ "// Generated by chipformer/web.py (python -m chipformer.web space). Do not edit by hand.",+ "",+ 'export const ASSET_BASE = "./assets";',+ f'export const ASSET_VERSION = "{d["digest"]}";',+ f"export const MACHINE = {json.dumps(d['machine'])};",+ f"export const MODEL_FACTS = {json.dumps(d['facts'])};",+ f"export const LIBRARY = {json.dumps(d['library'])};",+ "/** Per layer: the names of its attention heads (in model order) and its FFN unit count. */",+ f"export const LAYERS = {json.dumps(d['layers'])};",+ f"export const GAMES = {json.dumps(d['games'], indent=1, ensure_ascii=False)};",+ "",+ ])+ (out / "data.js").write_text(js, encoding="utf-8")+ size = sum(f.stat().st_size for f in out.rglob("*") if f.is_file())+ print(f"space: {len(d['games'])} games, {size / 1e6:.2f} MB -> {out}")+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("command", choices=["build", "space"])+ ap.add_argument("--blog", type=Path, help="build: the website repo to publish into")+ ap.add_argument("--out", type=Path, default=ROOT / "dist" / "space", help="space: output directory")+ ap.add_argument("--build-wasm", action="store_true", help="space: rebuild engine.wasm first")+ args = ap.parse_args()+ if args.command == "build":+ if not args.blog:+ ap.error("build needs --blog")+ build(args.blog)+ else:+ space(args.out, args.build_wasm)+++if __name__ == "__main__":+ main()diff --git a/docs/q35-size.md b/docs/q35-size.mdnew file mode 100644index 0000000..efdf834--- /dev/null+++ b/docs/q35-size.md@@ -0,0 +1,243 @@+# Making the merged checkpoint smaller++The merged Qwen3.5-4B + CHIP-8 checkpoint (v15s, the overlay build) is 15.15 GB against the base+model's 9.34 GB, although the circuit itself is only about 0.7M nonzero weights. This note+measures where the extra bytes go, ranks the ways to remove them, and reports the ones built.++The rules every option must keep:+- The result loads in stock transformers: config-only changes, no `trust_remote_code`, no custom+ kernels.+- Qwen's own text behaviour stays within the v15 noise floor.+- The circuit stays exact in bf16.++## Where the bytes go (v15s, measured from the safetensors)++| Part | Params | GB (bf16) | Nonzero |+|---|---|---|---|+| 16 chip layers (width 3072) | 2.168B | 4.34 | 0.7M |+| Qwen's 32 layers (width 2560 → 3072) | 4.284B | 8.57 | 3.570B (the rest is zero padding) |+| Embedding (257,088 × 3072, tied) | 0.790B | 1.58 | 0.640B |+| Vision tower | 0.336B | 0.67 | 0.334B |+| **Total** | **7.577B** | **15.15** | |++The stock architecture forces one hidden size and one intermediate size on every layer. So both+levers are the same two numbers:+- **The chip layer count.** One chip layer at width 3072 is 135.5M params (0.27 GB).+- **The hidden size.** Every dim costs about 2.2M params (4.4 MB) across all 44-48 layers and the+ embedding.++The overlay already put the circuit's chip-token-only dims inside Qwen's 2560. The width above+2560 holds only the dims the circuit needs at text positions (the "text-visible" slots: 499 in+v15s) and the ballast.++## Ranked options++GB is the saving at 4B. "Stock" means the result loads in stock transformers.++| # | Option | Saves | Stock | Risk | Status |+|---|---|---|---|---|---|+| 1 | Fewer chip layers: paired RAM value heads, list scheduling, fused exit (16 → 12) | 4 × 0.26 GB ≈ 1.0 GB | yes | low (IR and GPU exact) | **built** |+| 2 | Narrower: text-visible slots 499 → 318, so hidden 3072 → 2944 | 128 dims ≈ 0.56 GB | yes | low | **built** |+| 3 | Text-only export (`Qwen3_5ForCausalLM`, no vision tower) | 0.67 GB | yes, but no image input | none for text | **built** (`textonly.py`) |+| 4 | Download size: Hub chunk dedup plus compression | (see below) | n/a | none | measured |+| 5 | More text-visible slot cuts (call capture, j thermometer, ctx, div) → 2880 / 2816 | 0.28 GB per 64 dims | yes | medium (circuit rework) | not done |+| 6 | Shorter critical path (the BMUL digit chain sets the 12 layers now) | 0.26 GB per layer | yes | high (macro redesign) | not done |+| 7 | FP8 Qwen layers (`quantization_config`) | ~4 GB | yes (fp8 loader) | changes Qwen's outputs; no official Qwen3.5-4B FP8 | rejected |+| 8 | Interleave the circuit into Qwen's own layers | negative (see below) | — | — | rejected |++## What was built++### Chip layers 16 → 12 (4B)+1. **Paired RAM value heads.**+ - Qwen3.5-4B has two value heads per key head, and the circuit used only the first.+ - Now each RAM key head holds both 2 KB halves of CHIP-8's memory, one per value head.+ - The write gate (beta) picks the half by address bit 11, which a "prev" head column+ delivers from the ADDR token.+ - On a read, the output gate passes only the half asked for.+ - Both RAM copies drop from 32 key heads in two attention sublayers to 16 in one.+ - The value-head machinery is the store fork's (`ir.VHead`, per-value-head beta and outputs).+2. **RAM reads summed into shared dims.** Every RAM head adds its 8 read bits into the same 8+ dims (head `into`): a query matches at most one head and the rest read exactly 0. Before, each+ head had 8 dims of its own (256 per copy).+3. **List scheduling** (`ir.Graph.reschedule`).+ - Build-time placement is greedy in build order, so a node off the critical path could take+ the room a critical one needed. The ramA fetch heads lost their sublayer to `trim.R`,+ `stack` and `digit.count`.+ - The scheduler re-places every node sublayer by sublayer, longest remaining path first.+ - It keeps the build-order meaning of dims several nodes write: a reader sees only the writes+ built before it.+4. **Fused exit.** The exit computes the output code straight from the output cases+ (`2κ·AND(case, bit, owner) − κ·owner`), so the separate `next.*` select sublayer before it is+ gone. The IR check now decodes the predicted token from the exit code itself (`c.next_bits`),+ which is what the lm_head reads.++Result: 4B 12 layers (the critical path, uncapped, is also 12). The tiny / 0.8B targets+(16 value heads, no pairing) go 16 → 13. The 0.8B's overlay pool is full, though, and the packed+schedule needs more chip-region slots there. So `merge.py` builds both the scheduled and the+build-order circuit and keeps the smaller checkpoint.++### Text-visible slots 499 → 318 (hidden 3072 → 2944)+The slot audit (`dist/size/audit.py`) finds the dims live at the peak of the chip region. The big+items were representations that need many dims where a few would do:++| Item | Before | After | How |+|---|---|---|---|+| ADDR token's address | 49 one-hot dims | 13 bits | Write keys read them through a conv lag tap, so they can't overlay. Each key is now an AND of 13 bit literals (`bit_lit`: 0 on non-ADDR tokens, like the one-hots). |+| Vector-read address (`vread_addr`) | 48 one-hots | 12 bits | `prims.add12_bits`. Each RAM row's query is an AND of bit literals. It is needed on text tokens because a print reads its digits there. |+| Read bits (sum into shared dims) | 128 | 8 | |+| `I+j` | | 12 bits | |+| Display target row/column | 40 one-hots | 8 bits | |++With 318 chip slots (the table above plus the display target gated to 0 on text, and the+scheduler's headroom below), `pick_hidden` gives 2944 (m = 46 ballast dims;+2560 + 46 + 1 + 318 ≤ 2944, 19 to spare).++### Text-only export+`python -m chipformer.q35.textonly MERGED OUT` writes the language model as a stock+`Qwen3_5ForCausalLM`: keys `model.language_model.*` → `model.*`, `text_config` as the config, and+no vision tower. `evals.load_model` (used by evals and arith_bench) loads either kind.++### Two fixes the shrink needed+- **Counter slots are never handed over** (`emit.allocate`, "sticky").+ - The counter heads' outputs reach 2^-114.+ - Once normed by the ballast, such a value underflows, so the a-form unit that hands a slot+ over (or the exit's clear) cannot subtract it exactly.+ - Before, a residue always happened to be absorbed by a larger value sharing the slot.+ - Below the head's declared `tiny` (2^-90) the residue is left in an unread chip dim, and+ `test_q35_v2` allows exactly that.+- **The scheduler leaves 1/16 of every MLP free.** A full sublayer leaves the allocator no unit+ for a handover, which costs slots: 339 at a 9216 cap, 318 at 8640, same depth.++## Results (q35-batch, L4)++**Size** (bytes on disk; base Qwen3.5-4B is 9.34 GB):++| Checkpoint | Hidden | Chip layers | Params | GB | vs v15s |+|---|---|---|---|---|---|+| v15s (start) | 3072 | 16 | 7.577B | 15.15 | |+| **v15x** (this branch) | **2944** | **12** | **6.756B** | **13.53** | **−1.62 GB (−11%)** |+| v15x text-only | 2944 | 12 | 6.421B | 12.86 | −2.29 GB (−15%) |++v15x breakdown:++| Part | Params | Nonzero |+|---|---|---|+| Chip layers | 1.558B | 0.8M |+| Qwen layers | 4.105B | 3.570B |+| Embedding | 0.757B | |+| Vision | 0.335B | |++So the extra over base is now 4.2 GB, against 5.8 GB before:+- 3.1 GB of chip-layer zeros+- 1.1 GB of Qwen padding to 2944++The tiny standalone checkpoint goes from 16 layers at 2048 to 13 layers at 1536, about 38% fewer+bytes per token.++**Exactness and behaviour:**+- **Circuit (IR):** the test ROMs on two quirk profiles and the fuzz pass on the 4B and tiny+ targets with 0 bad predictions.+- **Tests:** the full q35 suite passes, one file at a time (147 passed):+ - v2 12, chip8 16 (including the HF tiny checkpoint, bf16 and fp32)+ - inline 74, macros 18, overlay 6 (including the noisy-IR leak checks and the synthetic merged+ checkpoint)+ - store 10, mute 2, numerics 5, ir 3, vheads 1+- **Real 4B v15x on GPU:**+ - The evals selftest: 7 tool calls, 0 wrong.+ - Inline `tc` arith: 14/20, with all 61 computer results exact. v15s got 16 on the same+ problems: 0 wins, 2 losses, both Qwen's own text, not significant. v15z, the same Qwen+ weights at 13 chip layers, got 15. The spread is batch numerics: with `tc`, Qwen's text+ can't depend on the chip layers.+ - DM mul_big (8 problems): 7/8, with 10/10 results exact.+ - IR replay of those records: 1,440 owned positions, 0 mismatches, 0 stray chip tokens.+- **Text vs base** (`text_eval`, same items as v15s):++| Model | MMLU choice KL | MMLU acc diff | ARC choice KL | ARC acc_norm diff |+|---|---|---|---|---|+| v15s | 6.9e-4 | −0.5 | 8.8e-4 | +0.2 |+| v15x | 7.2e-4 | −0.1 (p=1.0) | 8.3e-4 | +0.2 (p=0.73) |++ That is unchanged.+- **Text-only export:** loads as `Qwen3_5ForCausalLM`, gives logits identical to the full model+ (max |diff| 0.0), and its evals selftest is 7/7 exact.++## Download size++Measured by hashing 64 KB blocks (chunk dedup, as the Hub's Xet storage does) and compressing+the unique blocks (zlib level 1 with bf16 byte grouping, close to Xet's LZ4 plus byte grouping):++| Checkpoint | On disk | All-zero blocks | Unique blocks | Unique, compressed |+|---|---|---|---|---|+| base Qwen3.5-4B | 9.32 GB | 0.00 | 9.32 | 6.71 |+| v15s | 15.15 GB | 4.17 | 10.84 | 6.78 |+| v15x | 13.51 GB | 2.90 | 10.47 | **6.75** |+| v15x text-only | 12.84 GB | 2.90 | 9.81 | **6.27** |++So a deduplicating, compressing transfer moves about as much for the merged model as for the+base (6.75 vs 6.71 GB). The zero padding sits in short runs inside rows, so chunk dedup misses+it, but compression removes it. What remains bigger is GPU memory and decode bandwidth: those+follow the 13.5 GB on disk.++No stock-loadable format stores zeros cheaply:+- compressed-tensors dropped sparse compression, and transformers' loader only handles+ quantization formats.+- safetensors has no shared or zero tensors.++So the dedup and compression of the transfer is what a download gets.++## FP8 (option 7)++- **No official FP8 4B.** Qwen publishes FP8 checkpoints only for Qwen3.5-27B and larger. An FP8+ Qwen3.5-4B would be our own quantization, so it is no longer "Qwen as shipped".+- **The chip layers can't follow.** Of the 2,193 chip-weight 128×128 blocks that have nonzeros,+ 1,684 are exactly e4m3 under some power-of-2 block scale, but 509 aren't: those weights need 5+ to 8 significant bits. The chip layers would have to stay bf16 (`modules_to_not_convert`).+- **What it would save:** Qwen's 32 layers at 1 byte per weight, about 4.1 GB at 2944.+- Rejected under the "Qwen as shipped" rule. It is worth revisiting if an official FP8 4B+ appears; widening it would need re-blocking the padded columns.++## Interleaving the circuit into Qwen's layers (option 8)++The idea: raise `intermediate_size` and the GDN head counts a little for all 32 layers, and run+the circuit inside Qwen's own layers instead of 12 extra ones. It doesn't pay, for three+measured reasons.++1. **Chip tokens.** At chip positions Qwen's sublayers still run. In stock generate (no `tc`),+ Qwen's conv windows and full-attention KV then take in each chip token's projection.+ - Today that projection is exactly 0, apart from the mute marker, because the chip dims are+ cleared before Qwen's layers.+ - With the circuit inside Qwen's layers, chip values would sit in the residual there. With+ the overlay they would sit in Qwen's own dims, and Qwen's in_proj/k_proj read every one of+ them, so text after a trace would change.+ - So interleaving rules out the overlay: every chip slot needs its own dim, and hidden goes+ back to about 2560 + 1700 = 4250 (the v15 layout). That alone costs more than the 12 chip+ layers (+1.7 B params of padding on Qwen's 32 layers versus 1.56 B for the chip layers).+2. **Text tokens.** The circuit computes at text positions too: capture, the print, the owner+ bit.+ - That needs the ballast, which pins every RMSNorm on a known power of 2, but Qwen's+ sublayers share those norms and need Qwen's own RMS.+ - So the text-position work can't move into Qwen's layers, and a front block of chip layers+ stays anyway.+3. **Capacity.** The full-attention layers (8 of 32) have no DeltaNet heads, and the per-layer+ head and unit increase applies to all 32 layers. The circuit's peaks (16 RAM heads in one+ sublayer, about 9k units in one MLP) would need X ≈ 2-3k extra units and about 6 extra key+ heads per layer, roughly 1.1B params by themselves.++Only a decoder-only variant (always `tc`, Qwen never sees chip tokens) avoids reason 1. It breaks+the computer in stock `generate()`, so it is rejected too.++## What's left (by payoff)++1. **More text-visible slot cuts.** At the peak the largest groups are:+ - the tool-call capture (`call.*`, 72)+ - the write-index thermometer `j>` (65)+ - ctx copies (43)+ - division (40)++ Getting to ≤ 274 slots gives hidden 2880, and ≤ ~210 gives 2816, about 0.28 GB each.+2. **The critical path** is the BMUL digit chain: ctx → digit reads → a·b → s//10 → wrap → carry+ → digit one-hot → exit, 12 layers. Each layer saved is about 0.26 GB, and it needs a macro+ redesign.+3. **Non-uniform ballast**, so hidden needn't be m·4^k. Then hidden = 2560 + slots + a few+ ballast dims, rounded to 64. The power-of-4 rule needs only Σ b_i² = hidden·4^r with b_i+ powers of 2.diff --git a/docs/q35-skills.md b/docs/q35-skills.mdnew file mode 100644index 0000000..eb5fda8--- /dev/null+++ b/docs/q35-skills.md@@ -0,0 +1,407 @@+# q35: more built-in skills (plan)++Status: design and estimates only, written against v15 (5efa9e6) with v16 (signed division, `mod`,+`gcd`) in progress. Every token count here is either measured on the reference trace+(`BridgeMachine`, v15 BIOS) or derived from those measurements with the per-record costs below.+Circuit costs are estimates, using what v14 division actually cost as the yardstick.++## 0. Starting point (measured)++**What the computer does today.** It does exact integer add, sub, mul and div:+- Operands are up to 24 digits each (`MAX_DIGITS`, `MAX_ARG`).+- A print holds at most 58 digits (`MAX_PRINT`), and a record holds at most 64 writes (`JMAX`).+- div prints the quotient, its terminating decimal, or 10 decimals then "..." (`DEC = 10`).+- Inline triggers are "A op B =" with exactly one operator. A "." breaks the window, so+ "2.5 × 3 =" is left alone.++**Trace cost per call.** The v15 reference counts the tokens after the CALL record, including the+print. The median of 20 random cases:++| call | tokens | | call | tokens |+|---|---|---|---|---|+| add 4+3 digits | 30 | | mul 8x8 | 206 |+| add 12+12 | 55 | | mul 12x12 | 394 |+| sub 8-5 | 42 | | div 4/4 | 526 |+| mul 2x2 | 44 | | div 6/2 | 570 |+| mul 4x3 | 68 | | div 12/4 | 751 |++- **Per-record costs** (used for every estimate below):+ - A plain CHIP-8 instruction costs about 5 tokens.+ - FX55/FX65 over n registers, or a macro record with n writes, costs about 2n + 3.+ - The CALL record costs about 2 tokens per argument digit, plus about 10.+- **Division overhead.** Division spends about 58 plain records (about 300 tokens) in BIOS setup+ and its strip/print loop. That loop, not BDIV, is division's largest overhead.+- **Speed.** Every estimate below is in tokens. At an assumed 50 tok/s per stream for the merged+ 4B, 1k trace tokens is about 20 s. With `tc`, trace length affects only speed and context+ budget, never Qwen's text.++**Circuit occupancy (v15, 16 layers = 32 sublayers).**+- 104 of 256 head slots and 24,463 of 57,344 FFN units at intermediate 3584, with 3,623 dims.+- Full MLP sublayers: 2, 12 and 14 (3584 each). Sublayers 16 and 20 are about 89% full.+ Sublayers 22-32 are nearly empty, but they are deep.+- Full attention sublayers: 3, 5, 21 and 23 (16 heads each).+- At the 4B, intermediate is 9216, so MLP room is no issue there. The binding costs are:+ - **Hidden width.** About 1,580 chip dims sit on top of Qwen's 2,560, giving hidden 4224.+ v14 division alone took it from 3840 to 4224.+ - **Depth.** BDIV only fit after moving its quotient digit to the record's first write.+- Macro opcodes: 5XY1-5XY8 are used (5XY0 is SE), so 9, A, B, C, D, E and F are free. The Y+ field gives variants, as 5X13 BMUL0 does.+- ROM: the BIOS occupies 0x200-0x2AE (174 bytes).+ - Free space: about 850 bytes at 0x2AE-0x5FF, and about 1.8 KB above 0x900.+ - Data: A, B, LEN and R sit at 0x600-0x6BE, and W runs from 0x6BF to about 0x6E2.+ - Macro jump targets (T = 0x200 + 2·XY) must sit below 0x400.++**What Qwen actually writes.**+- Calculations Qwen wrote before an "=" (classified by `scratchpad/skills/qwenwrites.py`):++| GSM-Hard, 200 problems | base + nudge | v14 inline + tc |+|---|---|---|+| "... =" calculations | 1891 | 1803 |+| integer, one operator (today's trigger) | 61.8% | 60.1% |+| decimal operand, one operator | 8.9% | 9.9% |+| integer chains (2+ operators) | 3.9% | 4.1% |+| containing `\frac` | 4.8% | 3.9% |+| containing parentheses | 4.2% | 4.9% |+| problems with any decimal | 85 | 95 |+| problems with `\frac` | 71 | 66 |+| problems with a negative operand | 26 | 20 |+| `\sqrt`, `\log`, `^`, `!`, `\binom`, `mod` | ≤ 9 problems each | |++- **Gold calculator annotations.**+ - GSM8K test (4282 annotations):+ - 4.9% have a decimal input (10.8% of problems).+ - 13.1% are multi-operator (35% of problems).+ - 0.7% are inexact divisions.+ - Powers and percents: none.+ - GSM-Hard solutions:+ - 22.6% contain decimal literals, 23% have non-integer answers and 11.1% have negative answers.+ - `%` appears in 12.5%.+- **DeepMind modules, fraction of all 10k questions:**++| module | decimal operands | negative operands | chains with parentheses | fraction answers |+|---|---|---|---|---|+| mul_big (extrapolate) | 75% | 74% | – | 0% |+| add_or_sub_big | 74% | 74% | – | 0% |+| div_big | 0% | 74% | – | 45% |+| mul_div_multiple_longer | 0% | 99% | all | 0% |+| mixed_longer | 0% | 99% | all | 0% |+| mixed (interpolate) | 0% | 96% | all | 49% |++So the integer-only computer can be used on about a quarter of DM's mul_big and add_or_sub_big,+which is why the `dmxc_` clean subsets exist. Decimals, negatives and chains are worth far more+than logs or roots on every benchmark we run today. Roots, logs and powers matter for MATH,+SciBench and synthetic sets.++## 1. The architecture decision: an expression tape, not one trigger per op++Today each op needs its own text class, scanner state, capture columns, CALL-entry logic and+digit ports. These capture costs are in the circuit; the arithmetic itself lives in the BIOS.+Doing that for 20 more ops would cost circuit width per op. It would also never cover chains,+parentheses or mixed notation.++**Proposal: two levels.**+1. **The fast path (keep).** Single binary add/sub/mul/div with the existing triggers, extended+ to decimal and signed operands (§3, P1). These are 60-70% of Qwen's calculations and the+ cheapest in tokens.+2. **The expression tape (new, P2).** Every other calculation goes through it.+ - **Capture.** While a candidate window is open, each text token's class code (6-7 bits) is+ stored in a new DeltaNet "tape" head, keyed by its index in the window (≤ 64 tokens). This is+ the same mechanism as today's digit port, which keys digits by (argument, digit index), but+ keyed by token index. No CALL record copies the expression, so the 64-write limit doesn't+ bind.+ - **Fire.** The window fires at " =" or "=" if it contains at least one operator or function+ and passes the fire rules in §5.+ - **Hand-off.** The CALL record then writes only the tape length, MODE and PC = `eval`.+ - **Read.** A new instruction, **BTAPE 5XYD: VX = tape[VY]**, lets the BIOS read the tape one+ token per record (about 5 tokens each).+ - **Evaluate.** A BIOS parser (shunting-yard, ≤ 8 operands, paren depth ≤ 4) evaluates with+ exact rational bignums. It calls the existing routines as subroutines; they get RET-ending+ entry variants.+ - **Print.** An integer, a terminating decimal, a reduced fraction or a 10-decimal truncation+ (§4).+ - **New ops become BIOS-only.** sqrt, ^, log, mod, gcd and ! need only text classes for+ their spellings plus BIOS code. They add zero circuit heads.+ - **Tool mode.** It gets one tool, `calc(expression)`. Its parameter text goes through the same+ tape. ("=calc" tokenizes as "=", "calc", so it needs one FN class.)++**The trade-off.**+- **Tokens.** Parsing costs about 25-40 trace tokens per input token, so a 20-token expression+ adds about 0.5-0.8k tokens before any arithmetic. That's why single ops keep the fast path.+- **Coverage.** The tape covers chains, parentheses, signs, `\frac`, `^` and functions in one+ circuit change. It lets Qwen write a whole DM expression once and get the exact answer. In+ inline mode those modules currently fail by running out of context while Qwen evaluates by hand.+- **Verdict.** Build the tape before any new op, and route every new skill through it.++## 2. New shared macros (promote hot loops; each is one record)++| opcode | name | semantics | uses | est. circuit cost |+|---|---|---|---|---|+| 5XYB | BMOV | A[0..n) = R[z..z+n), LA = n, with z = R's leading zeros (Y bit: into B) | every loop that feeds a result back (powers, factorial, gcd, Newton) | 1-2 heads, 0.5-1k units, 30-60 dims |+| 5XY9 | BSDIV | in place A = A div VX for 2 ≤ VX ≤ 25, MSB first, VF = remainder (r·10 + d ≤ 249 stays bf16-exact) | base conversion, small-prime trial division, halving, mod small | 1-2 heads, 1-1.5k units, ~50 dims |+| 5XYA | BSMAD | in place A = A·VX + VY for VX ≤ 16, VY ≤ 15 (9·16 + 15 = 159 < 256) | Horner (from base k), n! and nCr with small factors, ×10^k alignment | 1-2 heads, 1-1.5k units, ~50 dims |+| 5XYC | BSQRT (Y: add-back) | one square-root digit, modelled on BDIV: window -= (D·10 + q)·q with q estimated from the window's top digits over D's (D = 2·root so far); P[t] = q; flags like BDIV | sqrt | 3-5 heads, 2-3k units, 150-250 dims (BDIV+BADB took about 380 dims) |+| 5XYD | BTAPE | VX = tape[VY] | the evaluator | tape head 1-2 heads + 0.3k units, ~30 dims |+| 5XYE | BSHAD | in place W += W >> VX (Briggs step), VF = [W ≥ 10^m] | log, exp, a^x | 1-2 heads, ~1k units, ~50 dims |+| (print) | PRINT variants | MODE bit 16 prints nibbles 10-15 as a-f (hex); a register gives a "." position, so decimals print without shifting memory | hex output, every decimal result | 50-150 units |++**Strategy.** Write each skill first as a BIOS program on the existing macros, which costs zero+circuit width. Prove it exact against the Python reference. Measure where its tokens go, then+promote only the loops that dominate. BMOV comes first, because every iterative skill copies R+back into A.++## 3. Skill table++Trace estimates are for typical sizes. "BIOS" means no circuit change beyond the text classes.++| skill | how Qwen writes it (measured Qwen3.5 tokens) | output | algorithm | BIOS / macro | trace tokens | circuit | unblocks |+|---|---|---|---|---|---|---|---|+| **decimal operands** (add/sub/mul/div) | `' ','3','.','7','5',' \\','times',' ','2','.','4',' ='`; "1,234.56" keeps its commas | exact decimal, trailing zeros stripped ("7.5"), "-" sign; div keeps its 10-decimal "..." form | capture counts fraction digits fa, fb per operand (PERIOD becomes neutral between digits; a second "." breaks the window). The BIOS aligns: add/sub pad the shorter fraction with zeros (one FX55); mul puts the point at fa+fb; div computes A·10^fb / (B·10^fa) | capture change + BIOS; later the print-with-"." variant | +60-90 over the integer op (add 12.5+3.75 ≈ 110; mul 4x3 decimal ≈ 130) | 2-3 heads, 0.5-1k units, 30-60 dims (fraction counters, CALL writes LEN+3/4) | DM mul_big and add_or_sub_big full modules (75% decimal), GSM-Hard (~10% of calculations), NUPA Float tasks, FinQA |+| **negative operands** (all ops) | `' -','1','2'` (" -" is MINUS, the binary-minus class: unary when p = 0 or after an operator), `' (-','1','2',')'` (" (-" and "(-" are fused tokens: new classes; ")" becomes neutral) | " -digits" (MODE bit 8, already printed by readout) | sign bits sa, sb in MODE. mul/div: sign = sa xor sb. add/sub: compare magnitudes with BSUB's branch and route to add, sub or BRSUB (exists) | capture + BIOS | +20-40 | ~1 head, 0.3k units, ~15 dims | DM (74-99% negative operands), GSM-Hard (11% negative answers), BBH |+| **expression evaluator** | chains `350,000 + 17,500 + 42,000 =`, `' (','2',' +',' ','3',')',' \\','times',...`, `\frac{1785}{13201915} =` (`'}{'` and `'}'` are tokens), DM-style `(-1)/(-1)*(-10)/(-4)*...` | integer; exact decimal if any operand is decimal; reduced "p/q" for an inexact all-integer expression; else 10 decimals + "..." | tape → shunting-yard → rational bignums. Reduce by gcd only when the final p/q isn't an integer (try BDIV first) | tape + BTAPE + BIOS (~300-400 bytes) | parse 25-40/token; 5 operands of ≤ 6 digits ≈ 1.5-3k; DM mixed_longer (~25 tokens, 8 operands) ≈ 3-6k | ~3-5 heads, 1-2k units, 60-100 dims (tape, trigger generalisation, ~25-30 new text classes) | DM mixed, mixed_longer, mul_div_multiple(_longer), add_sub_multiple(_longer), div_big fractions (45%); BBH mat2; GSM chains (4%) and `\frac` (4-5%) |+| **integer powers** a^n | `' ','2','^{','1','0','}',' ='`, `'^'` (61), `' ^'`, `'**'` (332); `'^{-'` is fused (negative exponents → rationals) | integer up to 58 digits, else scientific "1.2676506002e30" (digits, ".", "e" are all single-spelling classes) | n ≤ 16: repeated BMUL + BMOV. Larger n: square and multiply | BIOS, BMOV | 7^20 (17 digits) ≈ 1.3k; 2^64 ≈ 1.0-1.5k; 3^100 (48 digits) ≈ 4k | classes only (+BMOV) | MATH counting/number theory, synthetic, SciBench |+| **factorial, nCr, nPr** | `' ','5','!',' ='`, `'\\','bin','om','{',...,'}{',...`; "C(10, 3)" is ambiguous (skip) | integer / sci | n!: BSMAD per factor ≤ 16, then BMUL. nCr: multiplicative with exact BSDIV by i ≤ 25 (above 25, BDIV) | BIOS, BSMAD/BSDIV | 20! ≈ 0.8k; C(50,25) ≈ 1.8k | classes only (+macros) | MATH counting & probability |+| **sqrt** | `' \\','sqrt','{',...,'}',' ='` (with `$\\` fused when after "$"); `'sqrt','('` | integer if a perfect square, else 10 decimals + "..." | digit by digit (pairs of digits, like long division). The BIOS prototype estimates the digit with 8-bit register maths, then checks with BMUL + BSUB; BSQRT later | BIOS first, then BSQRT | 6-digit radicand → 13 digits: BIOS ≈ 4-5k, with BSQRT ≈ 1.2k | 0, then BSQRT | MATH, SciBench, synthetic; "√77449 ≈ 278" style prime checks in GSM-Hard |+| **n-th roots** | `'\\','sqrt','[','3',']{'...`, `'^{','1','/','3','}'` | integer if exact, else 10 decimals + "..." | exact integer roots: digit-by-digit search with a power check. General: exp10(log10(x)/n) | BIOS | perfect cubes ≈ 2k; general ≈ log + exp (§ logs) | 0 | SciBench, MATH (rare) |+| **log10, ln, log2, log_b** | `' \\','log','_{','1','0','}',...`, `' \\','log','_','2',...`, `' \\','ln',' ',...`, `'\\','log','('` | 10 decimals + "..." (truncated; integer when exact, e.g. log2 8 = 3, found by an exact-power pre-check) | Briggs pseudo-division: normalise x into [1, 10) (shift count = characteristic). For k = 0..12, multiply x by (1 + 10^-k) while ≤ 10, subtracting log10(1 + 10^-k) from a 14-digit accumulator. ROM table: 13 constants × 14 digits ≈ 182 bytes. ln = log10 × 2.30258509299404 (one BMUL); log2 ≈ × 3.32192809488736; log_b a = log a / log b | BIOS first; BSHAD makes it practical | BIOS ≈ 14k (about 58 inner steps × about 240); with BSHAD plus a table-load BMOV variant ≈ 3.5k; log_b ≈ 2 logs + div | 0, then BSHAD | SciBench (chemistry pH, decibels, half-lives), synthetic vs mpmath |+| **exp, a^x (real x), e^x** | `' e','^{','0','.','5','}'` (" e" is a word token: a class only after a math opener), `' \\','exp','('` | 10 decimals + "..." | inverse Briggs: subtract table constants from y, multiply by (1 + 10^-k). a^x = exp10(x·log10 a) | BIOS, then BSHAD | ≈ the same as log (3.5-14k); a^x ≈ 2× | 0 | SciBench, finance (compound interest with non-integer periods) |+| **mod, floor-div** | `' ','1','7',' \\','mod',...`, `' \\','b','mod'` ('b' is the hex-digit class: allow BSL, b, mod), `' mod'` (1410), `'\\','p','mod','{'` (congruence, no "="; skip), `' //'` | integer remainder / quotient | div's BDIV loop stopped at t = LA; print the W window or P (v16) | BIOS (v16) | ≈ (LA−LB+1)·17 + 100: 12/4 digits ≈ 250 | classes only | DM numbers__div_remainder, MATH number theory, GSM-Hard (12.5% use %) |+| **gcd, lcm** | `' \\','gcd','('`, `' gcd','('`, `'\\','text','{l','cm','}('` (lcm is awkward; tool form) | integer | Euclid via BDIV remainders + BMOV (v16); lcm = a / gcd · b | BIOS (v16) + BMOV | 12-digit pair ≈ 23 steps × 120 ≈ 2.8k; lcm +0.8k | classes only | DM numbers__gcd, numbers__lcm, fraction reduction in the evaluator |+| **modular exponentiation** | tool `powmod(a, e, m)`; inline is rare | integer | square and multiply with a mod each step | BIOS | 10-digit m, 30-bit e ≈ 35k (too slow inline; tool only) | 0 | MATH number theory (rare) |+| **primality, factorization** | "Is 25439 prime?" is prose (tool `is_prime`, `factor`); `=is`,`prime` and `=f`,`actor` fuse in tool names | "yes"/"no" is awkward (output classes); print the smallest factor, or the factor list "2, 3, 3, 7" (COMMA exists) | trial division: BSDIV by primes ≤ 25 (one record each), then BDIV by a ROM table of primes < 1000 (336 bytes). Complete for n < 10^6 | BIOS, BSDIV | composite: early exit, usually < 1k; prime near 10^6: ≈ 20k | 0 (+BSDIV) | DM numbers__is_prime, numbers__list_prime_factors (check sizes after download) |+| **base conversion** | `' ','1','0','1','1','_','2'`, `' (','1','0','1','1',')_','2'` (')_' fused), `'_{','1','6','}'`, `' 0','x','1','F'` ('x' and uppercase A-F aren't classes); DM uses prose "(base 11)" → tool `to_base(n, k)` / `from_base(s, k)` | digits in base k, with a-f printed through the hex print bit (MODE 16) | to base k: BSDIV passes (one record per output digit). From base k: Horner with BSMAD | BIOS + BSDIV/BSMAD + hex print | 12-digit decimal → binary ≈ 1.4k, → hex ≈ 0.4k; 40-bit binary → decimal ≈ 1.2k | +60 units (print) and ~8 input classes ('x', 'A'-'F' as input-only, '_', '_{', ')_') | DM numbers__base_conversion, arithmetic__add_or_sub_in_base |+| **percent** | `' ','1','5','%',' of',...` (" of" is a word: allowed only straight after "%"), `'\\','%'`, `' %'`; Qwen mostly writes 0.15 × 80 itself | exact decimal | x% → x/100 (decimal point shift, free), "of" → × | BIOS in the evaluator | ≈ decimal mul | 3 classes | FinQA, GSM (Qwen rarely needs it) |+| **rounding** | prose ("round to 2 dp"); tool `round(x, n)` | decimal | truncate at n plus round-half-even/up on the guard digit (a carry via BADD) | BIOS | ≈ 100 | 0 | DM numbers__round_number, FinQA |+| **comparison, min/max** | prose, tool `max(...)` | the chosen number | BSUB's branch | BIOS | ≈ 50 per comparison | 0 | DM comparison__* (Qwen already does these well: low value) |+| **trig (sin, cos, tan, atan)** | `' \\','sin','('`, `'^\\','circ'` (degrees) | 10 decimals + "..." | CORDIC with a ROM atan table (≈ 13 × 14 digits), degrees → radians by one BMUL | BIOS (+BSHAD) | ≈ 10-20k | 0 | SciBench physics only. **Low priority.** |++**Ambiguities to settle when building.**+- **The DIV class must split.** "/" is today's DIV class with extra spellings " /" and "÷". To+ print reduced fractions "p/q", "/" must become a single-spelling printable class and the others+ move to an input-only DIV2. This is the same fix MINUS/MINUS2 needed.+- **Scientific notation.** Printing "× 10^{n}" would need printable "^{" and "}" classes, and+ " ×" has several spellings. So print "1.2676506002e30" instead: digits, "." and "e" are all+ single-spelling already.+- **Precision of irrational results.** These use 14-15 internal digits for 10 printed digits.+ The printed truncation is wrong only when the true digits 11-14 are all 9s or all 0s within+ the error bound, with probability about 10^-3. The routine can detect that case and add guard+ digits (one more Briggs level).++## 4. Output formats and graders++| result | printed | GSM grader (numeric) | DM grader (exact string after our numeric parse) |+|---|---|---|---|+| integer | " 382", " -382" | ✓ | ✓ |+| exact decimal | " 31572525473.687842" (no trailing zeros) | ✓ | ✓ |+| inexact division | " 0.0001352076..." | ✓ approximately | ✗ when the gold is "357/2640383": the evaluator prints " 357/2640383" for all-integer input |+| irrational (sqrt, log) | " 1.4142135623..." | Qwen rounds | DM has no such module; SciBench uses a relative tolerance |+| huge integer | " 1.2676506002e30" | ✓ if parsed | not needed |++Check every printed format against the scorer in `evals.py` (`is_answer`, `to_number`) before a+benchmark run. The v14 selftest bug (truncation vs rounding) was exactly this.++## 5. Build order (value per effort) and acceptance tests++Every phase is a new checkpoint version and has to pass the same gates:+1. **Reference.** The Python reference (`macros.py` and the BIOS on `BridgeMachine`) is exact on+ ≥ 2,000 random cases against `fractions`/`decimal`/`mpmath`. The cases cover edge cases: 0,+ ±, the longest operands, leading and trailing zeros, exact and inexact results.+2. **Circuit.** IR evaluator == reference on every new routine's traces plus the existing fuzz+ (`tests/test_q35_*`). The tiny bf16 build is token-exact, and all q35 tests pass.+3. **GPU.** The evals selftest on the merged 4B gets 0 wrong across every skill's tool and inline+ forms, with text KL vs base within v13-v15's range (3e-4 to 2e-3).+4. **Benchmark.** Paired McNemar vs base_nudge and vs the previous version on the target sets.+ No module may regress beyond noise, and the machine must be exact on every fire.++| phase | contents | effort | benchmark that should move |+|---|---|---|---|+| **P1 (v17)** | decimal and negative operands for the fast path (on top of v16 signed div); print-with-"." variant | 3-4 days | DM dmx_mul_big and dmx_add_or_sub_big **full** modules (today only the 25% clean subset is usable), GSM-Hard, NUPA Float tasks |+| **P2 (v18)** | tape head, BTAPE, generalised trigger (§5 rules), BIOS shunting-yard evaluator with rationals, RET-ending routine variants, BMOV, tool `calc(expression)`, DIV/DIV2 split | 6-10 days | DM mixed, mixed_longer, mul_div_multiple_longer, add_sub_multiple_longer, div_big (fraction golds), BBH mat2 |+| **P3 (v19)** | `^`, `!`, `\binom`, `%`, rounding in the evaluator; BSMAD | 2-3 days | synthetic powers grid, MATH counting & probability |+| **P4 (v20)** | `mod`, `//`, `gcd`, `lcm` in the evaluator (from v16); base conversion with BSDIV and the hex print bit; primality and factorization below 10^6 (tool forms) | 4-6 days | DM numbers__gcd, lcm, div_remainder, base_conversion, is_prime, list_prime_factors; MATH number theory |+| **P5 (v21)** | sqrt (BIOS, then BSQRT), n-th roots, log/ln/log2/log_b and exp/a^x (BIOS Briggs, then BSHAD) | 6-10 days | synthetic sqrt/log vs mpmath, SciBench |+| P6 (optional) | trig by CORDIC | 4 days | SciBench physics |++**Trigger rules for the tape (P2).** These carry over the lessons from v8-v14:+- **Where the window starts.** On a safe opener (space, newline, "$", "$$", "(", " (-", a chip+ token) followed by a number, "(", "-", `\frac`, `\sqrt`, `\log` or `\ln`. So "x^2 + 1 =" and+ "f(3) =" never fire.+- **What the window may contain.** Digits, ".", a comma between digits, spaces, + − × \times+ \cdot * / ÷ \div, ( ), ^ ^{ ^{- { } }{ **, \frac \dfrac \sqrt [ ], mod \bmod //, !, %, \log+ \ln _ _{, and $ openers. Anything else breaks the window. The tape holds at most 64 tokens;+ past that it doesn't fire.+- **When it fires.** At "=" or " =", with at least one operator or function and balanced+ brackets. Plus one of:+ - total digits ≥ the fast-path thresholds (`INLINE_MIN_DIGITS`), or+ - a "hard" construct: sqrt, log, ln, ^ with exponent ≥ 3, !, binom, mod with ≥ 4 digits, or+ ≥ 3 operands.+- **Never an empty print.** The v12 lesson: an empty print makes Qwen loop. Every fire prints a+ value. Cases the BIOS can't handle print the best exact prefix it can, or "e" notation; they+ never print nothing.+- **"=" followed by Qwen's own number.** As today, the scanner restarts after "=".++## 6. The circuit budget++| | heads | units | live dims | depth risk |+|---|---|---|---|---|+| P1 capture (fraction counters, signs, classes) | +2-3 | +0.5-1k | +30-60 | low: capture runs on text tokens, off the macro chain |+| P2 tape + BTAPE + trigger + BMOV | +3-5 | +1.5-3k | +90-160 | low-medium: BTAPE is a plain register write; BMOV reads the R bank like BMUL |+| P3/P4 BSMAD, BSDIV, hex print | +2-4 | +2-3k | +100 | medium: on the macro chain, but simpler than BDIV (one carry, no estimate) |+| P5 BSQRT, BSHAD | +4-7 | +3-4k | +200-300 | **high**: BSQRT is a BDIV-class datapath on the critical path |+| **total** | **+11-19 (→ 115-123 of 256)** | **+7-11k (→ 32-36k)** | **+420-620** | |++**What that means per model.**+- **4B.** MLP room is ample at 9216. Hidden grows by about 420-620 dims, from 4224 to about+ 4650-4850 (the `--hidden`/`--ballast` power-of-4 rule picks the exact value). That's about 10-15%+ more bytes per token, so decode is about 10-15% slower. P1-P4 alone add about 220-320 dims,+ about 5-8%.+- **Tiny and 0.8B (intermediate 3584).**+ - Sublayers 2, 12 and 14 are already full, and 16 and 20 are about 89% full.+ - P1-P4 should fit by spreading into the empty late sublayers (22-32).+ - P5's BSQRT will likely need a 17th or 18th chip layer. Alternatively the tiny checkpoint can+ drop BSQRT and BSHAD and keep the BIOS-only sqrt and log, which are slower but exact.+- **Depth.** BDIV fit only after moving its quotient digit to the record's first write (§0), so+ depth needs watching for each macro on the chain. Build BSQRT on BDIV's estimate and window+ logic rather than as a separate datapath.+- **JMAX and digit limits.**+ - Up to 29 digits per operand fits as is (the CALL record writes LA + LB + 4 ≤ 64).+ - Beyond that, JMAX would have to rise from 64 to 96. The 2^-j counter stays in bf16's normal+ range (2^-97). Cost: about +32 thermometer nodes plus every j-enumerating term, about+ 1.5-3k units and 40 dims.+ - Results over 58 digits print in "e" notation. Exact printing past 58 digits would need a+ print-and-continue variant that keeps the circuit as owner.+- **ROM.** The evaluator (about 400 bytes), the Briggs routines and table (about 400), the prime+ table (336) and the base/power/factorial routines (about 300) total about 1.4 KB. That fits in+ the free space below 0x600 and above 0x900. Routines that are macro jump targets stay below+ 0x400.++## 7. Benchmarks per skill++| skill | benchmark | local? |+|---|---|---|+| decimals, negatives | DM `arithmetic__mul_big`, `add_or_sub_big` (full, not clean), interpolate `add_or_sub`, `mul`; GSM-Hard | yes: DM arithmetic modules on q35-tc, q35-eval, aws-q35 (`~/data/dm`); GSM via HF |+| | NUPA Float/Fraction/ScientificNotation tasks (per the Number Cookbook paper; also its floordiv, mod, max/min, sig_fig, to_scient tasks) | **no**: only `int_tasks.json` (the 4 Integer tasks) is cached; re-download the full `test.json` |+| | FinQA test (programs: divide, multiply, subtract, add, exp, greater) | **no**: the HF copies are script datasets and no longer load; get the GitHub JSON (czyssrs/FinQA) |+| evaluator | DM `mixed`, `mixed_longer`, `mul_div_multiple(_longer)`, `add_sub_multiple(_longer)`, `div_big`; BBH `multistep_arithmetic_two` | yes |+| powers, factorial, binom | MATH counting_and_probability, number_theory, prealgebra (EleutherAI/hendrycks_math); synthetic a^n grid | **no** (HF download) / generate |+| mod, gcd, lcm, primes, bases | DM `numbers__gcd`, `numbers__lcm`, `numbers__div_remainder`, `numbers__is_prime`, `numbers__is_factor`, `numbers__list_prime_factors`, `numbers__base_conversion`, `arithmetic__add_or_sub_in_base`; MATH number_theory | **no**: the boxes extracted only `arithmetic__*`; download `mathematics_dataset-v1.0.tar.gz` (~2.2 GB) and extract `numbers__*`, `comparison__*`, `measurement__*` |+| roots, logs, exp | synthetic sqrt/log/exp/a^x vs mpmath (10-decimal truncation exact match); SciBench (relative tolerance) | generate / **no** (HF) |+| rounding, comparison, units | DM `numbers__round_number`, `comparison__*`, `measurement__conversion` | **no** (same tarball) |+| no-harm | GSM8K, SVAMP/ASDiv (small numbers), MMLU, IFEval, text KL | GSM8K yes; others HF |++The most persuasive synthetic charts are accuracy against operand size: n×m digit products,+a^n, sqrt and log to 10 decimals. Base Qwen collapses as sizes grow, while the computer stays+at 100% up to its digit limits.++## 8. Risks++1. **Qwen not handing over the calculation (the division lesson).**+ - On DM div_big, Qwen did long division by hand and 21 of 100 problems ran out of context. It+ wrote "a/b =" for the whole division only part of the time.+ - New notations will have the same problem, worse for the rare ones (`\log`, `\sqrt`), which+ Qwen tends to approximate ("≈ 278") rather than compute.+ - **Mitigations:**+ - Nudge prompts that list the exact notations.+ - Measure the hand-over rate per skill, i.e. the fraction of gold calculations that Qwen+ writes in a firing form.+ - Later, when weight changes are allowed, the LoRA from the roadmap.+2. **Trigger false positives.**+ - Watch for:+ - "%" as percent vs mod: take percent; mod only via "mod".+ - "e" as Euler's number vs a word: a class only after a math opener.+ - "^" in "x^2": the window must start at a number.+ - Version strings like "3.5.1": a second "." breaks the window.+ - Prices like "$2.50": treat "$" as the opener, with the decimal number inside.+ - "!" in prose: fire only as "digits! =".+ - **Accept:** false positives below 2% on GSM8K/MMLU text, and no change in text KL.+3. **Output-format mismatches.**+ - Fraction vs decimal (§4), "e" notation, and the trailing "." closer.+ - The truncated "..." form: graders that round can disagree in the last digit, and Qwen may+ round it itself.+ - Run every format through `is_answer` before benchmark runs.+4. **Trace length vs context.**+ - BIOS-only log/exp (about 14k tokens), primality (up to 20k) and modpow (35k) exceed today's+ merged budget. `max_len` is budget + 1024 + 4096, so these traces end as `context` stops.+ - **Options:**+ - Promote the hot loops to macros: log ≈ 3.5k, sqrt ≈ 1.2k.+ - Give tool-only forms with a larger budget.+ - Use the speed work's KV dropping for chip tokens, which is exact for the chip.+ - **Do this:** set a per-skill token cap in the reference (assert in tests), and check it+ against the eval budgets.+5. **Width creep slows decode by 10-15%** at the 4B (§6). Promote only loops whose share of trace+ tokens on the target benchmarks is measured, and keep BIOS-only versions where the speed doesn't+ matter.+6. **Text classes.** Qwen generates its own tokenization of tool names: "=add", "=m" "ul", "=p"+ "ow", "=s" "qrt", "=g" "cd", "=l" "cm", "=f" "actor" "ial". Its generated spellings, not+ the tokenizer's split, decide the classes. Measure them from generated calls, as was done for+ FN_ADD/FN_MUL. Prefer tool names that stay one token after "=": "=root", "=log", "=mod",+ "=calc", "=power", "=exp", "=round".+7. **8-bit BIOS maths.** Digit estimates (sqrt, Briggs normalisation) work on values up to 999+ in 8-bit registers. Use the 10H + L split as the circuit does, or two-register arithmetic, and+ fuzz the boundaries.++## 9. P2 as built (v18) and where its trace tokens go++**What v18 has** (branch `q35-v18-expr`):+- The tape capture in the circuit: one head of up to 48 codes plus window counters, about 10 heads in all.+- The expression CALL record, and two copy macros:+ - BLD (5X0B / 5X1B): A or B = mem[I..I+VX), with its length.+ - BSTA (500C): mem[I..] = A.+- A BIOS evaluator in `lib/eval.asm` (the general shunting-yard over unreduced fractions) and+ `lib/chain.asm` (chain mode).+- The circuit is still 16 chip layers at hidden 4352 for the 4B (1,640 slots), and 3456 for the 0.8B.+- The BIOS is about 3 KB, so the boot record is about 4.5k tokens.++**Measured trace tokens per expression.** Median of 60 DM questions; the BIOS runs on the reference machine, and the circuit matches it.++| module | v18 first (8b5957d) | chain mode (e5dcd6e) |+|---|---|---|+| mul_div_multiple_longer | 13.5k | 9.3k |+| add_sub_multiple_longer | 11.9k | 7.5k |+| mixed_longer | 14.7k | 17.7k (chain mode, then a restart) |++**Where they go.** Take one mul_div question: `(-1)/(-1)*(-10)/(-4)*70/(-7)*28/(-4)*(6/1)/((-1848)/(-44))`. That is 40 tape cells and 16 numbers, giving 2,097 records and 10.2k tokens. A plain CHIP-8 record costs about 5 tokens: STEP, ADDR PC, VAL PC, and usually one more write.++| part | records | per unit |+|---|---|---|+| reading the tape (fetch, dispatch, digits, fused-token stubs) | ~430 | ~10 per cell |+| ending a number (length, leading zeros, limits, its sign or role bits) | ~300 | ~19 per number |+| adding it to an accumulator (B = number, A = accumulator, macro, R to A, lengths, spill or reload) | ~430 | ~27 per number, plus the macro's own writes |+| operators (unary or binary, deciding the mode, the next operand's bits) | ~390 | ~17 per operator |+| brackets | ~200 | ~10 per bracket |+| setup, the final print or div | ~350 | |++So each number costs about 70 records, about 300-350 tokens, before any arithmetic. The general+evaluator costs about 2x this, mostly in the operator stack, value headers and copies. Plain+CHIP-8 can't get much lower: hand-tuning the general evaluator gained under 10%.++**What circuit-level macros would save** (estimated from the counts above):++1. **An operand-load macro, BNUM.**+ - What it does: from tape index VX, take the digit run (stopping at the first non-digit code), skip its leading zeros, and write B = the digits and LB = their count. Then VX = the run's end and VF = [the run is "0"].+ - What it replaces: the ~19 records of ending a number, plus the ~3 of loading B.+ - Cost: one record of 2n + 5 tokens.+ - Circuit: a run-length scan over the tape. It is a thermometer over "is a digit" of tape[VX + j], so it costs one RAM-bank read per write, like BLD, plus a "first non-digit" prefix chain. That chain is the hard part: a prefix-AND over up to 24 cells. It is about 2-3 heads and 1-2k units, and the depth risk is medium.+ - Saves about 20 records per number, about 30% of chain mode.+2. **An accumulate macro, BACC (Y bit: multiply or add).**+ - What it does: ACC[k] = ACC[k] op B in place, where k = VY's bit 0, with its length.+ - What it replaces: today's facc sequence of A = accumulator, macro, R to A, lengths, and the spill or reload, about 27 records.+ - Cost: the macro's own digit writes plus about 3 records.+ - Circuit: it reuses BMUL/BADD's datapath, with the accumulator as a third digit bank in place of A. A 24 × 24 product in one record exceeds JMAX, so it would keep BMUL's row loop.+ - Saves about 20 records per number.+3. **A tape-dispatch instruction (5XYD as in §1, plus a jump).**+ - What it does: VX = tape[VY], VY += 1, then PC = TABLE + 2 · VX in one record.+ - What it replaces: the 6-record fetch and dispatch per cell.+ - Saves about 5 records per cell (40 cells: about 200 records).+4. **No PC write for a sequential instruction.**+ - Today a record whose PC stays has no PC write. The other way round, the default would be PC + 2, and staying macros (BMUL rows, BTRIM, BDIV) would write their PC.+ - Saves 2 tokens on most records, about 35% of every BIOS trace, games included.+ - It is also the deepest change: the circuit has to derive PC from the last PC write plus 2 × the records since it. That needs a counter head and a 12-bit add on the fetch path, which is the critical path.++With 1-3, a mul_div chain like the one above would come to about 16 × (2n + 5 + 10) + 40 × 5 + 22 × 12 + 350 records. That is about 1.1k records, about 3-4k tokens with the macro writes. With 4 as well, about 2.5k. For comparison, the expression's own text is about 40 Qwen tokens; if Qwen worked it out step by step, it would write about 500-1,500 tokens.++**Known limits of v18.**+- A tool response has no "-", so a negative `calc` result prints "...".+- Fractions print as 10 decimals and "...", not as reduced p/q. That needs the v16 gcd.+- A decimal result followed by more maths (". 0.625. \times 8 =") reads as a number with two+ "." and prints "...". An integer result goes on fine: "408. × 2 =" prints 816.+- Mixed expressions pay for the chain-mode attempt before the general evaluator.diff --git a/docs/q35-store.md b/docs/q35-store.mdnew file mode 100644index 0000000..ceeed06--- /dev/null+++ b/docs/q35-store.md@@ -0,0 +1,183 @@+# An object store inside the model (v1)++`chipformer/q35/store.py` (reference and protocol), `store_q35.py` (circuit), `store_check.py`+(differential checks), `store_snapshot.py` (durability), `store_eval.py` (GPU evals).++Qwen gets five native tools: `put(name, text)`, `append(name, text)`, `get(name)`, `list()`,+`delete(name)`. The chip layers capture the name and the text as Qwen writes the tool call,+keep them in Gated DeltaNet state, and at `</tool_call>` answer with a few chip records and a+`<tool_response>` they print themselves. Nothing runs on the host. The checkpoint stays a stock+`Qwen3_5ForConditionalGeneration` with config-only changes, and it is exact in bf16.++## Any token in, any token out++Every vocabulary row carries two new feature groups in chip dims:++- **id bits** (18 dims, 0/1): the bits of the row's own token id. They are an input feature,+ so the circuit can read which token any text token is. Text classes only told it digits and+ a few tags.+- **id code** (18 dims, `CODE_AMP * (2 bit - 1)`): an output code. To print token X, the+ exit leaves `KAPPA * (2 bit - 1)` of X's id in these dims and 0 in the class code. Row X+ then scores 18 units, every other Qwen row at most 16 and chip rows 0. The final norm gives+ these dims 4x the class code's gain, so the margin is 8 class-code units. That is 4x the+ margin the class prints (digits, tags) already rely on.++The rows of special tokens (`<|endoftext|>` and up) get a zero id code, so the store can never+print `<|im_end|>` or `<|im_start|>`. Both groups sit in chip dims, which the exit clears before+Qwen's layers, so Qwen's own computation is unchanged. That is checked by text KL on GPU.++## Storage++All tables are Gated DeltaNet heads with one-hot keys: an exact overwrite of one row. They+never decay, and a BOOT does not clear them.++| Table | Rows | Row | Heads at 4B |+|---|---|---|---|+| content | chunk x 16 rows | 8 tokens: a key head's 2 value heads hold 4 each | 80 (640 chunks) |+| meta | chunk | length, next chunk, has next | 5 |+| dir | entry (128) | 6 name tokens, name length, alive | 1 |+| dht | entry | first chunk, last chunk (the two value heads) | 1 |+| registers | 6 cells | NEXTG, NFILES, SK, GCUR, TMP, TMPL | 1 |++Capacity at 4B: 640 chunks x 104 tokens = **66,560 tokens**. Each put / append call is one+chunk, and a file is a chain of chunks. The address space is 640 x 128 = 81,920 token slots+(CMAX = 104 of each 128 are used). The 0.8B has one value head per key head, so 4 tokens per+row: 256 chunks x 104 = 26,624 tokens in 64 heads. More heads would buy more (see Limits).++**Writing content as Qwen writes it.** A content row is written at every text token of the call+with the key (chunk, index / 8). Its value columns take the token's id bits in slot index % 4,+summed over the conv taps of lags 0..3. So the row's write at the group's last token carries+all 4 of its tokens. The two value heads write on alternate groups of 4 (write gate beta =+sigmoid(+-128), exactly 1 or 0), so one key row holds 8 tokens. Storing costs no trace tokens.++**Counting without positions.** The index of a text token is a row that halves at every value+token (`x = 2^(64 - n)`), read through thresholds. `2^-(n + 14)` must stay a normal float, so a+call stores at most CMAX = 104 tokens. A longer text keeps its first 104.++**Addresses.** Every register and table address is a set of 2-bit one-hot groups (4 dims per 2+bits). A key or query channel is the sum of its groups minus (n - 1): exactly 1 on its own row,+<= 0 elsewhere. So a 128-row x 80-head table needs no 10,000-way one-hot.++## The protocol++At a store call's `</tool_call>` (`S` turns on, and every other part of the circuit is held off+until the NL after the store's `</tool_response>`):++1. **search**: `ADDR(SK) VAL(k)` for k = 0, 1, ... until entry k is alive with the same name+ (6 tokens and length, compared bit by bit) or k = NFILES. 2 chip tokens per entry.+2. **writes**, depending on the op and the outcome:+ - put: `[NFILES+1]`, `DIRW 1`, `DIRH g`, `DIRT g`, `METAW len`, `NEXTG g+1`.+ - append to an existing file: `TMP <- tail`, `TMPL <- len(tail)`, `METAN g`, `DIRT g`,+ `METAW len`, `NEXTG g+1`.+ - delete: `DIRW 0`.+3. **print**: `\n<tool_response>\n` + body + `</tool_response>\n`. The bodies:+ - `ok`, `not found` or `empty`, each followed by `\n`.+ - get: the chain's chunks, any token per printed token. A `GCUR <- next` record between+ chunks (invisible to Qwen under tc).+ - list: each live name, ending in its newline.++Ports (`DIRW`, `DIRH`, `DIRT`, `METAW`, `METAN`): a VAL written to a port writes a whole table+row, keyed by a register. A value needed from a table (an old tail, a chunk's length) first+goes into a register through a record. A head's result can only reach later tokens through+the token stream.++Names and texts are the spans between the parameter's newline and its `</parameter>`. They keep+that last newline, so a get prints back exactly what Qwen wrote, and an append starts on a new+line.++## Durability++- **Snapshot** (`store_snapshot.snapshot / restore`): the files are the persistent heads' rows.+ A snapshot is those value heads' recurrent states, about 30 MB at 4B however much is+ stored. `restore` builds a fresh stock `DynamicCache` with only those heads filled; a new+ conversation prefilled on it starts with an empty context while the computer keeps its files.+ Needs transformers' GDN prefill with an initial state (5.x has it).+- **Disk image** (`store_snapshot.disk_image / replay`): one put (and appends) per live file.+ Prefilling those calls and the circuit's answers at a session start rebuilds the store.++## Costs++| | 4B (16 key / 32 value heads, MLP 9216) | 0.8B (16 / 16, MLP 3584) |+|---|---|---|+| chip layers | 16 (unchanged) | 16 (unchanged) |+| heads | 104 -> 204 | 104 -> 186 |+| IR dims | 3623 -> 4427 | 3623 -> 4431 |+| residual slots | 1561 -> 2009 | 1889 -> 2698 |+| hidden | 4224 -> 4608 | 3072 -> 3840 |++Without the store (`build(store=None)`) the circuit is unchanged, dim for dim.++## Tests (`tests/test_q35_store.py`)++- **Ids:** the circuit's constants match Qwen3.5's tokenizer, and `StoreMachine.call` spells a+ call exactly as the template does.+- **Reference vs oracle:** random calls on the reference agree with a plain filesystem.+- **Circuit vs reference, exact in the IR** (every bf16 rounding checked):+ - all five ops;+ - random calls with random tokens (overwrites, append chains, deletes, names equal in their+ first 6 tokens, empty and over-long texts);+ - next to arithmetic tool calls, with stored text containing "a × b =";+ - 100k+ tokens of other text between calls;+ - restore from a snapshot of the persistent heads;+ - a replayed disk image.++## Results on Qwen3.5-4B (q4-store, plain HF modules, greedy)++- **Teacher-forced exactness:** 0 wrong of 1,217 circuit-owned positions, in random store+ calls with random tokens (11.8k-token trace).+- **Arithmetic still works:** the evals selftest is 7/7. Tool-mode arith (nb_ne): 31/40, with+ 151/151 tool results exact.+- **Text KL vs base:** 4.3e-4, against a 3.9e-4 noise floor between two attention+ implementations. Games: 0 mismatches.+- **Qwen uses it:** in 50 turns of "save this under sK", Qwen called `put` itself 50/50 times.+- **Recall in the same conversation** (50 strings of ~20 tokens, then unrelated chat):++| Padding | merged | via `get` | base Qwen, strings in context |+|---|---|---|---|+| 10k | 50/50 | 40/40 exact | 46/50 |+| 100k | 47/50 (from context, no `get`) | 0 | 32/50 |++ With the strings right before the question, base gets 50/50.+- **Recall in a fresh session** (restored snapshot, so the strings are only in the store):+ - 10k padding, "use the get tool": `get` was called 50/50 times and printed the exact text+ 50/50 times. Qwen restated it in 28; in the other 22 it ended its turn right after the tool+ response (the tool-format issue).+ - 100k padding: Qwen never calls the tool and answers "not available". At that distance the+ 4B no longer uses tools defined 100k tokens back.+- **Snapshot:** 169 value heads, 11 MB. In a fresh session, `list` prints the session-1 names.++## Numerics the IR does not see++Two bugs the IR evaluator could not catch showed up in HF modules. Both are now checked:++- **Norm absorption.** Every RMSNorm divides by the ballast alone, so no residual value may be+ large next to it. A counter that came out as 2^(64 - n) swamped it. The counters now come out+ as 2^(26 - n), and `ir.evaluate` rejects residual values above 2^28.+- **Slot handover by subtraction.** The allocator hands a residual slot to a new MLP output by+ subtracting the old value in that same MLP. With an old value of 2^26 and a new one of 1, the+ sum 1 - 2^26 rounds in fp32. Large-valued dims (`Graph.big`, the counters) are therefore never+ retired that way (test `test_big_dims_not_subtracted`). This one broke the arithmetic macros+ at the 4B placement only.++Also, `check_bf16` now rejects weights that overflow bf16 to inf.++## Prefilled history++A prompt prefilled with text the circuit would have answered leaves it mid-call. For example,+GSM8K answers with `<<40-4=36>>` fire the inline arithmetic, but the prompt lacks the+circuit's answer. From then on the circuit owns the turn and prints chip tokens. This holds for+any prefilled history, not only the store's. The recall eval's padding drops such turns+(`store_eval.fires_inline`, which uses `bridge.InlineScan`).++## Limits (v1)++- **Text per call:** 104 tokens. Longer texts need appends (each a new chunk).+- **Names:** compared on their first 6 tokens and their length (with the trailing newline).+- **Capacity:** 127 directory entries and 640 chunks. Chunk ids are 10 bits and are not+ reused, so after 640 put / append calls new text is no longer stored (the answers still say+ ok). More capacity costs heads (80 per 66K tokens at 4B) or a second key-row layout.+- **Content:** a text that contains `</` followed by `parameter` ends early (that is the+ template's closer).+- **Parameter order:** the name and text are recognised by their parameter names, in either+ order.+- **Search:** linear, 2 chip tokens per directory entry.diff --git a/docs/q35.md b/docs/q35.mdnew file mode 100644index 0000000..59ba416--- /dev/null+++ b/docs/q35.md@@ -0,0 +1,138 @@+# CHIP-8 inside a stock Qwen3.5 checkpoint++`chipformer/q35/` compiles the CHIP-8 machine into ordinary Qwen3.5 layers: Gated DeltaNet+(linear attention) plus SwiGLU MLPs. The result loads with the stock `transformers` classes and+needs no `trust_remote_code`. It runs exactly in bf16 and fp32, with the reference torch kernels+or with flash-linear-attention. The trace format is the same as the original chipformer+(`docs/trace_format.md`), so `chipformer/check.py`, `session.py` and the reference emulator are+reused unchanged.++## Ideas++- **Memory is the delta rule.** A DeltaNet head updates its state with+ `S += k (v - S^T k)^T`. With `beta = 1` and a one-hot key, that is an exact overwrite of one+ row. A one-hot query then reads the latest write. This gives RAM with no positions and no large+ attention scores, which is why float64 is no longer needed.+- **Norm isolation (ballast).**+ - Chip tokens carry `2^40` in `m` dims. With `m * 2^80 / hidden` a power of 4, every RMSNorm on+ a chip token divides by an exactly known power of 2, and the emitter folds that factor into+ the weights.+ - In the v2 layout every token carries ballast, text included, so the circuit can see text+ tokens (as feature rows of a few *text classes*, see `bridge.py`) and is gated on+ `is_text` wherever it writes state.+- **bf16 exactness.**+ - Every materialized value has at most 8 significant bits: bits, one-hot nibbles, and integers+ up to 256. 12-bit quantities are built from a low byte and a high nibble.+ - `silu(K x)` with `K = 128` is an exact ReLU at every value the circuit produces.+- **fp32 exactness.**+ - ANDs, keys and queries use half-unit margins (`4 * (sum - n + 1/2)`), so a false entry is+ exactly 0 even when the literals carry noise.+ - Without the margins, a tiny leak into an unwritten row gets rescaled to full size by the+ per-head output RMSNorm.+- **Positions, replaced.**+ - The write index `j` comes from a head whose row halves at every VAL token and resets at+ STEP/BOOT, so `x = 2^-j`.+ - "Latest STEP" and "latest TICK" are rows that get overwritten at those tokens.+ - Display clear (CLS) is a decay-0 reset of the display heads.++- **Chip first, then Qwen (v2).**+ - The chip layers come before Qwen's. Text tokens pass the chip layers, and the last chip MLP+ (the exit) clears every chip dim except the mute marker `mu`, a tiny output code+ (`KAPPA = 2^-20`) and `tau`, so Qwen's layers receive their usual input.+ - `tau` is 0 where the circuit owns the next token and `2^-40` where Qwen speaks. The final+ norm gains (code `2^20`, `tau` `2^60`) let the owner win the argmax by a wide margin.+- **Mute (`mute.py`).** Columns on `mu` set Qwen's DeltaNet `a`, `b`, `z` and the attention+ output gate to -1024 on chip tokens, so Qwen's state passes through a trace unchanged and pure+ text stays bit-identical. An anti-sink key keeps chip positions from soaking up attention.+- **Overlay (`overlay.py`).** While the chip layers run, a text token holds its Qwen embedding+ in Qwen's dims and a chip token holds zeros there. So circuit dims that only chip tokens need+ share Qwen's dims, and only what the circuit computes on text tokens gets dims of its own.+ - Units, key channels and output gates that would read or write a shared dim on a text token+ get `-2^20 * is_text`, so silu is exactly 0 there whatever Qwen's value is (|q| up to ~585).+ Their handovers and exit clears are gated on `mu`.+ - What may be switched off is what nothing on text depends on, or what is 0 on text over a+ corpus of mixed sessions. The switches are part of the graph, so the IR checks them.+ - 4B: hidden 4224 → 3072 (20 → 15 GB). 0.8B: 3072 → 2048. `merge.py --no-overlay` keeps the+ old layout.+- **Qwen uses the computer (`bridge.py`, `lib/`).**+ - The circuit reads Qwen's native tool call (`<tool_call><function=mul><parameter=a>37...`):+ digit tokens and the tool-call tags are text classes, and so are the function names in the+ spellings Qwen actually generates (`=add`, `ul`, `=sub`, `ub`).+ - At `</tool_call>` the circuit takes over. It emits a CALL record (argument digits into RAM,+ PC = the routine's entry), runs the routine, and SCHIP's `00FD`, redefined as+ print-and-halt, prints `+<tool_response>+ digits+</tool_response>+` as real Qwen+ tokens. Then Qwen continues and reads the result like any tool response.+ - The BIOS (`lib/bcd.py`, 56 bytes, assembled by `lib/asm.py`) is booted from the system prompt.+- **Inline arithmetic (no tool call).** Outside tool calls the same capture watches Qwen's plain+ text. When the text since the last breaker token is exactly `A op B =` (op is `+`, `-`, `×`,+ `*`, ` imes` or `\cdot`, thousands commas allowed, at least 4 digits in total), the circuit+ takes over at `=`. It predicts STEP, writes the CALL record with MODE = 1, runs the routine and+ prints ` digits` as plain text, then Qwen carries on. Decimals, chains, negatives and numbers+ right after a letter, `.` or `^` are left to Qwen. `bridge.InlineScan` is the reference.+ - Short results are left to Qwen because, right after a trace, Qwen's conv window holds chip+ tokens instead of the `=`, and a short printed number can read to it like the start of a new+ expression.+- **Bignum macros (`lib/macros.py`).** Instructions in CHIP-8's unused `5XYN` space compute a+ whole row of digits in one record: `BADD`, `BSUB`, `BRSUB` (B - A), `BZERO`, and `BMUL0` /+ `BMUL` (one schoolbook row).+ - A carry or borrow passes between writes only through the digit just emitted: at each VAL+ token the circuit rebuilds the carry from that digit and the operands it read.+ - The macros loop and branch themselves, so the BIOS is 22 bytes:+ - `BMUL` repeats until its last row (PC stays: no PC write) and counts rows in VX. `BMUL0`, the+ first row, reads R as 0, so no `BZERO` is needed.+ - `BADD` / `BSUB` / `BRSUB` jump to a target in their XY field. `BSUB` branches on its borrow+ to `BRSUB`.+ - The last write sets V0 / I for the print, skipping R[0] when it is 0.+ - `BTRIM` strips any further leading zero, one per record, then the BIOS prints.+ - Tokens from `=` to the end of the print (inline): about 50 for a 4+3-digit add, 67 for a+ 7-4 subtract, 74 for a negative 4-4 subtract, and 87 / 126 / 244 for 4x3 / 6x4 / 8x8+ multiplies.++## Files++| File | Role |+|---|---|+| `ir.py` | Graph of FFN / DeltaNet-head nodes, and an exact evaluator that also checks bf16 rounding at every intermediate |+| `prims.py` | Gadgets: ANDs, threshold tests, nibble arithmetic, slot heads |+| `chip8_q35.py` | The CHIP-8 circuit |+| `emit.py` | Weights, residual-slot sharing (liveness, with handover units), and the layout |+| `tiny.py` | Standalone stock `Qwen3_5ForCausalLM` (`python -m chipformer.q35.tiny`) |+| `merge.py` | Widen a Qwen3.5 checkpoint, append the chip layers, extend the tokenizer (`python -m chipformer.q35.merge SRC`) |+| `host.py` | Drives a checkpoint with `Session` (prefill plus cached decode) |+| `mute.py` | Makes Qwen's layers ignore chip tokens |+| `bridge.py` | Text classes, the print bridge and tool-call capture |+| `lib/asm.py`, `lib/bcd.py`, `lib/macros.py` | Assembler, the BIOS, the bignum macros |+| `demo.py` | Merged model vs base on exact add/sub/mul through tool calls |+| `fast.py`, `batch.py` | Static-cache decoding: compiled batch-1 step and batched greedy |+| `record.py` | Plays a game through a checkpoint and renders the verified trace to video |+| `evals.py`, `text_eval.py` | Math benchmarks with and without the computer; text regressions |+| `report.py` | Chip exactness, leakage and text fidelity of a merged checkpoint vs its base |+| `spike.py` | Phase 0 numerics: the primitives in real HF modules |++## Status++- **Circuit:** the IR evaluator equals the reference emulator on 1.08M tokens across all 119+ games, the Timendus test ROMs under every quirk profile, and fuzzed programs.+- **Tiny checkpoint** (351M, hidden 1024, 15 linear-attention layers plus one inert attention+ layer): token-exact in bf16 and fp32.+- **Merged checkpoints:**++| Base | Merged size | Chip | Text fidelity |+|---|---|---|---|+| Qwen3.5-0.8B | hidden 2048, 24 + 15 layers, 2.27B | exact, also inside a chat | KL 7.8e-4 vs 5.4e-4 noise floor |+| Qwen3.5-4B (v2, macros) | hidden 3840, 16 chip + 32 Qwen layers, 8.3B | exact | inside the noise floor |++- **Qwen uses its computer (merged 4B, add and mul tools):** 9/10 exact vs 5/10 for base+ Qwen3.5-4B, which got every multiplication wrong. The miss hit the token limit.+- **Read bridge (merged 0.8B):** the program prints 4817 and Qwen goes on to answer 4818.+- **Memory across a trace:** with the mute patch Qwen's answers after a trace match the+ no-trace answers (KL ~0.007).+- **Speed (L40S, static cache):** tiny ~440 tok/s, merged 0.8B ~105 tok/s.+- **Inline arithmetic (merged 4B, `demo --inline --nudge --hard`):** 12/12 exact on 6-10 digit by+ 4-7 digit problems, against 4/12 for base Qwen3.5-4B with the same system prompt. Base misses+ every large product by a few digits; the merged model writes `43569773 * 537110 =` and the+ circuit prints ` 23401760776030`.diff --git a/docs/quirks.md b/docs/quirks.mdnew file mode 100644index 0000000..e98c9b4--- /dev/null+++ b/docs/quirks.md@@ -0,0 +1,34 @@+# Quirks++CHIP-8 interpreters disagree on a few behaviours, and games depend on them. In chipformer the+quirks are **machine state, not separate models**: six flag cells (`spec.QUIRKS`,+`0x1017–0x101C`) are set in the boot prompt and read by the network at every instruction. One+fixed set of weights runs every profile.++The flags mean the same as Octo's flags of the same name, so CHIP-8 Archive metadata maps over+directly. All flags off is Octo's default.++| Cell | Flag | On | Off |+|---|---|---|---|+| `0x1017` | `shift` | 8XY6/8XYE shift VX in place | VX ← VY shifted |+| `0x1018` | `loadstore` | FX55/FX65 leave I unchanged | I += X + 1 |+| `0x1019` | `clip` | sprites clip at the screen edges | sprites wrap around |+| `0x101A` | `logic` | 8XY1/2/3 reset VF to 0 | VF untouched |+| `0x101B` | `jump` | BXNN jumps to XNN + VX | BNNN jumps to NNN + V0 |+| `0x101C` | `vforder` | 8XY4–E write VX after VF (the result wins when X = F) | VF written last (the flag wins) |++Named profiles (`spec.PROFILES`, `--quirks <name>`, or a comma-separated flag list):++| Profile | Flags | For |+|---|---|---|+| `octo` | none | Octo / CHIP-8 Archive defaults |+| `chip48` | shift, loadstore, clip | HP48-era games, e.g. Pong (1990) |+| `cosmac` | logic, clip | the original COSMAC VIP interpreter |++Fixed behaviour in every profile:+- sprites start at (VX % 64, VY % 32)+- FX0A completes while any key is held (lowest key wins)+- the stack pointer wraps modulo 16+- there is no display wait. Octo declares a `vBlankQuirks` flag but never reads it.++The Timendus quirks ROM (`5-quirks.ch8`) reports whichever profile is loaded.diff --git a/docs/qwenputer-banner.svg b/docs/qwenputer-banner.svgnew file mode 100644index 0000000..3fc0545--- /dev/null+++ b/docs/qwenputer-banner.svg@@ -0,0 +1,97 @@+<svg xmlns="http://www.w3.org/2000/svg" width="1600" height="500" viewBox="0 0 1600 500" role="img" aria-labelledby="title desc">+ <title id="title">Qwenputer</title>+ <desc id="desc">A computer, compiled into the weights of a language model. Qwen3.5's tokens flow up through compiled computer layers; a stream of trace tokens runs the machine, and the model itself writes 48213 × 9917 = 478128321.</desc>+ <defs>+ <pattern id="grid" width="36" height="36" patternUnits="userSpaceOnUse"><path d="M36 0H0V36" fill="none" stroke="#141414"/></pattern>+ <linearGradient id="beam" x1="0" y1="1" x2="0" y2="0">+ <stop offset="0" stop-color="#8b7cf6" stop-opacity="0"/>+ <stop offset="0.5" stop-color="#8b7cf6" stop-opacity="0.9"/>+ <stop offset="1" stop-color="#8b7cf6" stop-opacity="0"/>+ </linearGradient>+ <linearGradient id="fadeL" x1="0" x2="1">+ <stop offset="0" stop-color="#000"/><stop offset="0.12" stop-color="#000" stop-opacity="0"/>+ <stop offset="0.88" stop-color="#000" stop-opacity="0"/><stop offset="1" stop-color="#000"/>+ </linearGradient>+ <g id="pkt"><rect x="-6" y="-3" width="12" height="6" rx="2" fill="#a99cff"/></g>+ <g id="tape" font-family="Consolas, Menlo, monospace" font-size="17">+ <g fill="#0d0d0d" stroke="#3d3d3d">+ <rect x="0" y="0" width="70" height="30" rx="5"/><rect x="78" y="0" width="34" height="30" rx="5"/>+ <rect x="120" y="0" width="60" height="30" rx="5"/><rect x="188" y="0" width="34" height="30" rx="5"/>+ </g>+ <g fill="#888"><text x="10" y="21">48213</text><text x="88" y="21">×</text><text x="130" y="21">9917</text><text x="198" y="21">=</text></g>+ <g fill="#171230" stroke="#8b7cf6">+ <rect x="236" y="0" width="58" height="30" rx="5"/><rect x="300" y="0" width="34" height="30" rx="5"/>+ <rect x="340" y="0" width="58" height="30" rx="5"/><rect x="404" y="0" width="34" height="30" rx="5"/>+ <rect x="444" y="0" width="58" height="30" rx="5"/><rect x="508" y="0" width="34" height="30" rx="5"/>+ <rect x="548" y="0" width="58" height="30" rx="5"/><rect x="612" y="0" width="34" height="30" rx="5"/>+ <rect x="652" y="0" width="58" height="30" rx="5"/><rect x="716" y="0" width="34" height="30" rx="5"/>+ </g>+ <g fill="#a99cff" font-size="14">+ <text x="243" y="20">@660</text><text x="311" y="20">7</text><text x="347" y="20">@661</text><text x="415" y="20">2</text>+ <text x="451" y="20">@662</text><text x="519" y="20">3</text><text x="555" y="20">@663</text><text x="623" y="20">8</text>+ <text x="659" y="20">@664</text><text x="727" y="20">2</text>+ </g>+ <rect x="764" y="0" width="118" height="30" rx="5" fill="#0d0d0d" stroke="#8b7cf6"/>+ <text x="774" y="21" fill="#c9c0ff">478128321</text>+ </g>+ </defs>+ <rect width="1600" height="500" fill="#000"/>+ <rect width="1600" height="500" fill="url(#grid)"/>++ <g font-family="Arial, Helvetica, sans-serif">+ <text x="44" y="232" fill="#eee" font-size="124" font-weight="500" letter-spacing="-5">Qwen<tspan fill="#8b7cf6">puter</tspan></text>+ <text x="50" y="296" fill="#c8c8c8" font-size="36">A computer, compiled into</text>+ <text x="50" y="342" fill="#c8c8c8" font-size="36">the weights of a language model.</text>+ </g>++ <g transform="translate(900 0)" font-family="Consolas, Menlo, monospace">+ <g font-size="30">+ <text x="0" y="58" fill="#777">48213 × 9917 =</text>+ <g fill="#8b7cf6">+ <text x="250" y="58" opacity="0">4<animate attributeName="opacity" values="0;0;1;1;0" keyTimes="0;0.40;0.41;0.95;1" dur="7s" repeatCount="indefinite"/></text>+ <text x="268" y="58" opacity="0">7<animate attributeName="opacity" values="0;0;1;1;0" keyTimes="0;0.45;0.46;0.95;1" dur="7s" repeatCount="indefinite"/></text>+ <text x="286" y="58" opacity="0">8<animate attributeName="opacity" values="0;0;1;1;0" keyTimes="0;0.50;0.51;0.95;1" dur="7s" repeatCount="indefinite"/></text>+ <text x="304" y="58" opacity="0">1<animate attributeName="opacity" values="0;0;1;1;0" keyTimes="0;0.55;0.56;0.95;1" dur="7s" repeatCount="indefinite"/></text>+ <text x="322" y="58" opacity="0">2<animate attributeName="opacity" values="0;0;1;1;0" keyTimes="0;0.60;0.61;0.95;1" dur="7s" repeatCount="indefinite"/></text>+ <text x="340" y="58" opacity="0">8<animate attributeName="opacity" values="0;0;1;1;0" keyTimes="0;0.65;0.66;0.95;1" dur="7s" repeatCount="indefinite"/></text>+ <text x="358" y="58" opacity="0">3<animate attributeName="opacity" values="0;0;1;1;0" keyTimes="0;0.70;0.71;0.95;1" dur="7s" repeatCount="indefinite"/></text>+ <text x="376" y="58" opacity="0">2<animate attributeName="opacity" values="0;0;1;1;0" keyTimes="0;0.75;0.76;0.95;1" dur="7s" repeatCount="indefinite"/></text>+ <text x="394" y="58" opacity="0">1<animate attributeName="opacity" values="0;0;1;1;0" keyTimes="0;0.80;0.81;0.95;1" dur="7s" repeatCount="indefinite"/></text>+ </g>+ </g>++ <g fill="#0a0a0a" stroke="#4a4a4a" stroke-width="2">+ <rect x="0" y="92" width="480" height="20" rx="4"/><rect x="0" y="120" width="480" height="20" rx="4"/>+ <rect x="0" y="148" width="480" height="20" rx="4"/><rect x="0" y="176" width="480" height="20" rx="4"/>+ <rect x="0" y="204" width="480" height="20" rx="4"/><rect x="0" y="232" width="480" height="20" rx="4"/>+ </g>+ <text x="500" y="178" fill="#777" font-size="17">Qwen3.5</text>++ <g fill="#120f24" stroke="#8b7cf6" stroke-width="2">+ <rect x="0" y="278" width="480" height="20" rx="4"><animate attributeName="fill" values="#120f24;#3b2f8a;#120f24;#120f24" keyTimes="0;0.08;0.2;1" dur="1.2s" repeatCount="indefinite"/></rect>+ <rect x="0" y="306" width="480" height="20" rx="4"><animate attributeName="fill" values="#120f24;#120f24;#3b2f8a;#120f24;#120f24" keyTimes="0;0.15;0.23;0.35;1" dur="1.2s" repeatCount="indefinite"/></rect>+ <rect x="0" y="334" width="480" height="20" rx="4"><animate attributeName="fill" values="#120f24;#120f24;#3b2f8a;#120f24;#120f24" keyTimes="0;0.3;0.38;0.5;1" dur="1.2s" repeatCount="indefinite"/></rect>+ <rect x="0" y="362" width="480" height="20" rx="4"><animate attributeName="fill" values="#120f24;#120f24;#3b2f8a;#120f24;#120f24" keyTimes="0;0.45;0.53;0.65;1" dur="1.2s" repeatCount="indefinite"/></rect>+ </g>+ <text x="500" y="336" fill="#8b7cf6" font-size="17">CPU</text>++ <g><use href="#pkt"/><animateMotion path="M90 388V272" dur="1.2s" repeatCount="indefinite"/></g>+ <g><use href="#pkt"/><animateMotion path="M210 388V272" dur="1.2s" begin="-0.4s" repeatCount="indefinite"/></g>+ <g><use href="#pkt"/><animateMotion path="M330 388V272" dur="1.2s" begin="-0.8s" repeatCount="indefinite"/></g>+ <g><use href="#pkt"/><animateMotion path="M420 388V272" dur="1.2s" begin="-0.2s" repeatCount="indefinite"/></g>++ <rect x="230" y="76" width="20" height="320" fill="url(#beam)" opacity="0.3"/>+ <circle cx="240" r="5" fill="#c9c0ff">+ <animate attributeName="cy" values="396;84" dur="2.4s" repeatCount="indefinite"/>+ <animate attributeName="opacity" values="0;1;1;0" keyTimes="0;0.1;0.85;1" dur="2.4s" repeatCount="indefinite"/>+ </circle>+ </g>++ <g transform="translate(0 432)">+ <g>+ <animateTransform attributeName="transform" type="translate" from="0 0" to="-900 0" dur="9s" repeatCount="indefinite"/>+ <use href="#tape" x="40"/><use href="#tape" x="940"/><use href="#tape" x="1840"/>+ </g>+ </g>+ <rect x="0" y="420" width="1600" height="54" fill="url(#fadeL)"/>+</svg>diff --git a/docs/running.md b/docs/running.mdnew file mode 100644index 0000000..a04178b--- /dev/null+++ b/docs/running.md@@ -0,0 +1,111 @@+# Running Qwenputer (q4-int)++`q4-int` is stock Qwen3.5-4B with a small exact computer compiled into extra layers. The computer+is a CHIP-8 machine plus a bignum arithmetic BIOS. Only config numbers change: hidden size, layer+count and vocabulary. It loads as `Qwen3_5ForConditionalGeneration` (or, in the text-only variant,+`Qwen3_5ForCausalLM`) in stock transformers. The circuit is bf16-exact, and Qwen's own weights are+unchanged (zero-padded).++This note describes branch `q35-int`. The checkpoint path, commit and measured numbers are at the+end.++## What is in it++| | |+|---|---|+| Base | Qwen3.5-4B (32 layers, hidden 2560) |+| Computer | 12 chip layers *before* Qwen's 32 (layers 0-11), then Qwen's layers |+| Hidden size | 3200 (Qwen's 2560 dims + ballast + the circuit's text-visible dims). Circuit dims that only chip tokens need are overlaid on Qwen's dims (`overlay.py`) |+| Size | ~14 GB bf16 (text-only variant: ~13 GB, no vision tower) |+| Vocabulary | Qwen's, plus 8708 chip tokens (`<\|chip…\|>`) at `chip_token_offset` |+| Circuit | 12 layers, 3220 dims, 84 heads, 28,253 MLP units (`chipformer.json` → `circuit`) |++**What the computer does (all exact):**+- **Inline.** When Qwen writes `A op B =`, the circuit takes over at the `=`, runs the BIOS+ routine and prints the result as ordinary text.+ - Forms: `\times`, `\cdot`, `×`, `*`, `+`, `-`, `/`, `÷`, `\div`, `mod`, `\bmod`, `%` and+ `gcd(A, B) =`.+ - Signed operands for mul, div and gcd (`-5`, `(-5)`).+ - Thresholds: mul/div/mod/gcd need ≥ 4 digits in A and B together, add/sub ≥ 7.+- **Whole expressions (v18).** For example `(3 + 4) \times 12 / 7 =` or `2^{10} - 5 =`.+ - The BIOS evaluates them with exact fractions: + − × / ^, brackets, unary minus, decimals.+ - The single-op path wins when both match.+- **Tools.** Qwen's native tool calls `add`, `sub`, `mul`, `div`, `mod`, `gcd` and `calc(expression)`.+ - The circuit answers at `</tool_call>` with `\n<tool_response>\nR\n</tool_response>\n`, then+ an empty thinking block `<think>\n\n</think>\n\n` (two `\n` tokens each), as Qwen's next+ assistant turn starts.+ - Negative results print `-R`.+- **Output format:**+ - Integers as digits.+ - A quotient that ends within 10 decimals prints as that decimal (`0.625`).+ - Otherwise the reduced fraction `p/q` (`1785/13201915 = 357/2640383`).+ - Division by 0, or a result the BIOS can't represent (more than 24 digits, a malformed+ expression), prints `...`.+ - Inline results end with a closer: `.`, or `$` / `$$` inside math.++## How to run it++1. **The BIOS boot record.** Every prompt starts with the BIOS's boot record, as chip tokens+ (about 5,000 tokens, prefill only):+ ```python+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens+ meta = json.load(open(f"{path}/chipformer.json"))+ bios = [t + meta["chip_token_offset"] for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ ids = bios + tok.apply_chat_template(msgs, tools=TOOLS, add_generation_prompt=True,+ enable_thinking=False)["input_ids"]+ ```+ - The entry points are compiled into the weights, so the BIOS must be the one this checkpoint+ was built with. Check `chipformer.json` → `bios.sha1`, `ops`.+ - `bcd.ROM.segments` carries code below 0x200; `Chip8` loads it.+2. **The chat template.** Qwen's own template (`chat_template.jinja`), unchanged.+ - Tools: `chipformer.q35.demo.TOOLS` (add, sub, mul, div, mod, gcd), plus `demo.CALC_TOOL`.+ - With tools, Qwen calls them in its native XML format.+ - Without tools, inline mode works on Qwen's plain text.+3. **System prompts that help** (`chipformer/q35/evals.py`):+ - Tool mode: `NUDGE` ("You have tools for exact arithmetic (...). Use them for every+ arithmetic step they cover instead of calculating by hand.").+ - Inline mode: `NUDGE3` (single operations, including division).+ - Whole expressions: `NUDGE4`.+ - Products and quotients inline, sums through tools: `NUDGE_MIX3`.+4. **Generation config.** `generation_config.json` bans `<|im_start|>` and `<|endoftext|>`+ (`bad_words_ids`).+ - Qwen ends its turns with `<|im_end|>`, which stays the EOS.+ - Without the bans, Qwen sometimes opens a fake user turn, or ends with `<|endoftext|>` right+ after a tool response, before answering.+ - Greedy decoding (`do_sample=False`).+5. **Decoders:**+ - **Stock `generate()`** (`cache_implementation="static"`) runs the computer exactly. The+ circuit needs nothing from the host: one `generate()` call computes and prints.+ `chipformer/q35/demo.py` is a complete example.+ - **The numbers below use `chipformer.q35.batch.BatchDecoder`** with two decoder-side options.+ Both change only position ids and the KV layout, not weights or computer outputs.+ - `freeze_chip_positions` (fp): Qwen's RoPE positions don't advance on chip tokens.+ - `transparent_chip` (tc): Qwen's layers don't see chip tokens. Its text computation then+ equals the same text with the trace deleted.+ - Inline mode is markedly better with fp + tc. Tool mode works with both decoders.+ - **`--fast` (`skip_qwen`)** runs only the chip layers for rows on chip tokens. It is+ token-identical and about 2× faster at batch 16.+ - **flash-linear-attention / causal-conv1d** are not needed. The reference kernels are exact.+6. **Cache length.** A trace costs hundreds to thousands of chip tokens: a multiplication about+ 100-400, a division with a fraction 1-9k, an expression 4-15k. Give the cache room, e.g.+ `--max-len 16384` (32768 for whole expressions); with tc, Qwen doesn't see them.++## chipformer.json++| Field | Meaning |+|---|---|+| `layout` | `v2`: chip layers first |+| `chip_token_offset`, `chip_vocab` | Where the 8708 chip tokens start in the vocabulary |+| `chip_layers`, `qwen_layers_from` | 12: Qwen's layers start at 12 |+| `hidden`, `chip_dims_offset`, `ballast_dims` | The widened residual: Qwen's dims `[0, 2560)`, then ballast and circuit slots |+| `circuit` | Layers, dims, heads, MLP units of the compiled circuit |+| `slots`, `overlay_slots`, `overlay` | Circuit dims of their own, dims overlaid on Qwen's, and the overlay plan's summary |+| `muted` | Qwen's layers are muted on chip tokens (no decay, no writes) |+| `text_ids` | The text classes the circuit reads and prints, by Qwen token id |+| `bios` | The BIOS the weights expect: sha1, size, segments, entry points |++## Measured (q35-inline, A10G)++Filled in by the release check (see below).diff --git a/docs/trace_format.md b/docs/trace_format.mdnew file mode 100644index 0000000..33155d2--- /dev/null+++ b/docs/trace_format.md@@ -0,0 +1,77 @@+# Trace format++The transformer never sees "a machine state". It sees a token stream of **write events**, and+its only job is to predict the next token. Source of truth: `chipformer/spec.py`,+`chipformer/ref/emu.py`, `chipformer/trace.py`.++## Cells++All state lives in one flat array of 4609 cells, each holding an integer `0..0xFFF`.++| Address | Cell | Range |+|---|---|---|+| `0x0000–0x0FFF` | RAM | byte |+| `0x1000–0x100F` | V0–VF | byte |+| `0x1010` | I | 12-bit |+| `0x1011` | PC | 12-bit |+| `0x1012` | SP | 0–15 (wraps) |+| `0x1013` / `0x1014` | DT / ST | byte |+| `0x1015` | RNG (12-bit Galois LFSR, mask `0x829`) | 1–4095 |+| `0x1016` | CLS: writing it zeroes the display | – |+| `0x1017–0x101C` | quirk flags (see `quirks.md`) | 0/1 |+| `0x1020–0x102F` | stack (return addresses) | 12-bit |+| `0x1030–0x103F` | keys 0–F | 0/1 |+| `0x1100–0x11FF` | display, 32 rows × 8 bytes, MSB is the leftmost pixel | byte |+| `0x1200` | NULL sink (writes with no effect) | – |++## Tokens++| Token | Id |+|---|---|+| `VAL v` | `v` (0–4095) |+| `ADDR a` | `4096 + a` |+| `STEP` | 8705 |+| `TICK` | 8706 |+| `BOOT` | 8707 |++A write is the pair `ADDR a, VAL v`.++## Records++```+BOOT <full state dump> STEP prompt (also used for context rollover)+<instruction writes> STEP generated by the model+TICK <DT write> <ST write> STEP TICK injected by the host, rest generated+<KEY cell write> STEP injected whole by the host+```++- Host injection only ever happens right after a `STEP`.+- **Every value in a record is computed from the state before the record.** The PC write is+ always last. Writes inside a record never read cells written earlier in the same record.+ The model reads register operands once, at the record's first position, and carries them+ forward.+- The state dump writes CLS first, then every other **nonzero** cell (except NULL) in address+ order. A read of a cell with no write in the context returns 0.++## Instruction records++Each record is a list of writes, then the PC write, then `STEP`.++| Opcode | Writes before PC |+|---|---|+| `00E0` | CLS |+| `00EE` | SP←SP−1, PC←stack[SP−1] |+| `2NNN` | stack[SP]←PC+2, SP←SP+1 |+| `1NNN`, `BNNN`, skips, `EX9E/EXA1`, `0NNN` | none |+| `6XNN`, `7XNN`, `8XY0–3`, `FX07`, `FX0A` | VX (8XY1–3 add VF←0 with `logic`) |+| `8XY4–7`, `8XYE` | VX, VF (VF, VX with `vforder`) |+| `ANNN`, `FX1E`, `FX29` | I |+| `CXNN` | VX←rnd & NN, RNG←lfsr(RNG) where rnd = low byte of the new state |+| `DXYN` | 2N display writes (left byte, right byte per row; off-screen bytes wrap, or go to NULL with `clip`), then VF |+| `FX15` / `FX18` | DT / ST |+| `FX33` | M[I], M[I+1], M[I+2] |+| `FX55` | M[I..I+X], then I←I+X+1 unless `loadstore` |+| `FX65` | V0..VX, then I←I+X+1 unless `loadstore` |++Record lengths depend only on the opcode (and N/X) and the quirk flags, never on the data. The one exception is+FX0A's PC value: if no key is held it rewrites VX unchanged and keeps PC.diff --git a/experimental/riscv/README.md b/experimental/riscv/README.mdnew file mode 100644index 0000000..af5e7e1--- /dev/null+++ b/experimental/riscv/README.md@@ -0,0 +1,30 @@+# Experimental: a RISC-V computer inside Qwen++This folder compiles a 32-bit **RV32IM** CPU into stock Qwen3.5 layers, using the same compiler as+the main project. The CPU also has Zba and a bignum coprocessor. It is a self-contained snapshot:+it carries its own copy of the `chipformer` package, because its compiler changes haven't been+merged with the main tree yet.++## What works++- The official riscv-tests suites `rv32ui`, `rv32um` and `rv32uzba` pass. They are checked against+ the reference model and against qemu.+- gcc-compiled C runs exactly: crc32, qsort, a sieve, mini-gmp and a fuzz suite, about 18M+ instructions checked free-running. A patched MicroPython build also runs exactly.+- A `calc` firmware with bignum `+ - * / // % ** gcd` is exact on 10,000 random cases.+- `rv-tiny`, a standalone stock `Qwen3_5ForCausalLM`, runs programs from a memory snapshot under+ plain `generate()`.+- Inside Qwen3.5-4B, Qwen calls `calc` and the RISC-V machine in the weights answers. A replay of+ the evaluation records found 0 mismatches.++The machine has 256 KB of code ROM, 128 KB of read-only data and 32 KB of RAM, all held in Gated+DeltaNet state. It takes 11 chip layers and about 1M nonzero weights, and costs about 1.2-1.4+tokens per instruction. A compact executor runs about 3,700 instructions a second per stream on+an L4.++[`docs/rv32.md`](docs/rv32.md) has the design and every measurement, and+[`docs/rv32-p3-status.md`](docs/rv32-p3-status.md) has the in-Qwen status.++```bash+cd experimental/riscv && uv sync && uv run pytest tests -q+```diff --git a/experimental/riscv/chipformer/__init__.py b/experimental/riscv/chipformer/__init__.pynew file mode 100644index 0000000..35172ec--- /dev/null+++ b/experimental/riscv/chipformer/__init__.py@@ -0,0 +1 @@+"""A hand-constructed transformer that executes CHIP-8 programs."""diff --git a/experimental/riscv/chipformer/check.py b/experimental/riscv/chipformer/check.pynew file mode 100644index 0000000..42e3db4--- /dev/null+++ b/experimental/riscv/chipformer/check.py@@ -0,0 +1,87 @@+"""Differential testing: circuit predictions vs the reference trace, token by token."""++import random+from dataclasses import dataclass++import numpy as np++from chipformer import spec+from chipformer.dsl.batch import evaluate_batch, output_batch+from chipformer.dsl.graph import Graph+from chipformer.ref.emu import fetch, opcode_class+from chipformer.trace import Machine+++def run_machine(rom: bytes, steps: int, ipf: int = 10, seed: int = 0,+ key_rate: float = 0.1, quirks: "str | tuple[str, ...] | None" = None) -> Machine:+ """Reference run with a timer tick every ``ipf`` instructions and random key changes."""+ rng = random.Random(seed)+ m = Machine(rom, quirks)+ m.ops = [] # opcode class of each executed instruction, for diagnostics+ for i in range(steps):+ if i % ipf == 0:+ m.tick()+ if rng.random() < key_rate:+ m.set_key(rng.randrange(16), rng.random() < 0.5)+ m.ops.append((len(m.tokens), opcode_class(fetch(m.emu.cells))))+ m.step()+ return m+++@dataclass+class Mismatch:+ pos: int # position whose prediction was wrong (predicts token pos+1)+ want: int+ got: float+ op: str++ def __str__(self) -> str:+ return (f"position {self.pos} in {self.op} record: predicted {self.got} "+ f"({spec.token_name(int(self.got)) if float(self.got).is_integer() else '?'}), "+ f"want {self.want} ({spec.token_name(self.want)})")+++def check(graph: Graph, m: Machine, limit: int = 10) -> tuple[int, list[Mismatch]]:+ """Returns (number of checked predictions, first mismatches)."""+ res = evaluate_batch(graph, m.tokens)+ pred = output_batch(graph, "next", res)+ want = np.array(m.tokens[1:] + [-1])+ gen = np.array(m.generated[1:] + [False])+ bad = np.nonzero(gen & (pred != want))[0]+ starts = [p for p, _ in getattr(m, "ops", [])]+ out = []+ for p in bad[:limit]:+ i = np.searchsorted(starts, p + 1, side="right") - 1+ op = m.ops[i][1] if i >= 0 and starts else "?"+ out.append(Mismatch(int(p), int(want[p]), float(pred[p]), op))+ return int(gen.sum()), out+++def verify_trace(tokens: list[int]) -> tuple[int, str | None]:+ """Check a free-running trace against the reference semantics, record by record.++ Works for any key/tick schedule: host-injected key records are applied as+ given, every instruction and tick record must equal what the reference+ emulator computes from the state so far. Returns (records checked, error).+ """+ from chipformer.ref.emu import apply_writes, instruction_writes, state_writes, tick_writes+ from chipformer.trace import parse++ cells = [0] * spec.NUM_CELLS+ records = parse(tokens, allow_reboot=True)+ for i, r in enumerate(records):+ if r.kind == "boot":+ # first boot: the prompt; later boots (rollover) must dump the current state exactly+ expected = r.writes if i == 0 else state_writes(cells)+ cells = [0] * spec.NUM_CELLS+ elif r.kind == "tick":+ expected = tick_writes(cells)+ elif len(r.writes) == 1 and spec.KEYS <= r.writes[0][0] < spec.KEYS + 16:+ expected = r.writes # injected key change+ else:+ expected = instruction_writes(cells)+ if r.writes != expected:+ op = opcode_class(fetch(cells))+ return i, f"record {i} ({r.kind}, {op}): got {r.writes[:6]}, want {expected[:6]}"+ apply_writes(cells, r.writes)+ return len(records), Nonediff --git a/experimental/riscv/chipformer/chip8.py b/experimental/riscv/chipformer/chip8.pynew file mode 100644index 0000000..cc56d1d--- /dev/null+++ b/experimental/riscv/chipformer/chip8.py@@ -0,0 +1,307 @@+"""CHIP-8 as a circuit graph: the transformer's "program".++Every position runs the same circuit and outputs ``next``, the id of the next+token. The computation has two halves:++1. Context (meaningful at record starts, i.e. STEP/BOOT positions): fetch PC+ and the opcode, decode, read operands, and compute every write of the+ record (addresses, values, count, next PC).+2. Micro-step (every position): copy the context from the latest record start+ with one attention head, work out which write of the record comes next+ (j) and whether we are predicting its ADDR or its VAL, do the per-write+ reads (FX55/FX65/DXYN), and pick the output.++Reading the context at the record start means every value is computed from+the state before the record, as the trace format requires (see+docs/trace_format.md), even after earlier writes in the same record.+"""++from chipformer import spec+from chipformer.dsl.graph import Expr+from chipformer.dsl.lib import MAX_POS, Circuit++JMAX = 40 # write indices are clamped here (largest record: DXYN with 32 writes)+NOT_FOUND = 2 * MAX_POS # pushes "latest flagged position" checks negative when no flag exists++++class Chip8Circuit(Circuit):+ def __init__(self) -> None:+ super().__init__()+ f = self.token_feature+ self.is_val = f("is_val", lambda t: int(spec.is_val(t)))+ self.is_addr = f("is_addr", lambda t: int(spec.is_addr(t)))+ self.is_step = f("is_step", lambda t: int(t == spec.STEP))+ self.is_tick = f("is_tick", lambda t: int(t == spec.TICK))+ self.is_boot = f("is_boot", lambda t: int(t == spec.BOOT))+ self.val = f("val", lambda t: t - spec.VAL_BASE if spec.is_val(t) else 0)+ self.addr = f("addr", lambda t: t - spec.ADDR_BASE if spec.is_addr(t) else 0)+ addr2 = f("addr^2", lambda t: (t - spec.ADDR_BASE) ** 2 if spec.is_addr(t) else 0)+ # at a VAL position: the address being written (from the ADDR token before it)+ self.wa, self.wa2 = self.prev([self.addr, addr2], "write_addr")++ def read(self, q: "Expr | int", name: str) -> Expr:+ """Current value of cell q: the latest write to it, or 0 if never written."""+ q = q if isinstance(q, Expr) else self.one * q+ v, found = self.latest_keyed(q, self.wa, self.wa2, self.is_val, [self.val],+ spec.NUM_CELLS, name)+ return self.eqglu(v, found - q, 0, f"{name}.hit")++ def read_display(self, q: Expr, name: str) -> Expr:+ """Like read, but a display byte written before the latest CLS reads as 0."""+ v, pos, found = self.latest_keyed(q, self.wa, self.wa2, self.is_val, [self.val, self.pos],+ spec.NUM_CELLS, name)+ hit = self.eqglu(v, found - q, 0, f"{name}.hit")+ (cls_pos,) = self.latest_keyed(self.one * spec.CLS, self.wa, self.wa2, self.is_val,+ [self.pos], spec.NUM_CELLS, f"{name}.cls")[:1]+ return self.stepglu(hit, pos - cls_pos - 1, f"{name}.after_cls")++ def flagged_since(self, pos: Expr, flag: Expr, start: Expr, name: str) -> Expr:+ """[latest position with flag=1 is after start]; pos/flag come from a latest() lookup."""+ return self.step(pos - start - 1 - NOT_FOUND * (1 - flag), name)+++def build() -> Chip8Circuit:+ c = Chip8Circuit()+ V, VF = spec.V, spec.V + 0xF++ # ---------------------------------------------------------------- record+ (p0,) = c.latest(c.is_step + c.is_boot, [c.pos], "rec_start")+ ptick, tflag = c.latest(c.is_tick, [c.pos, c.is_tick], "last_tick")+ in_tick = c.flagged_since(ptick, tflag, p0, "in_tick")++ # ---------------------------------------------------------------- context+ pc = c.read(spec.PC, "PC")+ ireg = c.read(spec.I, "I")+ sp = c.read(spec.SP, "SP")+ dt = c.read(spec.DT, "DT")+ st = c.read(spec.ST, "ST")+ rng = c.read(spec.RNG, "RNG")+ v0 = c.read(V, "V0")+ keys = [c.read(spec.KEYS + k, f"key{k:X}") for k in range(16)]+ q = {name: c.read(spec.QUIRKS + i, f"q.{name}") for i, name in enumerate(spec.QUIRK_NAMES)}++ pc1 = c.wrap(pc + 1, 0x1000, "pc+1")+ pc2 = c.wrap(pc + 2, 0x1000, "pc+2")+ pc4 = c.wrap(pc + 4, 0x1000, "pc+4")+ sp_dec = c.wrap(sp + 15, 16, "sp-1")+ sp_inc = c.wrap(sp + 1, 16, "sp+1")+ dt_dec = dt - c.step(dt - 1, "dt>0")+ st_dec = st - c.step(st - 1, "st>0")++ b1 = c.read(pc, "op_hi")+ b2 = c.read(pc1, "op_lo")+ stack_top = c.read(spec.STACK + sp_dec, "stack_top")+ hi, x = c.divmod(b1, 16, 15, "op.hi")+ y, n = c.divmod(b2, 16, 15, "op.y")+ nn, nnn = b2, 256 * x + b2+ vx = c.read(V + x, "VX")+ vy = c.read(V + y, "VY")++ ind = decode(c, b1, b2, hi, n)++ xb = c.bits(vx, 8, "vx.b")+ yb = c.bits(vy, 8, "vy.b")+ vx_lo = Circuit.from_bits(xb[:4])+ and_xy = c.band_bits(xb, yb, "vx&vy")+ s = vx + vy+ carry = c.step(s - 256, "carry")+ ge_xy = c.step(vx - vy, "vx>=vy")+ ge_yx = c.step(vy - vx, "vy>=vx")+ eq_x_nn = c.eq(vx, nn, "vx==nn")+ eq_x_y = c.eq(vx, vy, "vx==vy")+ hundreds = c.floor_div(vx, 100, 2, "vx/100")+ tens_all = c.floor_div(vx, 10, 25, "vx/10")+ i_plus = [c.wrap(ireg + m, 0x1000, f"i+{m}") for m in range(3)]++ # keys: EX9E/EXA1 test key[VX & 0xF]; FX0A takes the lowest held key+ key_vx = c.index(keys, vx_lo, "key[vx]")+ prefix = [sum(keys[:k], Expr()) for k in range(16)]+ lowest = c.ffn([t for k in range(16) for t in (+ (k, keys[k], prefix[k] + 1), (-2 * k, keys[k], prefix[k]), (k, keys[k], prefix[k] - 1))],+ "lowest_key")+ any_key = c.step(sum(keys, Expr()) - 1, "any_key")++ # CXNN: one Galois LFSR step, then the low byte masked with NN+ rb = c.bits(rng, 12, "rng.b")+ shifted = rb[1:] + [Expr()]+ sbits = []+ for i in range(12):+ if spec.RNG_MASK >> i & 1:+ sbits.append(shifted[i] + rb[0] - 2 * c.gate(shifted[i], rb[0], f"lfsr{i}"))+ else:+ sbits.append(shifted[i])+ rng_next = Circuit.from_bits(sbits)+ nnb = c.bits(nn, 8, "nn.b")+ rnd = c.ffn([(1 << i, nnb[i], sbits[i]) for i in range(8)], "rnd&nn")++ # 8XY6/8XYE shift VX in place with the shift quirk, otherwise VY+ shr = c.mux(q["shift"], Circuit.from_bits(xb[1:]), Circuit.from_bits(yb[1:]), "shr")+ shl = c.mux(q["shift"], 2 * Circuit.from_bits(xb[:7]), 2 * Circuit.from_bits(yb[:7]), "shl")+ lsb = c.mux(q["shift"], xb[0], yb[0], "lsb")+ msb = c.mux(q["shift"], xb[7], yb[7], "msb")++ # Instructions whose writes (besides PC) are computed at the record start, as+ # (condition, writes) variants: conditions are one-hot, quirks pick the variant.+ fixed: list[tuple[Expr, list]] = [(ind[op], w) for op, w in {+ "00E0": [(spec.CLS, 0)],+ "00EE": [(spec.SP, sp_dec)],+ "2NNN": [(spec.STACK + sp, pc2), (spec.SP, sp_inc)],+ "6XNN": [(V + x, nn)],+ "7XNN": [(V + x, c.wrap(vx + nn, 256, "vx+nn"))],+ "8XY0": [(V + x, vy)],+ "ANNN": [(spec.I, nnn)],+ "CXNN": [(V + x, rnd), (spec.RNG, rng_next)],+ "FX07": [(V + x, dt)],+ "FX0A": [(V + x, c.mux(any_key, lowest, vx, "fx0a"))],+ "FX15": [(spec.DT, vx)],+ "FX18": [(spec.ST, vx)],+ "FX1E": [(spec.I, c.wrap(ireg + vx, 0x1000, "i+vx"))],+ "FX29": [(spec.I, spec.FONT_BASE + 5 * vx_lo)],+ "FX33": [(i_plus[0], hundreds), (i_plus[1], tens_all - 10 * hundreds),+ (i_plus[2], vx - 10 * tens_all)],+ }.items()]+ for op, result in [("8XY1", vx + vy - and_xy), ("8XY2", and_xy), ("8XY3", vx + vy - 2 * and_xy)]:+ fixed.append((c.gate(ind[op], 1 - q["logic"], f"{op}.plain"), [(V + x, result)]))+ fixed.append((c.gate(ind[op], q["logic"], f"{op}.vfreset"), [(V + x, result), (VF, 0)]))+ for op, result, flag in [("8XY4", s - 256 * carry, carry),+ ("8XY5", vx - vy + 256 * (1 - ge_xy), ge_xy), ("8XY6", shr, lsb),+ ("8XY7", vy - vx + 256 * (1 - ge_yx), ge_yx), ("8XYE", shl, msb)]:+ fixed.append((c.gate(ind[op], 1 - q["vforder"], f"{op}.flaglast"),+ [(V + x, result), (VF, flag)]))+ fixed.append((c.gate(ind[op], q["vforder"], f"{op}.flagfirst"),+ [(VF, flag), (V + x, result)]))+ fa = [c.select([(cond, w[m][0]) for cond, w in fixed if len(w) > m], f"A{m}") for m in range(3)]+ fv = [c.select([(cond, w[m][1]) for cond, w in fixed if len(w) > m], f"W{m}") for m in range(3)]+ has = [sum((cond for cond, w in fixed if len(w) > m), Expr()) for m in range(3)]+ is_vec = ind["FX55"] + ind["FX65"] + ind["DXYN"]++ # FX55/FX65 advance I by X+1 after the transfer unless the loadstore quirk is set+ ls_extra = c.gate(ind["FX55"] + ind["FX65"], 1 - q["loadstore"], "i_advance")+ i_next = c.wrap(ireg + x + 1, 0x1000, "i+x+1")+ nw = (sum((len(w) * cond for cond, w in fixed), Expr()) + ls_extra+ + c.ffn([(1, x + 1, ind["FX55"]), (1, x + 1, ind["FX65"]), (1, 2 * n + 1, ind["DXYN"])],+ "vec_count"))++ skip = c.ffn([(1, ind["3XNN"], eq_x_nn), (1, ind["4XNN"], 1 - eq_x_nn),+ (1, ind["5XY0"], eq_x_y), (1, ind["9XY0"], 1 - eq_x_y),+ (1, ind["EX9E"], key_vx), (1, ind["EXA1"], 1 - key_vx)], "skip")+ wait = c.gate(ind["FX0A"], 1 - any_key, "wait")+ bnnn = c.wrap(nnn + c.mux(q["jump"], vx, v0, "jump_reg"), 0x1000, "jump_target")+ next_pc = pc2 + c.select([+ (skip, pc4 - pc2), (ind["1NNN"] + ind["2NNN"], nnn - pc2),+ (ind["00EE"], stack_top - pc2), (ind["BNNN"], bnnn - pc2), (wait, pc - pc2)], "next_pc")++ x0b, y0b = xb[:6], yb[:5] # sprite origin = (VX % 64, VY % 32)+ ctx = {+ "x": x, "n": n, "I": ireg, "nw": nw, "next_pc": next_pc,+ "fixed": 1 - is_vec, "has0": has[0], "has1": has[1], "has2": has[2],+ "A0": fa[0], "A1": fa[1], "A2": fa[2], "W0": fv[0], "W1": fv[1], "W2": fv[2],+ "fx55": ind["FX55"], "fx65": ind["FX65"], "dxyn": ind["DXYN"],+ "dx_byte": Circuit.from_bits(x0b[3:]), "dx_off": Circuit.from_bits(x0b[:3]),+ "dy": Circuit.from_bits(y0b), "dt_dec": dt_dec, "st_dec": st_dec,+ "ls_extra": ls_extra, "i_next": i_next, "clip": q["clip"],+ }+ k = dict(zip(ctx, c.latest(c.is_step + c.is_boot, list(ctx.values()), "ctx")))++ # ---------------------------------------------------------------- micro-step+ j_raw = (c.pos - p0 - in_tick - c.is_addr) / 2+ j = j_raw - c.relu(j_raw - JMAX, "j.clamp")+ j_is = [c.eq(j, m, f"j=={m}") for m in range(3)]+ row_i, side = c.divmod(j, 2, JMAX // 2, "j/2")+ is_pc = c.eq(j - k["nw"], 0, "j==nw")+ is_end = c.eq(j - k["nw"] - 1, 0, "j==nw+1")+ le_x = c.step(k["x"] - j, "j<=x")+ lt_2n = c.step(2 * k["n"] - 1 - j, "j<2n")+ eq_2n = c.eq(j - 2 * k["n"], 0, "j==2n")+ is_i_advance = c.eq(j - k["x"] - 1, 0, "j==x+1")++ i_j = c.wrap(k["I"] + j, 0x1000, "i+j")+ i_row = c.wrap(k["I"] + row_i, 0x1000, "i+row")+ q_vec = c.select([(k["fx55"], V + j), (k["fx65"], i_j), (k["dxyn"], i_row)], "q_vec")+ vread = c.read(q_vec, "vread")+ old = c.read_display(c.addr, "old") # at an ADDR position: the target byte's current value++ # DXYN rows: write j targets display byte (dy + j//2, dx_byte + j%2). Off-screen bytes+ # go to NULL with the clip quirk, otherwise the row and byte wrap around the screen.+ row = k["dy"] + row_i+ col = k["dx_byte"] + side+ row_w = row - 32 * c.step(row - 32, "row>=32")+ col_w = col - 8 * c.step(col - 8, "col>=8")+ on_screen = c.all_of(c.step(31 - row, "row<32"), c.step(7 - col, "col<8"), name="on_screen")+ drawn = 1 - c.gate(1 - on_screen, k["clip"], "clipped")+ target = spec.NULL + c.gate(spec.DISPLAY + 8 * row_w + col_w - spec.NULL, drawn, "target")+ shift = [c.eq(2 * k["dx_off"] + side, kk, f"shift{kk}") for kk in range(16)]+ sb = c.bits(vread, 8, "sprite.b")+ part_bits = []+ for p in range(8):+ terms = []+ for kk in range(16):+ o, right = kk // 2, kk % 2+ src = p - 8 + o if right else p + o # (s << (8-o)) & 0xFF or s >> o+ if 0 <= src < 8:+ terms.append((1, sb[src], shift[kk]))+ part_bits.append(c.ffn(terms, f"part{p}"))+ part = Circuit.from_bits(part_bits)+ ob = c.bits(old, 8, "old.b")+ overlap = c.ffn([(1 << p, ob[p], part_bits[p]) for p in range(8)], "old&part")+ new_byte = old + part - 2 * overlap+ not_tick = 1 - in_tick+ draw_row = c.all_of(c.is_addr, not_tick, k["dxyn"], lt_2n, name="draw_row")+ collided = c.step(overlap - 1 - 256 * (1 - draw_row), "collided")+ cpos, cflag = c.latest(collided, [c.pos, collided], "last_collision")+ any_collision = c.flagged_since(cpos, cflag, p0, "any_collision")++ # ---------------------------------------------------------------- output+ at_val, at_addr = c.is_addr, 1 - c.is_addr+ A = spec.ADDR_BASE+ cases: list[tuple[Expr, Expr | int]] = []++ def case(conds: list[Expr], value: "Expr | int", name: str) -> None:+ cases.append((c.all_of(*conds, name=name), value))++ for m in range(3):+ case([at_val, not_tick, k["fixed"], j_is[m], k[f"has{m}"]], k[f"W{m}"], f"fix{m}.val")+ case([at_addr, not_tick, k["fixed"], j_is[m], k[f"has{m}"]], A + k[f"A{m}"], f"fix{m}.addr")+ case([at_val, not_tick, k["fx55"], le_x], vread, "fx55.val")+ case([at_addr, not_tick, k["fx55"], le_x], A + i_j, "fx55.addr")+ case([at_val, not_tick, k["fx65"], le_x], vread, "fx65.val")+ case([at_addr, not_tick, k["fx65"], le_x], A + V + j, "fx65.addr")+ case([at_val, not_tick, k["ls_extra"], is_i_advance], k["i_next"], "i_advance.val")+ case([at_addr, not_tick, k["ls_extra"], is_i_advance], A + spec.I, "i_advance.addr")+ case([at_val, not_tick, k["dxyn"], lt_2n, drawn], new_byte, "draw.val")+ case([at_addr, not_tick, k["dxyn"], lt_2n], A + target, "draw.addr")+ case([at_val, not_tick, k["dxyn"], eq_2n], any_collision, "vf.val")+ case([at_addr, not_tick, k["dxyn"], eq_2n], A + VF, "vf.addr")+ case([at_val, not_tick, is_pc], k["next_pc"], "pc.val")+ case([at_addr, not_tick, is_pc], A + spec.PC, "pc.addr")+ case([at_addr, not_tick, is_end], spec.STEP, "end")+ case([at_val, in_tick, j_is[0]], k["dt_dec"], "dt.val")+ case([at_addr, in_tick, j_is[0]], A + spec.DT, "dt.addr")+ case([at_val, in_tick, j_is[1]], k["st_dec"], "st.val")+ case([at_addr, in_tick, j_is[1]], A + spec.ST, "st.addr")+ case([at_addr, in_tick, j_is[2]], spec.STEP, "tick.end")+ c.output("next", c.select(cases, "next"))+ return c+++def decode(c: Circuit, b1: Expr, b2: Expr, hi: Expr, n: Expr) -> dict[str, Expr]:+ """One-hot opcode classes (names as in ref.emu.opcode_class), one FFN sublayer."""+ ind = {}+ for h, name in [(1, "1NNN"), (2, "2NNN"), (3, "3XNN"), (4, "4XNN"), (6, "6XNN"),+ (7, "7XNN"), (0xA, "ANNN"), (0xB, "BNNN"), (0xC, "CXNN"), (0xD, "DXYN")]:+ ind[name] = c.eq(hi, h, name)+ full = 256 * b1 + b2+ ind["00E0"] = c.eq(full, 0x00E0, "00E0")+ ind["00EE"] = c.eq(full, 0x00EE, "00EE")+ hn = 16 * hi + n+ ind["5XY0"] = c.eq(hn, 0x50, "5XY0")+ ind["9XY0"] = c.eq(hn, 0x90, "9XY0")+ for m in [0, 1, 2, 3, 4, 5, 6, 7, 0xE]:+ ind[f"8XY{m:X}"] = c.eq(hn, 0x80 + m, f"8XY{m:X}")+ hb = 256 * hi + b2+ for code in [0x9E, 0xA1]:+ ind[f"EX{code:02X}"] = c.eq(hb, 0xE00 + code, f"EX{code:02X}")+ for code in [0x07, 0x0A, 0x15, 0x18, 0x1E, 0x29, 0x33, 0x55, 0x65]:+ ind[f"FX{code:02X}"] = c.eq(hb, 0xF00 + code, f"FX{code:02X}")+ return inddiff --git a/experimental/riscv/chipformer/compile.py b/experimental/riscv/chipformer/compile.pynew file mode 100644index 0000000..0273658--- /dev/null+++ b/experimental/riscv/chipformer/compile.py@@ -0,0 +1,191 @@+"""Compile a Graph into the weight tensors of a plain transformer.++Architecture (no CHIP-8 knowledge; see chipformer/dense.py):++ r_t = E[token_t] + W_pos @ [t, t^2] (polynomial position embedding)+ for each layer:+ r += sum_h Wo_h V_h[argmax-softmax_s (Wq_h r_t) . (Wk_h r_s)] (s <= t)+ r += W_down (ReLU(W_gate r) * (W_up r)) (ReGLU)+ logits = U r++Residual dimensions are the graph's dims, one to one. Each attention head+has 2D queries/keys; the second query coordinate is the constant-one dim, so+scores are ``q * kx + ky``. Softmax runs at temperature 1/BETA; since scores+are integers that differ by at least 1, BETA = 1000 makes every attention row+exactly one-hot in fp64.++The unembedding row for token v is ``2v * W_next - v^2 * e_one``, so+``logit_v = y^2 - (y - v)^2`` with y the "next" output: argmax is exactly y.+"""++import json+from pathlib import Path++import numpy as np++from chipformer import spec+from chipformer.dsl.graph import FFN, ONE, Expr, Graph, Lookup++BETA = 1000.0+++def _row(e: Expr, d: int) -> np.ndarray:+ r = np.zeros(d)+ for k, v in e.terms.items():+ r[k] = float(v)+ return r+++def compile_graph(g: Graph, vocab_size: int = spec.VOCAB_SIZE,+ output: str = "next") -> tuple[dict[str, np.ndarray], dict]:+ d = len(g.dims)+ w: dict[str, np.ndarray] = {}++ emb = np.zeros((vocab_size, d))+ emb[:, ONE] = 1.0+ for dim, fn in g.token_fns.items():+ emb[:, dim] = [float(fn(t)) for t in range(vocab_size)]+ w["embed.token"] = emb++ pos_names = {"pos": 1, "pos^2": 2}+ wpos = np.zeros((d, 2))+ for dim, fn in g.pos_fns.items():+ name = g.dims[dim].name+ if name not in pos_names:+ raise ValueError(f"position feature {name!r} is not t or t^2")+ wpos[dim, pos_names[name] - 1] = 1.0+ w["embed.pos_poly"] = wpos++ n_layers = g.n_layers()+ layers = []+ for layer in range(n_layers):+ heads = [n for n in g.nodes if isinstance(n, Lookup) and n.level == 2 * layer + 1]+ ffns = [n for n in g.nodes if isinstance(n, FFN) and n.level == 2 * layer + 2]+ for h, node in enumerate(heads):+ p = f"layers.{layer}.attn.{h}"+ w[f"{p}.wq"] = np.stack([_row(node.q, d), _row(g.one, d)])+ w[f"{p}.wk"] = np.stack([_row(node.kx, d), _row(node.ky, d)])+ w[f"{p}.wv"] = np.stack([_row(v, d) for v in node.values])+ wo = np.zeros((d, len(node.out)))+ for i, dim in enumerate(node.out):+ wo[dim, i] = 1.0+ w[f"{p}.wo"] = wo+ units = [(node.out, c, a, b) for node in ffns for c, a, b in node.terms]+ gate = np.zeros((len(units), d))+ up = np.zeros((len(units), d))+ down = np.zeros((d, len(units)))+ for u, (out, c, a, b) in enumerate(units):+ gate[u] = _row(b, d)+ up[u] = _row(a, d)+ down[out, u] = float(c)+ w[f"layers.{layer}.ffn.w_gate"] = gate+ w[f"layers.{layer}.ffn.w_up"] = up+ w[f"layers.{layer}.ffn.w_down"] = down+ layers.append({"heads": len(heads), "ffn_units": len(units),+ "head_names": [n.name for n in heads],+ "q_ranges": [list(n.q_range) for n in heads]})++ y = _row(g.outputs[output], d)+ v = np.arange(vocab_size, dtype=np.float64)+ unembed = 2.0 * v[:, None] * y[None, :]+ unembed[:, ONE] -= v * v+ w["unembed"] = unembed++ config = {+ "machine": machine_spec(),+ "d_model": d, "n_layers": n_layers, "vocab_size": vocab_size, "beta": BETA,+ "pos_features": ["t", "t^2"], "layers": layers,+ "params": int(sum(a.size for a in w.values())),+ "nonzero_params": int(sum(np.count_nonzero(a) for a in w.values())),+ }+ return w, config+++def machine_spec() -> dict:+ """Constants a host needs to boot the model and talk to it (from chipformer.spec)."""+ names = ["VAL_BASE", "ADDR_BASE", "STEP", "TICK", "BOOT", "NUM_CELLS", "NULL", "CLS", "PC",+ "RNG", "KEYS", "DISPLAY", "DISPLAY_W", "DISPLAY_H", "PROGRAM_START", "FONT_BASE",+ "RNG_SEED", "QUIRKS"]+ out = {n: getattr(spec, n) for n in names}+ out["FONT"] = list(spec.FONT)+ out["QUIRK_NAMES"] = list(spec.QUIRK_NAMES)+ out["PROFILES"] = {k: list(v) for k, v in spec.PROFILES.items()}+ return out+++def save(w: dict[str, np.ndarray], config: dict, out_dir: Path) -> None:+ from safetensors.numpy import save_file++ out_dir.mkdir(parents=True, exist_ok=True)+ save_file({k: np.ascontiguousarray(v) for k, v in w.items()}, out_dir / "model.safetensors")+ (out_dir / "config.json").write_text(json.dumps(config, indent=1))+++def load(model_dir: Path) -> tuple[dict[str, np.ndarray], dict]:+ from safetensors.numpy import load_file++ return load_file(model_dir / "model.safetensors"), json.loads(+ (model_dir / "config.json").read_text())+++SPARSE_MAGIC = b"CHFM"+++def save_sparse(w: dict[str, np.ndarray], config: dict, path: Path) -> None:+ """The same weights as a compact sparse file (for the browser).++ Layout: b"CHFM", u32 version (1), u32 header length, UTF-8 JSON header+ {"config": ..., "tensors": [{"name", "shape", "nnz"}]}, then for each tensor+ in header order: nnz u32 rows, nnz u32 cols, nnz f64 values (little-endian,+ row-major order). Exactly the nonzero entries of the dense tensors.+ """+ tensors, blobs = [], []+ for name, a in w.items():+ a2 = a.reshape(a.shape[0], -1)+ rows, cols = np.nonzero(a2)+ tensors.append({"name": name, "shape": list(a2.shape), "nnz": len(rows)})+ blobs += [rows.astype("<u4").tobytes(), cols.astype("<u4").tobytes(),+ a2[rows, cols].astype("<f8").tobytes()]+ header = json.dumps({"config": config, "tensors": tensors}).encode()+ with open(path, "wb") as f:+ f.write(SPARSE_MAGIC + np.array([1, len(header)], dtype="<u4").tobytes() + header)+ f.writelines(blobs)+++def load_sparse(path: Path) -> tuple[dict[str, np.ndarray], dict]:+ data = path.read_bytes()+ assert data[:4] == SPARSE_MAGIC, "not a chipformer sparse model"+ _, hlen = np.frombuffer(data[4:12], dtype="<u4")+ header = json.loads(data[12:12 + hlen])+ off, w = 12 + int(hlen), {}+ for t in header["tensors"]:+ n = t["nnz"]+ rows = np.frombuffer(data, "<u4", n, off)+ cols = np.frombuffer(data, "<u4", n, off + 4 * n)+ vals = np.frombuffer(data, "<f8", n, off + 8 * n)+ off += 16 * n+ a = np.zeros(t["shape"])+ a[rows, cols] = vals+ w[t["name"]] = a+ return w, header["config"]+++def main() -> None:+ import argparse++ from chipformer.chip8 import build++ ap = argparse.ArgumentParser(description="Build the CHIP-8 transformer weights")+ ap.add_argument("--out", type=Path, default=Path("model"))+ args = ap.parse_args()+ w, config = compile_graph(build())+ save(w, config, args.out)+ save_sparse(w, config, args.out / "model.chipformer")+ print(f"{args.out}: {config['n_layers']} layers, d_model {config['d_model']}, "+ f"{sum(lay['heads'] for lay in config['layers'])} heads, "+ f"{sum(lay['ffn_units'] for lay in config['layers'])} FFN units, "+ f"{config['params']:,} params ({config['nonzero_params']:,} nonzero)")+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/dense.py b/experimental/riscv/chipformer/dense.pynew file mode 100644index 0000000..dcdc18a--- /dev/null+++ b/experimental/riscv/chipformer/dense.py@@ -0,0 +1,36 @@+"""A plain decoder-only transformer forward pass. Knows nothing about CHIP-8.++Standard pieces only: token embedding plus position embedding, causal+multi-head softmax attention with residual connections, ReGLU feed-forward+layers, and a linear unembedding. Everything is fp64.+"""++import numpy as np+++def softmax_attention(q: np.ndarray, k: np.ndarray, v: np.ndarray, beta: float) -> np.ndarray:+ n = q.shape[0]+ scores = q @ k.T+ scores[np.triu_indices(n, 1)] = -np.inf # causal mask+ scores -= scores.max(axis=1, keepdims=True)+ p = np.exp(beta * scores)+ p /= p.sum(axis=1, keepdims=True)+ return p @ v+++def forward(w: dict[str, np.ndarray], config: dict, tokens: list[int]) -> tuple[np.ndarray, np.ndarray]:+ """Returns (final residual stream, logits), both of shape (n_positions, ...)."""+ t = np.arange(len(tokens), dtype=np.float64)+ r = w["embed.token"][np.asarray(tokens)] + np.stack([t, t * t], 1) @ w["embed.pos_poly"].T+ for layer, spec in enumerate(config["layers"]):+ attn = np.zeros_like(r)+ for h in range(spec["heads"]):+ p = f"layers.{layer}.attn.{h}"+ out = softmax_attention(r @ w[f"{p}.wq"].T, r @ w[f"{p}.wk"].T, r @ w[f"{p}.wv"].T,+ config["beta"])+ attn += out @ w[f"{p}.wo"].T+ r = r + attn+ p = f"layers.{layer}.ffn"+ hidden = np.maximum(r @ w[f"{p}.w_gate"].T, 0.0) * (r @ w[f"{p}.w_up"].T)+ r = r + hidden @ w[f"{p}.w_down"].T+ return r, r @ w["unembed"].Tdiff --git a/experimental/riscv/chipformer/dsl/__init__.py b/experimental/riscv/chipformer/dsl/__init__.pynew file mode 100644index 0000000..a5941b5--- /dev/null+++ b/experimental/riscv/chipformer/dsl/__init__.py@@ -0,0 +1 @@+"""Circuit DSL: describe a computation as nodes that map 1:1 onto transformer parts."""diff --git a/experimental/riscv/chipformer/dsl/batch.py b/experimental/riscv/chipformer/dsl/batch.pynew file mode 100644index 0000000..7053846--- /dev/null+++ b/experimental/riscv/chipformer/dsl/batch.py@@ -0,0 +1,77 @@+"""Teacher-forced evaluation of a Graph over a whole token sequence at once.++Equivalent to feeding the tokens one by one through ``Evaluator``, but each+node is computed as a column over all positions with numpy. Values are+float64, which is exact here because every quantity is a dyadic rational+below 2^53 (lookups assert the bound).+"""++import numpy as np++from chipformer.dsl.evaluate import EXACT_LIMIT, LiChao, TieError+from chipformer.dsl.graph import FFN, ONE, Expr, Graph+++def ev_batch(e: Expr, m: np.ndarray) -> np.ndarray:+ if not e.terms:+ return np.zeros(m.shape[0])+ idx = np.fromiter(e.terms.keys(), dtype=np.int64, count=len(e.terms))+ coef = np.array([float(v) for v in e.terms.values()])+ return m[:, idx] @ coef+++def prefix_argmax(score: np.ndarray) -> np.ndarray:+ """For each t, the unique s <= t maximizing score[s]."""+ run = np.maximum.accumulate(score)+ prev = np.concatenate([[-np.inf], run[:-1]])+ if np.any(score == prev):+ t = int(np.argmax(score == prev))+ raise TieError(f"tie at position {t}")+ new = score > prev+ return np.maximum.accumulate(np.where(new, np.arange(len(score)), 0))+++def evaluate_batch(g: Graph, tokens: list[int]) -> np.ndarray:+ """Residual stream after the last sublayer: an (n_positions, n_dims) array."""+ n = len(tokens)+ m = np.zeros((n, len(g.dims)))+ m[:, ONE] = 1.0+ toks = np.asarray(tokens, dtype=np.int64)+ uniq, inv = np.unique(toks, return_inverse=True)+ for d, fn in g.token_fns.items():+ m[:, d] = np.array([float(fn(int(u))) for u in uniq])[inv]+ for d, fn in g.pos_fns.items():+ m[:, d] = [float(fn(t)) for t in range(n)]++ for node in g.nodes:+ if isinstance(node, FFN):+ acc = np.zeros(n)+ for c, a, b in node.terms:+ acc += float(c) * ev_batch(a, m) * np.maximum(ev_batch(b, m), 0.0)+ m[:, node.out] = acc+ continue++ kx, ky, q = ev_batch(node.kx, m), ev_batch(node.ky, m), ev_batch(node.q, m)+ lo, hi = node.q_range+ worst = np.abs(kx).max() * max(abs(lo), abs(hi)) + np.abs(ky).max()+ if worst >= EXACT_LIMIT:+ raise OverflowError(f"{node.name}: score magnitude {worst:.3g} >= 2^53")+ if np.any(q != np.round(q)) or q.min() < lo or q.max() > hi:+ raise ValueError(f"{node.name}: query outside integer range [{lo}, {hi}]")+ if lo == hi:+ src = prefix_argmax(kx * lo + ky)+ else:+ src = np.empty(n, dtype=np.int64)+ tree = LiChao(lo, hi)+ for t, (a, b, x) in enumerate(zip(kx.astype(np.int64).tolist(),+ ky.astype(np.int64).tolist(),+ q.astype(np.int64).tolist())):+ tree.insert(a, b, t)+ src[t] = tree.query(x)+ for d, v in zip(node.out, node.values):+ m[:, d] = ev_batch(v, m)[src]+ return m+++def output_batch(g: Graph, name: str, m: np.ndarray) -> np.ndarray:+ return ev_batch(g.outputs[name], m)diff --git a/experimental/riscv/chipformer/dsl/evaluate.py b/experimental/riscv/chipformer/dsl/evaluate.pynew file mode 100644index 0000000..a06905c--- /dev/null+++ b/experimental/riscv/chipformer/dsl/evaluate.py@@ -0,0 +1,148 @@+"""Exact autoregressive evaluation of a Graph.++Runs position by position, like a transformer with a KV cache: each lookup+head keeps the key (a line ``kx * q + ky``) and value vector of every past+position. Arithmetic uses Python ints/Fractions, so results are exact.++Two cache implementations:+ - "lichao": a Li Chao tree per head, O(log range) insert and query+ - "brute": a scan over all positions that also asserts the argmax is unique+ (softmax would average ties, so a tie means a broken circuit)+"""++from fractions import Fraction++from chipformer.dsl.graph import FFN, ONE, Expr, Graph, Lookup, Num++EXACT_LIMIT = 1 << 53 # every quantity must stay exactly representable in fp64+++class TieError(AssertionError):+ pass+++class LiChao:+ """Max over lines at integer points of [lo, hi]; nodes are created lazily."""++ def __init__(self, lo: int, hi: int):+ self.lo, self.hi = lo, hi+ self.nodes: dict[int, tuple[Num, Num, int]] = {}++ def insert(self, m: Num, b: Num, ident: int) -> None:+ new = (m, b, ident)+ idx, lo, hi = 1, self.lo, self.hi+ while True:+ cur = self.nodes.get(idx)+ if cur is None:+ self.nodes[idx] = new+ return+ mid = (lo + hi) // 2+ left = new[0] * lo + new[1] > cur[0] * lo + cur[1]+ center = new[0] * mid + new[1] > cur[0] * mid + cur[1]+ if center:+ self.nodes[idx], new = new, cur+ if lo == hi:+ return+ if left != center:+ idx, hi = 2 * idx, mid+ else:+ idx, lo = 2 * idx + 1, mid + 1++ def query(self, x: int) -> int:+ if not self.lo <= x <= self.hi:+ raise ValueError(f"query {x} outside [{self.lo}, {self.hi}]")+ idx, lo, hi = 1, self.lo, self.hi+ best_v, best_id = None, -1+ while idx in self.nodes:+ m, b, ident = self.nodes[idx]+ v = m * x + b+ if best_v is None or v > best_v:+ best_v, best_id = v, ident+ if lo == hi:+ break+ mid = (lo + hi) // 2+ if x <= mid:+ idx, hi = 2 * idx, mid+ else:+ idx, lo = 2 * idx + 1, mid + 1+ return best_id+++class BruteCache:+ def __init__(self, lo: int, hi: int):+ self.lo, self.hi = lo, hi+ self.lines: list[tuple[Num, Num]] = []++ def insert(self, m: Num, b: Num, ident: int) -> None:+ assert ident == len(self.lines)+ self.lines.append((m, b))++ def query(self, x: int) -> int:+ if not self.lo <= x <= self.hi:+ raise ValueError(f"query {x} outside [{self.lo}, {self.hi}]")+ scores = [m * x + b for m, b in self.lines]+ best = max(scores)+ winners = [i for i, s in enumerate(scores) if s == best]+ if len(winners) > 1:+ raise TieError(f"tie between positions {winners[:4]} at q={x}")+ return winners[0]+++def ev(e: Expr, vals: list[Num]) -> Num:+ return sum(c * vals[d] for d, c in e.terms.items())+++def relu(x: Num) -> Num:+ return max(0, x)+++class Evaluator:+ def __init__(self, graph: Graph, mode: str = "lichao"):+ self.g = graph+ cache_cls = {"lichao": LiChao, "brute": BruteCache}[mode]+ self.heads = [n for n in graph.nodes if isinstance(n, Lookup)]+ self.caches = {id(h): cache_cls(*h.q_range) for h in self.heads}+ self.head_values: dict[int, list[list[Num]]] = {id(h): [] for h in self.heads}+ self.pos = 0+ self.rows: list[list[Num]] | None = None # optionally keep every residual row++ def keep_rows(self) -> "Evaluator":+ self.rows = []+ return self++ def feed(self, token: int) -> list[Num]:+ """Process one position; returns its final residual row."""+ g, t = self.g, self.pos+ vals: list[Num] = [0] * len(g.dims)+ vals[ONE] = 1+ for d, fn in g.token_fns.items():+ vals[d] = fn(token)+ for d, fn in g.pos_fns.items():+ vals[d] = fn(t)+ for node in g.nodes:+ if isinstance(node, FFN):+ vals[node.out] = sum(c * ev(a, vals) * relu(ev(b, vals)) for c, a, b in node.terms)+ else:+ cache = self.caches[id(node)]+ store = self.head_values[id(node)]+ kx, ky, q = ev(node.kx, vals), ev(node.ky, vals), ev(node.q, vals)+ if isinstance(q, Fraction):+ if q.denominator != 1:+ raise ValueError(f"{node.name}: non-integer query {q}")+ q = int(q)+ lo, hi = node.q_range+ worst = abs(kx) * max(abs(lo), abs(hi)) + abs(ky)+ if worst >= EXACT_LIMIT:+ raise OverflowError(f"{node.name}: score magnitude {worst:.3g} >= 2^53")+ cache.insert(kx, ky, t)+ store.append([ev(v, vals) for v in node.values])+ src = store[cache.query(q)]+ for d, v in zip(node.out, src):+ vals[d] = v+ self.pos += 1+ if self.rows is not None:+ self.rows.append(vals)+ return vals++ def output(self, name: str, row: list[Num]) -> Num:+ return ev(self.g.outputs[name], row)diff --git a/experimental/riscv/chipformer/dsl/graph.py b/experimental/riscv/chipformer/dsl/graph.pynew file mode 100644index 0000000..cf2e341--- /dev/null+++ b/experimental/riscv/chipformer/dsl/graph.py@@ -0,0 +1,173 @@+"""Computation graph whose nodes map 1:1 onto parts of a transformer.++Every value is a scalar *dimension* of the residual stream, at every position.+Dimensions come from:++ - the embedding: the constant ``one``, any function of the input token, and+ any function of the position (a learned absolute position table)+ - ``ffn``: a sum of ReGLU units ``coef * a * ReLU(b)``, with a and b linear in+ existing dims (one dim per node, many hidden units per node)+ - ``lookup``: a hardmax attention head with 2D query/key. At position t it+ selects the unique s <= t maximizing ``q_t * kx_s + ky_s`` and copies the+ value expressions evaluated at s. The query is ``(q, 1)`` and the key is+ ``(kx, ky)``, so every head is a "max over lines at q" query, which+ allows an exact O(log n) cache.++Linear combinations are free: they fold into the weight matrices of whatever+reads them, so ``Expr`` is a linear form over dims rather than a node.++Each node is placed at the earliest legal sublayer (ASAP): attention sublayers+are odd (2L-1), FFN sublayers are even (2L), and a node may only read dims+produced at strictly earlier sublayers. Level 0 is the embedding.+"""++from collections.abc import Callable+from dataclasses import dataclass+from fractions import Fraction++Num = int | Fraction+ONE = 0 # dim id of the constant-one embedding feature+++class Expr:+ """A linear form ``sum(coef * dim)``; the constant term lives on dim ONE."""++ __slots__ = ("terms",)++ def __init__(self, terms: dict[int, Num] | None = None):+ self.terms = {k: v for k, v in (terms or {}).items() if v != 0}++ def __add__(self, other: "Expr | Num") -> "Expr":+ other = as_expr(other)+ t = dict(self.terms)+ for k, v in other.terms.items():+ t[k] = t.get(k, 0) + v+ return Expr(t)++ __radd__ = __add__++ def __neg__(self) -> "Expr":+ return Expr({k: -v for k, v in self.terms.items()})++ def __sub__(self, other: "Expr | Num") -> "Expr":+ return self + (-as_expr(other))++ def __rsub__(self, other: "Expr | Num") -> "Expr":+ return as_expr(other) - self++ def __mul__(self, c: Num) -> "Expr":+ if not isinstance(c, (int, Fraction)):+ raise TypeError("Expr can only be scaled by a constant (use ffn/gate for products)")+ return Expr({k: v * c for k, v in self.terms.items()})++ __rmul__ = __mul__++ def __truediv__(self, c: int) -> "Expr":+ return self * Fraction(1, c)++ def __repr__(self) -> str:+ return " + ".join(f"{v}*d{k}" for k, v in self.terms.items()) or "0"+++def as_expr(x: "Expr | Num") -> Expr:+ if isinstance(x, Expr):+ return x+ if isinstance(x, (int, Fraction)):+ return Expr({ONE: x})+ raise TypeError(f"cannot convert {type(x).__name__} to Expr")+++@dataclass+class Dim:+ name: str+ kind: str # "one" | "token" | "pos" | "ffn" | "attn"+ level: int+++@dataclass+class FFN:+ name: str+ terms: list[tuple[Num, Expr, Expr]] # coef * a * ReLU(b)+ out: int+ level: int+++@dataclass+class Lookup:+ name: str+ q: Expr+ kx: Expr+ ky: Expr+ values: list[Expr]+ out: list[int]+ q_range: tuple[int, int] # inclusive bounds on q, needed by the exact cache+ level: int+++class Graph:+ def __init__(self) -> None:+ self.dims: list[Dim] = [Dim("one", "one", 0)]+ self.token_fns: dict[int, Callable[[int], Num]] = {}+ self.pos_fns: dict[int, Callable[[int], Num]] = {}+ self.nodes: list[FFN | Lookup] = []+ self.outputs: dict[str, Expr] = {}+ self.one = Expr({ONE: 1})++ # --- embedding features --------------------------------------------------++ def token_feature(self, name: str, fn: Callable[[int], Num]) -> Expr:+ d = self._new_dim(name, "token", 0)+ self.token_fns[d] = fn+ return Expr({d: 1})++ def pos_feature(self, name: str, fn: Callable[[int], Num]) -> Expr:+ d = self._new_dim(name, "pos", 0)+ self.pos_fns[d] = fn+ return Expr({d: 1})++ # --- nodes ---------------------------------------------------------------++ def ffn(self, terms: list[tuple[Num, "Expr | Num", "Expr | Num"]], name: str = "ffn") -> Expr:+ terms = [(c, as_expr(a), as_expr(b)) for c, a, b in terms if c != 0]+ lvl = self.level(*(e for _, a, b in terms for e in (a, b)))+ sub = lvl + 1 if lvl % 2 == 1 else lvl + 2+ d = self._new_dim(name, "ffn", sub)+ self.nodes.append(FFN(name, terms, d, sub))+ return Expr({d: 1})++ def lookup(self, q: "Expr | Num", kx: "Expr | Num", ky: "Expr | Num",+ values: list["Expr | Num"], q_range: tuple[int, int],+ name: str = "lookup") -> list[Expr]:+ q, kx, ky = as_expr(q), as_expr(kx), as_expr(ky)+ values = [as_expr(v) for v in values]+ lvl = self.level(q, kx, ky, *values)+ sub = lvl + 1 if lvl % 2 == 0 else lvl + 2+ outs = [self._new_dim(f"{name}[{i}]", "attn", sub) for i in range(len(values))]+ self.nodes.append(Lookup(name, q, kx, ky, values, outs, q_range, sub))+ return [Expr({d: 1}) for d in outs]++ def output(self, name: str, e: "Expr | Num") -> None:+ self.outputs[name] = as_expr(e)++ # --- introspection --------------------------------------------------------++ def level(self, *exprs: Expr) -> int:+ return max((self.dims[d].level for e in exprs for d in e.terms), default=0)++ def n_layers(self) -> int:+ top = max((n.level for n in self.nodes), default=0)+ return (top + 1) // 2++ def stats(self) -> dict[str, int]:+ ffns = [n for n in self.nodes if isinstance(n, FFN)]+ heads = [n for n in self.nodes if isinstance(n, Lookup)]+ return {+ "layers": self.n_layers(),+ "d_model": len(self.dims),+ "heads": len(heads),+ "ffn_units": sum(len(n.terms) for n in ffns),+ }++ def _new_dim(self, name: str, kind: str, level: int) -> int:+ self.dims.append(Dim(name, kind, level))+ return len(self.dims) - 1diff --git a/experimental/riscv/chipformer/dsl/lib.py b/experimental/riscv/chipformer/dsl/lib.pynew file mode 100644index 0000000..1d6352b--- /dev/null+++ b/experimental/riscv/chipformer/dsl/lib.py@@ -0,0 +1,145 @@+"""Exact integer primitives built from ReGLU units and 2D hardmax lookups.++All inputs are assumed to be integers within the stated ranges; outputs are+then exact. Key identities (x integer):++ step(x) = [x >= 0] = ReLU(x + 1) - ReLU(x)+ hat(x) = [x == 0] = ReLU(x + 1) - 2 ReLU(x) + ReLU(x - 1)+ a * c = a * ReLU(c) for c in {0, 1}+ a * step(x), a * hat(x) same identities with the gate a+"""++from chipformer.dsl.graph import Expr, Graph, Num++MAX_POS = 1 << 22 # positions are kept below this by context rollover+++class Circuit(Graph):+ def __init__(self) -> None:+ super().__init__()+ self.pos = self.pos_feature("pos", lambda t: t)+ self.pos2 = self.pos_feature("pos^2", lambda t: t * t)++ # --- comparisons -----------------------------------------------------------++ def step(self, x: Expr, name: str = "step") -> Expr:+ """[x >= 0]"""+ return self.stepglu(self.one, x, name)++ def stepglu(self, a: "Expr | Num", x: Expr, name: str = "stepglu") -> Expr:+ """a * [x >= 0]"""+ return self.ffn([(1, a, x + 1), (-1, a, x)], name)++ def eq(self, x: Expr, k: "Expr | Num" = 0, name: str = "eq") -> Expr:+ """[x == k]"""+ return self.eqglu(self.one, x, k, name)++ def eqglu(self, a: "Expr | Num", x: Expr, k: "Expr | Num" = 0, name: str = "eqglu") -> Expr:+ """a * [x == k]"""+ d = x - k+ return self.ffn([(1, a, d + 1), (-2, a, d), (1, a, d - 1)], name)++ def ge(self, x: Expr, y: "Expr | Num", name: str = "ge") -> Expr:+ return self.step(x - y, name)++ def gate(self, a: "Expr | Num", c: Expr, name: str = "gate") -> Expr:+ """a * c for c in {0, 1}"""+ return self.ffn([(1, a, c)], name)++ def mux(self, c: Expr, a: "Expr | Num", b: "Expr | Num", name: str = "mux") -> Expr:+ """a if c else b, for c in {0, 1}"""+ return b + self.gate(a - b, c, name)++ def select(self, cases: list[tuple[Expr, "Expr | Num"]], name: str = "select") -> Expr:+ """sum(value * cond) for one-hot conds; one FFN node."""+ return self.ffn([(1, v, c) for c, v in cases], name)++ def all_of(self, *conds: "Expr | Num", name: str = "and") -> Expr:+ """Logical AND of binary conds in one FFN node."""+ return self.step(sum(conds, Expr()) - len(conds), name)++ def index(self, values: list["Expr | Num"], idx: Expr, name: str = "index") -> Expr:+ """values[idx] for integer 0 <= idx < len(values); one FFN node."""+ terms = []+ for i, v in enumerate(values):+ d = idx - i+ terms += [(1, v, d + 1), (-2, v, d), (1, v, d - 1)]+ return self.ffn(terms, name)++ def relu(self, x: Expr, name: str = "relu") -> Expr:+ return self.ffn([(1, self.one, x)], name)++ # --- integer arithmetic ----------------------------------------------------++ def floor_div(self, x: Expr, d: int, q_max: int, name: str = "div") -> Expr:+ """floor(x / d) for 0 <= x < d * (q_max + 1)"""+ terms = []+ for k in range(1, q_max + 1):+ terms += [(1, self.one, x - d * k + 1), (-1, self.one, x - d * k)]+ return self.ffn(terms, name)++ def divmod(self, x: Expr, d: int, q_max: int, name: str = "divmod") -> tuple[Expr, Expr]:+ q = self.floor_div(x, d, q_max, name)+ return q, x - d * q++ def wrap(self, x: Expr, m: int, name: str = "wrap") -> Expr:+ """x mod m for 0 <= x < 2m"""+ return x - m * self.step(x - m, name)++ def bits(self, x: Expr, n: int, name: str = "bits") -> list[Expr]:+ """Bits of 0 <= x < 2**n, LSB first, all computed in one FFN sublayer."""+ out = []+ for i in range(n):+ terms = []+ for m in range(1 << (n - 1 - i)):+ on, off = (2 * m + 1) << i, (2 * m + 2) << i+ terms += [(1, self.one, x - on + 1), (-1, self.one, x - on)]+ if off < (1 << n):+ terms += [(-1, self.one, x - off + 1), (1, self.one, x - off)]+ out.append(self.ffn(terms, f"{name}{i}"))+ return out++ @staticmethod+ def from_bits(bits: list[Expr]) -> Expr:+ return sum((b * (1 << i) for i, b in enumerate(bits)), Expr())++ def band_bits(self, xb: list[Expr], yb: list[Expr], name: str = "and") -> Expr:+ return self.ffn([(1 << i, x, y) for i, (x, y) in enumerate(zip(xb, yb))], name)++ def band(self, x: Expr, y: Expr, n: int = 8, name: str = "and") -> Expr:+ return self.band_bits(self.bits(x, n, f"{name}.x"), self.bits(y, n, f"{name}.y"), name)++ def bxor(self, x: Expr, y: Expr, n: int = 8, name: str = "xor") -> Expr:+ return x + y - 2 * self.band(x, y, n, name)++ def bor(self, x: Expr, y: Expr, n: int = 8, name: str = "or") -> Expr:+ return x + y - self.band(x, y, n, name)++ # --- lookups -----------------------------------------------------------------++ def prev(self, values: list[Expr], name: str = "prev") -> list[Expr]:+ """Values at position t-1 (at t=0: position 0).++ Score 2s(t-1) - s^2 = (t-1)^2 - (s-(t-1))^2 peaks exactly at s = t-1.+ """+ return self.lookup(self.pos - 1, 2 * self.pos, -self.pos2, values,+ (-1, MAX_POS), name)++ def latest(self, flag: Expr, values: list[Expr], name: str = "latest") -> list[Expr]:+ """Values at the latest position s <= t with flag(s) = 1 (flag in {0, 1})."""+ big = 2 * MAX_POS+ return self.lookup(0, 0, self.pos + big * (flag - 1), values, (0, 0), name)++ def latest_keyed(self, q: Expr, key: Expr, key_sq: Expr, is_entry: Expr,+ values: list[Expr], n_keys: int, name: str = "keyed") -> list[Expr]:+ """Values at the latest entry whose integer key equals q.++ Score T(2*key*q - key^2) + s = T(q^2 - (key-q)^2) + s: the key match+ dominates, recency breaks ties. Non-entries are pushed below every+ entry. Returns ``values + [found_key]``; if no entry has key q, the+ nearest key wins, so callers compare found_key with q.+ """+ t = MAX_POS+ big = 4 * t * n_keys * n_keys+ return self.lookup(q, 2 * t * key, -t * key_sq + self.pos + big * (is_entry - 1),+ values + [key], (0, n_keys - 1), name)diff --git a/experimental/riscv/chipformer/fuzz.py b/experimental/riscv/chipformer/fuzz.pynew file mode 100644index 0000000..6d80673--- /dev/null+++ b/experimental/riscv/chipformer/fuzz.py@@ -0,0 +1,59 @@+"""Random CHIP-8 programs that keep executing and cover every opcode.++Layout: a straight-line body of random instructions (skips allowed, calls to+one subroutine), a jump back to the start, then the subroutine. Stores+(FX33/FX55) may land anywhere in RAM, including the program itself, which+exercises self-modifying code.+"""++import random++SUB_LEN = 4+++def random_instruction(rng: random.Random, sub_addr: int) -> int:+ x, y, n, nn = rng.randrange(16), rng.randrange(16), rng.randrange(16), rng.randrange(256)+ kind = rng.choice([+ "00E0", "2NNN", "3XNN", "4XNN", "5XY0", "6XNN", "7XNN", "8XY", "9XY0", "ANNN",+ "BNNN", "CXNN", "DXYN", "DXYN", "EX9E", "EXA1", "FX07", "FX0A", "FX15", "FX18",+ "FX1E", "FX29", "FX33", "FX55", "FX65", "6XNN", "7XNN", "8XY",+ ])+ if kind == "00E0":+ return 0x00E0+ if kind == "2NNN":+ return 0x2000 | sub_addr+ if kind == "BNNN": # jump to the start of the body plus V0 (V0 small keeps it in range)+ return 0xB200+ if kind == "8XY":+ return 0x8000 | x << 8 | y << 4 | rng.choice([0, 1, 2, 3, 4, 5, 6, 7, 0xE])+ if kind == "ANNN":+ return 0xA000 | rng.choice([rng.randrange(0x200, 0x1000), rng.randrange(0x300, 0x400)])+ code = {+ "3XNN": 0x3000 | x << 8 | nn, "4XNN": 0x4000 | x << 8 | nn,+ "5XY0": 0x5000 | x << 8 | y << 4, "9XY0": 0x9000 | x << 8 | y << 4,+ "6XNN": 0x6000 | x << 8 | nn, "7XNN": 0x7000 | x << 8 | nn,+ "CXNN": 0xC000 | x << 8 | nn, "DXYN": 0xD000 | x << 8 | y << 4 | n,+ "EX9E": 0xE09E | x << 8, "EXA1": 0xE0A1 | x << 8,+ }.get(kind)+ if code is not None:+ return code+ return 0xF000 | x << 8 | int(kind.removeprefix("FX"), 16)+++def random_quirks(seed: int) -> tuple[str, ...]:+ """A random quirk profile (each flag on with probability 1/2)."""+ from chipformer.spec import QUIRK_NAMES++ rng = random.Random(seed * 7919 + 1)+ return tuple(q for q in QUIRK_NAMES if rng.random() < 0.5)+++def random_program(seed: int, body: int = 120) -> bytes:+ rng = random.Random(seed)+ sub_addr = 0x200 + 2 * (body + 1)+ words = [random_instruction(rng, sub_addr) for _ in range(body)]+ words.append(0x1200)+ words += [random_instruction(rng, sub_addr) for _ in range(SUB_LEN - 1)]+ words = [w if w != 0x2000 | sub_addr or i < body else 0x6000 for i, w in enumerate(words)]+ words.append(0x00EE)+ return b"".join(w.to_bytes(2, "big") for w in words)diff --git a/experimental/riscv/chipformer/games.py b/experimental/riscv/chipformer/games.pynew file mode 100644index 0000000..bcefa2e--- /dev/null+++ b/experimental/riscv/chipformer/games.py@@ -0,0 +1,387 @@+"""Game library: fetch ROMs and metadata, build the manifest, validate on the engine.++ uv run python -m chipformer.games fetch # download sources into games/sources/+ uv run python -m chipformer.games build # games/manifest.json + games/roms/+ uv run python -m chipformer.games validate # run every game on the engine, verify traces++Sources:+ - CHIP-8 Archive (github.com/JohnEarnest/chip8Archive): CC0 games with Octo quirk flags+ - classic games (github.com/kripod/chip8-roms, the Revival Studios pack: "can be freely+ distributed in its original form")+ - CHIP-8 database (github.com/chip-8/chip-8-database): title, authors, platform, tickrate+ and keys for known ROMs, keyed by SHA-1+"""++import argparse+import hashlib+import json+import urllib.parse+import urllib.request+from pathlib import Path++ROOT = Path(__file__).resolve().parent.parent+GAMES = ROOT / "games"+SOURCES = GAMES / "sources"++ARCHIVE = "https://raw.githubusercontent.com/JohnEarnest/chip8Archive/master"+DATABASE = "https://raw.githubusercontent.com/chip-8/chip-8-database/master/database"+CLASSICS_API = "https://api.github.com/repos/kripod/chip8-roms/contents/games"+CLASSICS_RAW = "https://raw.githubusercontent.com/kripod/chip8-roms/master/games"++LICENSES = {+ "archive": "CC0",+ "classics": "freely distributable (Revival Studios CHIP-8 pack)",+}+++def _get(url: str) -> bytes:+ req = urllib.request.Request(url, headers={"User-Agent": "chipformer"})+ with urllib.request.urlopen(req, timeout=60) as r:+ return r.read()+++def fetch() -> None:+ """Download every source into games/sources/ (data only; nothing is executed)."""+ (SOURCES / "database").mkdir(parents=True, exist_ok=True)+ for name in ["programs.json", "sha1-hashes.json", "platforms.json", "quirks.json"]:+ (SOURCES / "database" / name).write_bytes(_get(f"{DATABASE}/{name}"))++ arch = SOURCES / "archive"+ (arch / "roms").mkdir(parents=True, exist_ok=True)+ for name in ["programs.json", "authors.json"]:+ (arch / name).write_bytes(_get(f"{ARCHIVE}/{name}"))+ programs = json.loads((arch / "programs.json").read_text(encoding="utf-8"))+ for key, p in programs.items():+ if p.get("platform", "chip8") == "chip8":+ (arch / "roms" / f"{key}.ch8").write_bytes(_get(f"{ARCHIVE}/roms/{key}.ch8"))++ cls = SOURCES / "classics"+ cls.mkdir(parents=True, exist_ok=True)+ listing = json.loads(_get(CLASSICS_API))+ for entry in listing:+ if entry["name"].lower().endswith(".ch8"):+ url = f"{CLASSICS_RAW}/{urllib.parse.quote(entry['name'])}"+ (cls / entry["name"]).write_bytes(_get(url))+ print(f"archive: {len(list((arch / 'roms').glob('*.ch8')))} ROMs, "+ f"classics: {len(list(cls.glob('*.ch8')))} ROMs, database: "+ f"{len(json.loads((SOURCES / 'database' / 'programs.json').read_text(encoding='utf-8')))} programs")+++def sha1(data: bytes) -> str:+ return hashlib.sha1(data).hexdigest()+++# CHIP-8 database platforms we can run (lores CHIP-8 instruction set), best first+RUNNABLE_PLATFORMS = ["modernChip8", "originalChip8", "hybridVIP", "chip48", "superchip1",+ "superchip"]+# Octo's keyboard convention: WASD = keypad 5/7/8/9, E = keypad 6+OCTO_KEYS = {"up": 5, "left": 7, "down": 8, "right": 9, "a": 6}++# Games shown in the demo, in display order, with key roles where the database has none+# (from each game's own documentation) and a one-line control hint. Role names follow the+# CHIP-8 database: up/down/left/right/a/b, player2Up/player2Down.+FEATURED = {+ "tetris": ({}, "Left/Right move, Down drops faster, Space rotates"),+ "space-invaders": ({}, "Left/Right move, Space fires (press Space to start)"),+ "brix": ({"left": 4, "right": 6}, "Left/Right move the paddle"),+ "blinky": ({"up": 6, "down": 3, "left": 7, "right": 8}, "Arrow keys steer through the maze"),+ "pong": ({"up": 1, "down": 4, "player2Up": 12, "player2Down": 13},+ "Left paddle A/D (or W/S), right paddle Left/Right (or Up/Down)"),+ "super-pong": ({}, "WASD moves the paddles, Space (E) resets"),+ "ufo": ({"left": 4, "a": 5, "right": 6}, "Left/Space/Right fire left, up, right"),+ "missile-command": ({}, "Space fires"),+ "br8kout": ({}, "Left/Right move the paddle"),+ "cave-explorer": ({}, "Arrow keys move"),+ "outlaw": ({}, "Arrow keys move, Space shoots"),+ "flight-runner": ({}, "Up/Down steer"),+ "down8": ({}, "Left/Right move"),+ "dino-run": ({}, "Up or Space jumps"),+}++# Per-game play tuning. "ipf" overrides the platform's default speed. "tap" lists key roles+# delivered as one short pulse per physical press: games that act on every pass of their loop+# while a key is held (Tetris rotates once per pass) otherwise repeat a press several times.+# Measured on Tetris: at 30 instructions/frame a 150 ms press rotates twice; at 15 it rotates+# once even for a 200 ms hold.+#+# "game_over" lets the cabinet end games that never end themselves: Fran Dachille's Tetris+# (1991, HP48) has no game-over detection (its manual says "ENTER - restart"), so once the+# well is full it spawns pieces into the stack forever. The host declares the game over+# when any pixel in the given screen rows/columns stays lit for `seconds`. For Tetris, pieces+# spawn in rows 2-5; rows 6-7 are only crossed by falling pieces (at most 2.4 s in simulated+# play) until the stack reaches them. This is the host reading the screen, not the model.+TUNING = {+ "tetris": {"ipf": 15, "tap": ["a"],+ "game_over": {"rows": [6, 7], "cols": [27, 35], "seconds": 5,+ "message": "The well is full"}},+}+TAP_FRAMES = 3 # length of a tap pulse, in 60 Hz frames++# Ids for classics whose titles collide: the well-known original gets the plain name.+CLASSIC_IDS = {+ "Pong [Paul Vervalin, 1990].ch8": "pong",+ "Pong (1 player).ch8": "pong-1-player",+}+++def _slug(title: str) -> str:+ return "".join(ch if ch.isalnum() else "-" for ch in title.lower()).strip("-").replace("--", "-")+++def _short(text: str | None, limit: int = 280) -> str:+ text = " ".join((text or "").split())+ return text if len(text) <= limit else text[: limit - 1].rsplit(" ", 1)[0] + "…"+++def _db_quirks(platform: dict) -> tuple[list[str], list[str]]:+ """Map a CHIP-8 database platform's quirks onto our flags; returns (flags, notes)."""+ q = platform["quirks"]+ flags, notes = [], []+ if q.get("shift"):+ flags.append("shift")+ if q.get("memoryLeaveIUnchanged") or q.get("memoryIncrementByX"):+ flags.append("loadstore")+ if q.get("memoryIncrementByX"):+ notes.append("platform increments I by X after FX55/FX65; run with I unchanged")+ if not q.get("wrap"):+ flags.append("clip")+ if q.get("jump"):+ flags.append("jump")+ if q.get("logic"):+ flags.append("logic")+ if q.get("vblank"):+ notes.append("platform waits for vblank on draw; not modelled")+ return flags, notes+++def build() -> list[dict]:+ """Assemble games/manifest.json and games/roms/ from the downloaded sources."""+ db = json.loads((SOURCES / "database" / "programs.json").read_text(encoding="utf-8"))+ hashes = json.loads((SOURCES / "database" / "sha1-hashes.json").read_text(encoding="utf-8"))+ platforms = {p["id"]: p for p in json.loads(+ (SOURCES / "database" / "platforms.json").read_text(encoding="utf-8"))}+ archive = json.loads((SOURCES / "archive" / "programs.json").read_text(encoding="utf-8"))+ authors = json.loads((SOURCES / "archive" / "authors.json").read_text(encoding="utf-8"))+ old = {g["id"]: g for g in load_manifest()} if (GAMES / "manifest.json").exists() else {}++ games: list[dict] = []+ seen: set[str] = set()++ def add(entry: dict, rom: bytes, gid: str | None = None) -> None:+ base = gid = gid or _slug(entry["title"])+ n = 2+ while gid in seen:+ gid, n = f"{base}-{n}", n + 1+ seen.add(gid)+ entry = {"id": gid, **entry, "file": f"roms/{gid}.ch8", "sha1": sha1(rom)}+ for keep in ("status", "result"): # validation results survive rebuilds+ if keep in old.get(gid, {}):+ entry[keep] = old[gid][keep]+ if gid in FEATURED:+ keys, hint = FEATURED[gid]+ entry["keys"] = {**entry["keys"], **keys}+ entry["controls"] = hint+ entry["featured"] = list(FEATURED).index(gid)+ tuning = TUNING.get(gid, {})+ entry["ipf"] = tuning.get("ipf", entry["ipf"])+ entry["tap"] = tuning.get("tap", [])+ if "game_over" in tuning:+ entry["game_over"] = tuning["game_over"]+ (GAMES / "roms").mkdir(parents=True, exist_ok=True)+ (GAMES / "roms" / f"{gid}.ch8").write_bytes(rom)+ games.append(entry)++ def db_entry(rom: bytes):+ i = hashes.get(sha1(rom))+ if i is None:+ return None, None+ prog = db[int(i)]+ return prog, prog["roms"][sha1(rom)]++ for key, p in archive.items():+ if p.get("platform", "chip8") != "chip8":+ continue+ rom = (SOURCES / "archive" / "roms" / f"{key}.ch8").read_bytes()+ opts = p.get("options", {})+ flags = [name for name, opt in [("shift", "shiftQuirks"), ("loadstore", "loadStoreQuirks"),+ ("clip", "clipQuirks"), ("logic", "logicQuirks"),+ ("jump", "jumpQuirks"), ("vforder", "vfOrderQuirks")]+ if opts.get(opt) in (True, 1, "true")]+ _, dbrom = db_entry(rom)+ add({+ "title": p["title"],+ "authors": [authors.get(a, {}).get("name", a) if isinstance(authors.get(a), dict) else a+ for a in p.get("authors", [])],+ "year": (p.get("release") or "")[:4] or None,+ "description": _short(p.get("desc")),+ "source": f"{ARCHIVE.replace('raw.githubusercontent.com', 'github.com')}/roms/{key}.ch8",+ "license": LICENSES["archive"],+ "platform": "octo",+ "quirks": flags,+ "ipf": int(opts.get("tickrate", 20)),+ "keys": (dbrom or {}).get("keys") or OCTO_KEYS,+ "notes": [],+ }, rom)++ for path in sorted((SOURCES / "classics").glob("*.ch8")):+ if "(alt)" in path.name:+ continue+ rom = path.read_bytes()+ prog, dbrom = db_entry(rom)+ name = path.stem.split(" [")[0].split(" (")[0]+ plats = [pl for pl in (dbrom or {}).get("platforms", []) if pl in RUNNABLE_PLATFORMS]+ plat = plats[0] if plats else "originalChip8"+ flags, notes = _db_quirks(platforms[plat])+ if not plats:+ notes.append("not in the CHIP-8 database as a lores CHIP-8 game; COSMAC quirks assumed")+ add({+ "title": prog["title"] if prog else name,+ "authors": (prog or {}).get("authors", []),+ "year": (prog or {}).get("release"),+ "description": _short((dbrom or {}).get("description") or (prog or {}).get("description")),+ "source": f"https://github.com/kripod/chip8-roms/blob/master/games/{urllib.parse.quote(path.name)}",+ "license": LICENSES["classics"],+ "platform": plat,+ "quirks": flags,+ "ipf": int((dbrom or {}).get("tickrate") or platforms[plat].get("defaultTickrate", 15)),+ "keys": (dbrom or {}).get("keys") or {},+ "notes": notes,+ }, rom, CLASSIC_IDS.get(path.name))++ missing = set(FEATURED) - {g["id"] for g in games}+ if missing:+ raise ValueError(f"featured games not found: {sorted(missing)}")+ (GAMES / "manifest.json").write_text(json.dumps({"games": games}, indent=1), encoding="utf-8")+ write_attribution(games)+ print(f"manifest: {len(games)} games, {len(FEATURED)} featured")+ return games+++def write_attribution(games: list[dict], path: Path | None = None) -> None:+ lines = [+ "# Game credits",+ "",+ "ROMs in `roms/`. CHIP-8 Archive games are CC0. The classics come from the Revival",+ "Studios CHIP-8 pack, which \"can be freely distributed in its original form\"; each",+ "game belongs to its author.",+ "",+ "| Game | Authors | Year | License | Source |",+ "|---|---|---|---|---|",+ ]+ for g in sorted(games, key=lambda g: g["title"].lower()):+ lines.append(f"| {g['title']} | {', '.join(g['authors']) or 'unknown'} | {g.get('year') or ''}"+ f" | {g['license']} | [link]({g['source']}) |")+ (path or GAMES / "ATTRIBUTION.md").write_text("\n".join(lines) + "\n", encoding="utf-8")+++def load_manifest() -> list[dict]:+ return json.loads((GAMES / "manifest.json").read_text(encoding="utf-8"))["games"]+++# opcodes outside lores CHIP-8 (SCHIP / XO-CHIP); executing one means the game won't run right+def foreign_opcode(op: int) -> str | None:+ hi, n, nn = op >> 12, op & 0xF, op & 0xFF+ if hi == 0 and op not in (0x00E0, 0x00EE):+ return f"{op:04X} (machine code / SCHIP)"+ if hi == 0xD and n == 0:+ return "DXY0 (SCHIP 16x16 sprite)"+ if hi == 0x5 and n != 0:+ return f"5XY{n:X} (XO-CHIP)"+ if hi == 0xF and nn in (0x00, 0x01, 0x02, 0x30, 0x3A, 0x75, 0x85):+ return f"FX{nn:02X} (SCHIP/XO-CHIP)"+ return None+++def validate(ids: list[str] | None = None, instr: int = 20000) -> None:+ """Reference profile + free-running engine run + trace verification per game."""+ import os+ import random+ import subprocess++ import numpy as np++ from chipformer.check import verify_trace+ from chipformer.ref.emu import fetch as fetch_op+ from chipformer.trace import Machine++ exe = ROOT / "engine" / "target" / "release" / ("engine.exe" if os.name == "nt" else "engine")+ games = load_manifest()+ tmp = Path(os.environ.get("TEMP", "/tmp")) / "chipformer-validate"+ tmp.mkdir(exist_ok=True)+ shots = GAMES / "shots"+ shots.mkdir(exist_ok=True)+ for g in games:+ if ids and g["id"] not in ids:+ continue+ rom = (GAMES / g["file"]).read_bytes()+ quirks = ",".join(g["quirks"])+ # 1. reference: executed opcodes, tokens per instruction+ rng = random.Random(0)+ m = Machine(rom, tuple(g["quirks"]))+ foreign: dict[str, int] = {}+ for i in range(instr):+ if i % g["ipf"] == 0:+ m.tick()+ if rng.random() < 0.2:+ m.set_key(rng.randrange(16), rng.random() < 0.5)+ f = foreign_opcode(fetch_op(m.emu.cells))+ if f:+ foreign[f] = foreign.get(f, 0) + 1+ m.step()+ tok_per_instr = (len(m.tokens) - m.generated.index(True)) / instr+ need = tok_per_instr * g["ipf"] * 60+ # 2. the transformer, free-running, verified record by record+ dump = tmp / f"{g['id']}.u32"+ r = subprocess.run([str(exe), "run", str(ROOT / "model"), str(GAMES / g["file"]),+ "--instr", str(instr), "--ipf", str(g["ipf"]), "--quirks", quirks,+ "--dump", str(dump)], check=False, capture_output=True, text=True)+ if r.returncode != 0:+ verified, err = 0, "engine error: " + r.stderr.strip().splitlines()[-1]+ else:+ toks = np.fromfile(dump, dtype="<u4").tolist()+ verified, err = verify_trace(toks)+ save_screenshot(toks, shots / f"{g['id']}.png")+ status = "unsupported" if foreign else ("ok" if err is None else "mismatch")+ g["status"] = status+ g["result"] = {"verified_records": verified, "error": err, "foreign_opcodes": foreign,+ "tokens_per_instruction": round(tok_per_instr, 2),+ "realtime_tok_per_s": round(need)}+ print(f"{g['id']:32} {status:11} {verified:7,} records {need:8,.0f} tok/s needed"+ f" {', '.join(foreign) if foreign else ''}{err or ''}", flush=True)+ (GAMES / "manifest.json").write_text(json.dumps({"games": games}, indent=1), encoding="utf-8")+++def save_screenshot(tokens: list[int], path: Path, scale: int = 6) -> None:+ """The final screen of a (verified) trace, rendered from the model's own writes."""+ from PIL import Image++ from chipformer import spec+ from chipformer.ref.emu import apply_writes, display_pixels+ from chipformer.trace import parse++ cells = [0] * spec.NUM_CELLS+ for rec in parse(tokens, allow_reboot=True):+ if rec.kind == "boot":+ cells = [0] * spec.NUM_CELLS+ apply_writes(cells, rec.writes)+ img = Image.new("RGB", (spec.DISPLAY_W, spec.DISPLAY_H))+ img.putdata([(221, 245, 158) if p else (25, 34, 31) for row in display_pixels(cells) for p in row])+ img.resize((spec.DISPLAY_W * scale, spec.DISPLAY_H * scale), Image.NEAREST).save(path)+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("command", choices=["fetch", "build", "validate"])+ ap.add_argument("ids", nargs="*", help="validate only these game ids")+ ap.add_argument("--instr", type=int, default=20000)+ args = ap.parse_args()+ if args.command == "fetch":+ fetch()+ elif args.command == "build":+ build()+ else:+ validate(args.ids or None, args.instr)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/hf_export.py b/experimental/riscv/chipformer/hf_export.pynew file mode 100644index 0000000..377985d--- /dev/null+++ b/experimental/riscv/chipformer/hf_export.py@@ -0,0 +1,262 @@+"""Write a Hugging Face model repo for chipformer (transformers, trust_remote_code).++ uv run python -m chipformer.hf_export --out dist/hf++The weights are compiled fresh from the circuits (0.3 s), so the export never depends on a+stale model/ directory. The repo holds config.json (with auto_map), model.safetensors (fp64),+the configuration/modeling code from hf/, the sparse model.chipformer used by the browser+engine, a model card and a Tetris example.+"""++import argparse+import json+import shutil+from pathlib import Path++import numpy as np++from chipformer import spec+from chipformer.dsl.lib import MAX_POS++ROOT = Path(__file__).resolve().parent.parent+HF_SRC = ROOT / "hf"+GAMES = ROOT / "games"+ROLLOVER_AT = 250_000 # same as the engine+LINKS = {+ "github": "https://github.com/divagr18/chipformer",+ "space": "https://huggingface.co/spaces/divagr1925/chipformer",+ "demo": "https://divagr.com/chipformer",+}+EXAMPLE_GAME = "tetris"+++def hf_config(w: dict[str, np.ndarray], config: dict) -> dict:+ layers = []+ for i, lay in enumerate(config["layers"]):+ heads = [[int(w[f"layers.{i}.attn.{h}.wq"].shape[0]), int(w[f"layers.{i}.attn.{h}.wv"].shape[0])]+ for h in range(lay["heads"])]+ layers.append({"heads": heads, "ffn_units": lay["ffn_units"], "head_names": lay["head_names"]})+ return {+ "architectures": ["ChipformerForCausalLM"],+ "auto_map": {+ "AutoConfig": "configuration_chipformer.ChipformerConfig",+ "AutoModelForCausalLM": "modeling_chipformer.ChipformerForCausalLM",+ },+ "model_type": "chipformer",+ "d_model": config["d_model"],+ "vocab_size": config["vocab_size"],+ "beta": config["beta"],+ "layers": layers,+ "machine": config["machine"],+ "max_position": MAX_POS,+ "rollover_at": ROLLOVER_AT,+ "params": config["params"],+ "nonzero_params": config["nonzero_params"],+ "dtype": "float64",+ "torch_dtype": "float64",+ "tie_word_embeddings": False,+ }+++def example_game() -> dict:+ games = json.loads((GAMES / "manifest.json").read_text())+ games = games["games"] if isinstance(games, dict) else games+ return next(g for g in games if g["id"] == EXAMPLE_GAME)+++def model_card(config: dict, game: dict) -> str:+ heads = sum(lay["heads"] for lay in config["layers"])+ units = sum(lay["ffn_units"] for lay in config["layers"])+ keys = game["keys"]+ return f"""---+license: mit+library_name: transformers+pipeline_tag: text-generation+tags:+- chip-8+- emulator+- mechanistic-interpretability+- compiled-transformer+- custom_code+---++# chipformer++**Tetris, running entirely inside a transformer.**++This model *is* a CHIP-8 computer. Give it a game as the prompt and greedy decoding runs the game:+every token it predicts is one write to the machine's memory (a register, the program counter, a+row of pixels). Paint those writes onto a 64×32 screen and you are playing Tetris. Your keypresses+and the 60 Hz clock go into the context window. Everything else, every frame, every collision,+every point on the scoreboard, comes out of the model.++There is no emulator behind it. It is the same loop every language model runs: read the context,+predict the next token, append it, repeat.++- One set of weights plays Tetris, Space Invaders, Brix, Pong and 100+ other games. Switching games+ means switching the prompt.+- An ordinary decoder-only architecture: {config['n_layers']} layers, d_model {config['d_model']},+ {heads} attention heads, {units:,} ReGLU units, vocabulary {config['vocab_size']:,}, float64.++**Play it in your browser:** [{LINKS['demo'].removeprefix('https://')}]({LINKS['demo']}) ·+[Hugging Face Space]({LINKS['space']}) · Source: [GitHub]({LINKS['github']})++## How it works++- **The ROM is the prompt.** The prompt is `BOOT`, then one `ADDR a, VAL v` token pair per nonzero+ byte of machine state, then `STEP`.+- **Every output token is a write.** Each instruction becomes a short record of writes such as+ `@V3 =1F @PC =22A STEP`. The screen is drawn from these writes.+- **The host injects the outside world.** A `TICK` token arrives once per 60 Hz frame, and keypresses+ are injected as writes. Every other token is the model's greedy prediction.+- **Attention is memory.** To read register V3, a head attends to the most recent write to V3.+ Keys use a parabola over the address plus a recency term, so the hardmax lands exactly there.+- **ReLU is the ALU.** Addition, comparison, XOR and opcode decoding are all built from ReLUs,+ using identities like step(x) = ReLU(x+1) - ReLU(x).++The architecture is ordinary:+- token embedding plus a position embedding from [t, t²]+- causal softmax attention (β = {config['beta']:g}) with residual connections+- ReGLU feed-forward layers+- a linear unembedding+- no normalization++`modeling_chipformer.py` contains nothing CHIP-8 specific apart from the host loop, which only feeds+tokens in and reads the screen back.++**Where the weights come from.** They are compiled from a description of the CHIP-8 circuit+(written in a small DSL in the GitHub repo) rather than learned, which is what makes every+instruction exact. {config['nonzero_params']:,} of the {config['params']:,} parameters are nonzero.++## Usage++```python+import torch+from huggingface_hub import hf_hub_download+from transformers import AutoModelForCausalLM++repo = "divagr1925/chipformer"+model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True, dtype=torch.float64)++rom = open(hf_hub_download(repo, "roms/{game['id']}.ch8"), "rb").read()+game = model.play(rom, quirks="{','.join(game['quirks'])}", ipf={game['ipf']})+for frame in range(120):+ game.frame(keys={{{keys['left']}}} if frame % 20 < 4 else ()) # CHIP-8 key {keys['left']} = left+print(game.ascii())+```++`play_tetris.py` animates the game in a terminal: `python play_tetris.py`.++The model is a normal `PreTrainedModel`, so you can also call it directly:+- `model(input_ids, past_key_values=cache)` returns next-token logits over the {config['vocab_size']:,}-token vocabulary.+- `game.tokens` (with `keep_tokens=True`) gives you the full trace.++### Notes++- **Use float64.** The memory-read heads compare attention scores of about 10¹³, and these are only+ exact in float64. fp16 and bf16 will break the machine.+- **Speed.** Decoding is greedy, one token at a time with a KV cache. On a CPU in PyTorch that is+ about 200 tokens/s, or 2–3 frames per second of Tetris (the game wants 60).+ - `game = model.play(...)` multiplies by sparse copies of the big feed-forward matrices, since only+ 0.4% of the weights are nonzero. The outputs are identical; pass `sparse=False` to run the dense+ weights exactly as loaded.+ - The Rust/wasm engine in the GitHub repo uses the same weights with sparse matmuls and exact+ attention shortcuts. It runs the games at full speed in a browser.+- **Context.** Positions must stay below {MAX_POS:,} (2²²). `ChipformerGame` restarts the context+ from a state dump every {ROLLOVER_AT:,} tokens, as the engine does.+- **Coverage.** The weights implement the original CHIP-8 instruction set with Octo's quirk flags.+ SCHIP/XO-CHIP opcodes and the vBlank display-wait quirk are not supported.++## Files++| File | |+|---|---|+| `model.safetensors` | dense float64 weights (what `transformers` loads) |+| `model.chipformer` | the same weights, sparse (nonzeros only), for the Rust/wasm engine |+| `configuration_chipformer.py`, `modeling_chipformer.py` | the architecture plus the host loop |+| `play_tetris.py` | a terminal demo |+| `roms/{game['id']}.ch8` | {game['title']} by {', '.join(game['authors'])} ({game['year']}), {game['license']} |+"""+++EXAMPLE = '''"""Watch chipformer play Tetris in your terminal. Every pixel is a token the model predicted.++ python play_tetris.py # from the Hugging Face Hub+ python play_tetris.py path/to/repo # from a local copy+"""++import sys+import time+from pathlib import Path++import torch+from transformers import AutoModelForCausalLM++sys.stdout.reconfigure(encoding="utf-8") # the screen is drawn with block characters+REPO = sys.argv[1] if len(sys.argv) > 1 else "divagr1925/chipformer"+FRAMES = int(sys.argv[2]) if len(sys.argv) > 2 else 400+LEFT, RIGHT, DOWN, ROTATE = __KEYS__++model = AutoModelForCausalLM.from_pretrained(REPO, trust_remote_code=True, dtype=torch.float64)+if Path(REPO).is_dir():+ rom = Path(REPO, "roms/tetris.ch8").read_bytes()+else:+ from huggingface_hub import hf_hub_download++ rom = Path(hf_hub_download(REPO, "roms/tetris.ch8")).read_bytes()++game = model.play(rom, quirks="__QUIRKS__", ipf=__IPF__)+# a little scripted play: rotate (a tap), slide, drop+plan = {0: [ROTATE], 1: [], 6: [LEFT], 7: [], 8: [LEFT], 9: [], 14: [DOWN]}+t0, fed0 = time.time(), game.fed+print("\\x1b[2J", end="")+for f in range(FRAMES):+ keys = plan.get(f % 40, [] if f % 40 < 14 else [DOWN])+ game.frame(keys)+ rate = (game.fed - fed0) / (time.time() - t0)+ print(f"\\x1b[H{game.ascii()}\\nframe {f + 1}/{FRAMES} · {game.instructions} instructions · "+ f"{rate:,.0f} tokens/s · position {game.position:,}", flush=True)+'''+++def export(out: Path) -> None:+ from safetensors.numpy import save_file++ from chipformer.chip8 import build+ from chipformer.compile import compile_graph, save_sparse++ out.mkdir(parents=True, exist_ok=True)+ w, config = compile_graph(build())+ assert config["vocab_size"] == spec.VOCAB_SIZE+ save_file({k: np.ascontiguousarray(v) for k, v in w.items()}, out / "model.safetensors",+ metadata={"format": "pt"})+ save_sparse(w, config, out / "model.chipformer")+ (out / "config.json").write_text(json.dumps(hf_config(w, config), indent=1))+ for f in ("configuration_chipformer.py", "modeling_chipformer.py"):+ shutil.copy(HF_SRC / f, out / f)++ game = example_game()+ (out / "roms").mkdir(exist_ok=True)+ shutil.copy(GAMES / game["file"], out / "roms" / f"{game['id']}.ch8")+ keys = game["keys"]+ (out / "play_tetris.py").write_text(+ EXAMPLE.replace("__KEYS__", f"{keys['left']}, {keys['right']}, {keys['down']}, {keys['a']}")+ .replace("__QUIRKS__", ",".join(game["quirks"])).replace("__IPF__", str(game["ipf"])),+ encoding="utf-8")+ (out / "README.md").write_text(model_card(config, game), encoding="utf-8")+ (out / ".gitattributes").write_text(+ "*.safetensors filter=lfs diff=lfs merge=lfs -text\n"+ "*.chipformer filter=lfs diff=lfs merge=lfs -text\n")+++def main() -> None:+ ap = argparse.ArgumentParser(description="Export chipformer as a Hugging Face model repo")+ ap.add_argument("--out", type=Path, default=Path("dist/hf"))+ out = ap.parse_args().out+ export(out)+ files = [f for f in out.rglob("*") if f.is_file()]+ print(f"{out}: {len(files)} files, {sum(f.stat().st_size for f in files) / 1e6:.1f} MB")+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/__init__.py b/experimental/riscv/chipformer/q35/__init__.pynew file mode 100644index 0000000..ff50fef--- /dev/null+++ b/experimental/riscv/chipformer/q35/__init__.py@@ -0,0 +1,8 @@+"""CHIP-8 inside a stock Qwen3.5 checkpoint: bf16-exact circuits built from Gated DeltaNet and SwiGLU."""++try: # the chip is exact only without bf16 split-K reductions, which cuBLAS may pick for some shapes+ import torch++ torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False+except ImportError:+ passdiff --git a/experimental/riscv/chipformer/q35/arith_bench.py b/experimental/riscv/chipformer/q35/arith_bench.pynew file mode 100644index 0000000..3711837--- /dev/null+++ b/experimental/riscv/chipformer/q35/arith_bench.py@@ -0,0 +1,1064 @@+"""Standard arithmetic benchmarks: merged (the computer in the weights) vs base Qwen3.5-4B.++ python -m chipformer.q35.arith_bench run COND DATA [DATA ...] [--model PATH] [--batch 16]+ python -m chipformer.q35.arith_bench replay COND DATA [DATA ...] # chip exactness (IR)+ python -m chipformer.q35.arith_bench identity --model MERGED # BatchDecoder == generate+ python -m chipformer.q35.arith_bench summary++Datasets (problems sampled with a fixed seed, so every condition sees the same ones):++ dmx_add_or_sub_big, dmx_mul_big, dmx_mixed_longer, dmx_add_sub_multiple_longer+ dmi_add_or_sub, dmi_mul, dmi_add_sub_multiple, dmi_mixed+ dmxc_add_or_sub_big, dmxc_mul_big+ the same extrapolate test files, sampled from their clean problems only+ DeepMind Mathematics Dataset v1.0 test files (extrapolate / interpolate); the files+ alternate question / answer lines+ nupa_add, nupa_sub, nupa_mulh, nupa_mule+ NUPA_text test.json, integer add / sub / multiply_hard / multiply_easy, sampled evenly+ over digit lengths 3-20 (groups 3-6, 7-10, 11-15, 16-20). NUPA's "Directly return the+ answer as an integer ..." preamble and its "= answer" are dropped; the task sentence+ ("Add two numbers: 830 + 70") is kept verbatim+ bbh_mat2 BIG-Bench Hard multistep_arithmetic_two (the trailing " =" is dropped)+ eleu_<subset> EleutherAI/arithmetic (the "Q: ... A:" wrapper is dropped)++Conditions, prompts, decoding and judging are evals.py's (same SUFFIX, budgets, BIOS, bans,+BatchDecoder): this file only adds the loaders, a multi-dataset runner (one model load per+condition), an exact answer check (pred == gold as rationals: no tolerance, decimals and+fractions compared exactly), the circuit replay and the summary.++``clean`` = every number in the problem is a non-negative integer, there is no division and+the answer is a non-negative integer (what the computer's add / sub / mul can do end to end).+"""++import argparse+import json+import math+import random+import re+import sys+import time+from decimal import Decimal, InvalidOperation+from fractions import Fraction+from pathlib import Path++from chipformer.q35 import evals as E++_fmt = E.fmt+++def _safe_fmt(v) -> str:+ """evals.fmt with enough decimal precision for huge values (runaway digit strings in a+ transcript would otherwise raise InvalidOperation and end the run)."""+ import decimal++ with decimal.localcontext() as c:+ c.prec = 100_000+ try:+ return _fmt(v)+ except (decimal.InvalidOperation, ValueError, OverflowError):+ return str(v)+++E.fmt = _safe_fmt # judge() looks fmt up in its module at call time+ROOT = Path(__file__).resolve().parents[2]+HOME = Path.home()+DATA = HOME / "data"+OUT = ROOT / "dist" / "arith"+sys.set_int_max_str_digits(0)++DM_MODULES = {"dmx_add_or_sub_big": "extrapolate/arithmetic__add_or_sub_big",+ "dmx_mul_big": "extrapolate/arithmetic__mul_big",+ "dmx_mixed_longer": "extrapolate/arithmetic__mixed_longer",+ "dmx_add_sub_multiple_longer": "extrapolate/arithmetic__add_sub_multiple_longer",+ "dmx_div_big": "extrapolate/arithmetic__div_big",+ "dmx_mul_div_multiple_longer": "extrapolate/arithmetic__mul_div_multiple_longer",+ "dmi_add_or_sub": "interpolate/arithmetic__add_or_sub",+ "dmi_mul": "interpolate/arithmetic__mul",+ "dmi_add_sub_multiple": "interpolate/arithmetic__add_sub_multiple",+ "dmi_mixed": "interpolate/arithmetic__mixed"}+NUPA_TASKS = {"nupa_add": ("add_Integer_Integer_Integer", "add", "+"),+ "nupa_sub": ("sub_Integer_Integer_Integer", "sub", "-"),+ "nupa_mulh": ("multiply_hard_Integer_Integer_Integer", "mul", "*"),+ "nupa_mule": ("multiply_easy_Integer_Integer_Integer", "mul", "*")}+NUPA_GROUPS = [(3, 6), (7, 10), (11, 15), (16, 20)]+ELEU = ["two_digit_addition", "two_digit_subtraction", "two_digit_multiplication",+ "three_digit_addition", "three_digit_subtraction", "four_digit_addition",+ "four_digit_subtraction", "five_digit_addition", "five_digit_subtraction",+ "single_digit_three_ops"]+ELEU_OPS = {"addition": ("add", "plus"), "subtraction": ("sub", "minus"),+ "multiplication": ("mul", "times")}+DMXC = ["dmxc_add_or_sub_big", "dmxc_mul_big"]+DM_DIV = ["dmx_div_big", "dmx_mul_div_multiple_longer", "dmxc_div_big"] # v14: division+ALL = [*DM_MODULES, *DMXC, "dmxc_div_big", *NUPA_TASKS, "bbh_mat2", *(f"eleu_{s}" for s in ELEU)]+DEFAULT_N = {"dmx": 100, "dmxc": 100, "dmi": 100, "nupa": 400, "bbh": 250, "eleu": 50}+CONDS = ["base", "base_nudge", "merged_nb", "merged_inline_fp", "merged_inline_nudge_fp",+ "merged_inline_fp_tc", "merged_inline_nudge_fp_tc",+ "merged_inline", "merged_inline_nudge", "pytool"]+++def family(data: str) -> str:+ return data.split("_")[0]+++# ---------------------------------------------------------------------------------- datasets++def _neg_literal(q: str) -> bool:+ """A negative number literal: a '-' right before a digit whose previous non-space character+ is not a digit or ')' (DM / BBH write binary minus as 'a - b')."""+ for m in re.finditer(r"-(?=\d)", q):+ before = q[:m.start()].rstrip()+ if not before or not (before[-1].isdigit() or before[-1] == ")"):+ return True+ return False+++def is_clean(q: str, gold: str) -> bool:+ g = Fraction(gold) if "/" in gold else Fraction(Decimal(gold))+ if g < 0 or g.denominator != 1:+ return False+ return not (re.search(r"\d\.\d", q) or "/" in q or _neg_literal(q))+++def load_dm(data: str, n: int, seed: int) -> list[dict]:+ """dmx_/dmi_: a seeded sample of the module's test file; dmxc_: of its clean problems only+ (the add / sub / mul the computer covers, without decimals or negatives)."""+ clean_only = data.startswith("dmxc_")+ mod = DM_MODULES["dmx_" + data[5:] if clean_only else data]+ f = DATA / "dm" / "mathematics_dataset-v1.0" / (mod + ".txt")+ lines = f.read_text(encoding="utf-8").splitlines()+ pairs = [(lines[i].strip(), lines[i + 1].strip()) for i in range(0, len(lines) - 1, 2)]+ pool = [i for i, (q, a) in enumerate(pairs) if is_clean(q, a)] if clean_only else range(len(pairs))+ idx = sorted(random.Random(seed).sample(pool, min(n, len(pool))))+ return [{"id": f"{data}-{i}", "question": pairs[i][0], "gold": pairs[i][1],+ "group": data.split("_", 1)[1], "clean": is_clean(pairs[i][0], pairs[i][1])}+ for i in idx]+++_NUPA = None+++def load_nupa(data: str, n: int, seed: int) -> list[dict]:+ """n problems, spread evenly over the digit lengths 3-20 (n // 18 per length, at least 1)."""+ global _NUPA+ cache = DATA / "nupa" / "int_tasks.json"+ if _NUPA is None:+ if not cache.exists(): # keep only the integer tasks (test.json is 690 MB)+ full = json.loads((DATA / "nupa" / "test.json").read_text(encoding="utf-8"))+ keep = {k: full[k] for k, _, _ in NUPA_TASKS.values()}+ cache.write_text(json.dumps(keep), encoding="utf-8")+ _NUPA = json.loads(cache.read_text(encoding="utf-8"))+ key, op, sym = NUPA_TASKS[data]+ lens = list(range(3, 21))+ per = max(1, n // len(lens))+ out = []+ for L in lens:+ items = _NUPA[key][str(L)]+ order = list(range(len(items))) # per-length shuffle: a larger n extends a smaller one+ random.Random(f"{seed}-{data}-{L}").shuffle(order)+ for j in sorted(order[:per]):+ s = items[j]+ task = s.split(" . ", 1)[1] # drop "Directly return the answer ... like 123 ."+ m = re.fullmatch(r"(.*?:\s*)(\d+) (\S) (\d+) = (\d+)", task)+ assert m and m[3] == sym, s+ g = next(f"{lo}-{hi}" for lo, hi in NUPA_GROUPS if lo <= L <= hi)+ out.append({"id": f"{data}-L{L}-{j}", "question": f"{m[1]}{m[2]} {m[3]} {m[4]}",+ "gold": m[5], "group": f"len {g}", "length": L, "clean": True,+ "calls": [[op, m[2], m[4]]]})+ return out+++def load_bbh(data: str, n: int, seed: int) -> list[dict]:+ import pyarrow.parquet as pq++ t = pq.read_table(DATA / "bbh" / "mat2.parquet").to_pylist()+ idx = sorted(random.Random(seed).sample(range(len(t)), min(n, len(t))))+ out = []+ for i in idx:+ q = re.sub(r"\s*=\s*$", "", t[i]["input"].strip())+ out.append({"id": f"{data}-{i}", "question": q, "gold": t[i]["target"].strip(),+ "group": "multistep_arithmetic_two", "clean": is_clean(q, t[i]["target"])})+ return out+++def load_eleu(data: str, n: int, seed: int) -> list[dict]:+ sub = data[5:]+ rows = [json.loads(x) for x in (DATA / "eleuther" / f"{sub}.jsonl").read_text(+ encoding="utf-8").splitlines() if x.strip()]+ idx = sorted(random.Random(seed).sample(range(len(rows)), min(n, len(rows))))+ out = []+ for i in idx:+ q = re.sub(r"^\s*Q:\s*|\s*A:\s*$", "", rows[i]["context"]).strip()+ gold = rows[i]["completion"].strip()+ p = {"id": f"{data}-{i}", "question": q, "gold": gold, "group": sub,+ "clean": is_clean(q, gold) and "three_ops" not in sub}+ for word, (op, w) in ELEU_OPS.items():+ m = re.fullmatch(rf"What is (\d+) {w} (\d+)\?", q)+ if word in sub and m:+ p["calls"] = [[op, m[1], m[2]]]+ out.append(p)+ return out+++def load(data: str, n: int | None = None, seed: int = 0) -> list[dict]:+ n = n or DEFAULT_N[family(data)]+ fam = family(data)+ if fam in ("dmx", "dmi", "dmxc"):+ return load_dm(data, n, seed)+ if fam == "nupa":+ return load_nupa(data, n, seed)+ if fam == "bbh":+ return load_bbh(data, n, seed)+ if fam == "eleu":+ return load_eleu(data, n, seed)+ raise ValueError(data)+++# ---------------------------------------------------------------------------------- judging++def gold_value(gold: str) -> Fraction:+ gold = gold.replace(",", "").strip()+ return Fraction(gold) if "/" in gold else Fraction(Decimal(gold))+++def _norm(s: str) -> str:+ s = re.sub(r"\\(?:text|mathrm|textbf|mbox|mathbf)\{([^{}]*)\}", r" \1 ", s)+ s = s.replace("\u2212", "-").replace("\\left", "").replace("\\right", "").replace("{,}", ",")+ for x in ("\\$", "$", "\\,", "\\!", "\\;", "\\ ", "~", "**"):+ s = s.replace(x, " ")+ s = re.sub(r"(?<=\d),(?=\d{3}(?!\d))", "", s) # thousands separators+ s = re.sub(r"(?<=\d)\s+(?=\d{3}(?!\d))", "", s) # 1 234 567+ s = re.sub(r"\\d?frac\s*\{([^{}]*)\}\s*\{([^{}]*)\}", r"(\1)/(\2)", s)+ return s.strip()+++NUMR = r"-?\s*\d+(?:\.\d+)?"+++def parse_value(s: str):+ """A boxed answer as an exact rational (decimals and a/b fractions), else None."""+ s = _norm(s)+ if "=" in s:+ s = s.rsplit("=", 1)[1]+ s = s.strip().rstrip(".").strip()+ m = re.fullmatch(rf"(-)?\s*\(?\s*({NUMR})\s*\)?\s*/\s*\(?\s*({NUMR})\s*\)?", s)+ if m:+ a, b = Fraction(Decimal(m[2].replace(" ", ""))), Fraction(Decimal(m[3].replace(" ", "")))+ if b == 0:+ return None+ return (-1 if m[1] else 1) * a / b+ m = re.fullmatch(rf"\(?\s*({NUMR})\s*\)?", s)+ if m:+ return Fraction(Decimal(m[1].replace(" ", "")))+ nums = re.findall(NUMR, s)+ if len(nums) == 1:+ return Fraction(Decimal(nums[0].replace(" ", "")))+ return None+++def strict_pred(text: str):+ """The answer as evals.judge reads it (user-turn cut, last \\boxed{} after the thinking and+ outside tool calls, else the last number), parsed exactly (no rounding)."""+ m = E.USER_TURN.search(text)+ if m:+ text = text[:m.start()]+ t = re.sub(r"<think>.*?</think>", " ", text, flags=re.DOTALL)+ t = re.sub(r"<tool_call>.*?(</tool_call>|$)", " ", t, flags=re.DOTALL)+ t = re.sub(r"<tool_response>.*?(</tool_response>|$)", " ", t, flags=re.DOTALL)+ t = re.sub(r"<\|[^|]*\|>(assistant|user)?", " ", E.MARK.sub(" ", t))+ boxes = [E._balanced(t, b.end()) for b in re.finditer(r"\\boxed\s*\{", t)]+ if boxes:+ try:+ return parse_value(boxes[-1]), "boxed"+ except (InvalidOperation, ValueError, ZeroDivisionError):+ return None, "boxed_unparsed"+ nums = re.findall(r"-?\d+(?:\.\d+)?", _norm(t))+ if nums:+ return Fraction(Decimal(nums[-1])), "last_number"+ return None, "none"+++def exact_fields(p: dict, text: str) -> dict:+ v, src = strict_pred(text)+ g = gold_value(p["gold"])+ return {"pred_exact": (str(v.numerator) if v.denominator == 1 else+ f"{v.numerator}/{v.denominator}") if v is not None else None,+ "src_exact": src, "exact": v is not None and v == g}+++# ---------------------------------------------------------------------------------- running++def _judgeable(p: dict) -> dict:+ """evals.judge reads the gold with Decimal(): give it a fraction gold as a 30-digit decimal+ (it only feeds the tolerant ``correct``; ``exact`` uses the true gold)."""+ if "/" not in p["gold"]:+ return p+ import decimal++ g = gold_value(p["gold"])+ with decimal.localcontext() as c:+ c.prec = 40+ dec = Decimal(g.numerator) / Decimal(g.denominator)+ return p | {"gold": str(dec)}+++def rescore_errors(args) -> None:+ """Re-judge records whose scoring failed (score_error) from their stored ids / text."""+ out = Path(args.out)+ for f in sorted(out.glob("*.jsonl")):+ parts = f.name.split(".")+ if len(parts) != 3:+ continue+ recs = [json.loads(x) for x in f.read_text(encoding="utf-8").splitlines() if x.strip()]+ bad = [r for r in recs if r.get("score_error")]+ if not bad:+ continue+ probs = {p["id"]: p for p in load(parts[0], 100_000)}+ for r in bad:+ p = probs[r["id"]]+ j = E.judge(_judgeable(p), r["text"], False)+ r.update(j)+ r.update(exact_fields(p, r["text"]))+ r["source"] = j["source"]+ r["gold"] = p["gold"]+ r.pop("score_error")+ r["rescored"] = True+ f.write_text("".join(json.dumps(r, ensure_ascii=False) + "\n" for r in recs),+ encoding="utf-8")+ print(f"{f.name}: rescored {len(bad)}", flush=True)+++def snapshot() -> str:+ snap = HOME / "snap.txt"+ if snap.exists():+ return snap.read_text().strip().splitlines()[-1].strip()+ return E.default_model("base")+++def setup(cond: str, mpath: str, batch: int, max_len: int, fast: bool, compile_: bool,+ view: int = 0):+ """evals.run's model setup: tokenizer, model, BIOS, Ctx, hook, ban, BatchDecoder."""+ import torch+ from transformers import AutoModelForImageTextToText, AutoTokenizer++ kind, flags = E.parse_cond(cond)+ tok = AutoTokenizer.from_pretrained(mpath)+ model = AutoModelForImageTextToText.from_pretrained(mpath, dtype=torch.bfloat16)+ model = model.cuda().eval()+ chip, bios, bios_id = None, [], None+ meta_path = Path(mpath) / "chipformer.json"+ if meta_path.exists():+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ meta = json.loads(meta_path.read_text())+ chip = (meta["chip_token_offset"], meta["chip_vocab"])+ if kind in E.BIOS_KINDS:+ import hashlib++ bios = [t + chip[0] for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ bios_id = f"{hashlib.sha1(bcd.ROM).hexdigest()[:10]} ({len(bcd.ROM)} B, OPS {bcd.OPS})"+ print(f"BIOS {bios_id}, boot record {len(bios)} tokens", flush=True)+ ctx = E.Ctx(tok, chip)+ hook = {"pytool_inline": E.executor("inline", ctx), "pytool": E.executor("template", ctx)}.get(kind)+ banned = []+ if "nb" in flags or "ne" in flags or kind == "merged_inline":+ gc = Path(mpath) / "generation_config.json"+ bad = json.loads(gc.read_text()).get("bad_words_ids", []) if gc.exists() else []+ # ne: <|endoftext|> too (Qwen otherwise ends a tool-mode turn right after the response)+ banned = sorted({w[0] for w in bad if len(w) == 1} | {E.IM_START}+ | ({getattr(E, "ENDOFTEXT", 248044)} if "ne" in flags else set()))+ model.lm_head = E.BannedHead(model.lm_head, banned, model.lm_head.weight.shape[0])+ print(f"banned token ids {banned}", flush=True)+ import torch._inductor.config++ from chipformer.q35.batch import BatchDecoder++ torch._inductor.config.shape_padding = False+ if fast and chip:+ from chipformer.q35 import fast as F++ print("compact", F.compact(model, qwen=False), flush=True)+ bd = BatchDecoder(model, batch=batch, max_len=max_len, eos=sorted(ctx.stop), sizes=[batch],+ compile=compile_, view=view,+ **({"skip_qwen": True} if fast and chip else {}),+ **({"freeze_chip_positions": True} if "fp" in flags else {}),+ **({"transparent_chip": True} if "tc" in flags else {}))+ return tok, model, ctx, hook, bios, bios_id, banned, bd+++def decode_views(bd, ctx, prompts: list[list[int]], budget: int, hook, on_done) -> None:+ """evals.decode_all (same rows, stop rules, forcing, chip steps), except that each step's+ attention reads only the smallest captured KV view covering every live row (BatchDecoder+ ``view``), instead of the whole static cache. Without views it is decode_all."""+ from collections import deque++ import torch++ if not getattr(bd, "views", None) or len(bd.views) < 2:+ return E.decode_all(bd, ctx, prompts, budget, hook, on_done)+ queue = deque(range(len(prompts)))+ n = bd.batch+ bd.prefill(prompts[0])+ level = bd._prepare(n)+ rows, tok, pos = level[:3]+ has_chip = level[4] is not None+ graphs = bd.view_graphs[n]+ # fp: RoPE positions frozen on chip tokens (a separate per-row buffer in BatchDecoder)+ rpos = bd.rpos[n] if getattr(bd, "freeze", False) else None+ first = bd.prefill(prompts[queue[0]])+ last = [0] * n+ live: list = [None] * n+ length = [0] * n # tokens in each row's cache after its next input++ def admit(b, first=None):+ while queue:+ i = queue.popleft()+ t = bd.prefill(prompts[i]) if first is None else first+ first = None+ r = E._Row(i, len(prompts[i]))+ if r.push(t, False, ctx, budget, bd.max_len, hook):+ on_done(i, r.result())+ continue+ rows.load_row(b, bd.small, len(prompts[i]))+ if hasattr(bd, "mark_prompt"): # _tc: its chip slots (no-op otherwise)+ bd.mark_prompt(n, b, prompts[i])+ tok[b, 0] = t+ pos[b] = len(prompts[i])+ if rpos is not None:+ rpos[b] = sum(not ctx.is_chip(x) for x in prompts[i])+ live[b], last[b], length[b] = r, t, len(prompts[i]) + 1+ return+ live[b] = None+ tok[b, 0] = 0+ pos[b] = 0+ length[b] = 0++ admit(0, first)+ for b in range(1, n):+ admit(b)+ with torch.no_grad():+ while any(r is not None for r in live):+ busy = [b for b in range(n) if live[b] is not None]+ use_chip = has_chip and all(ctx.is_chip(last[b]) for b in busy)+ need = max(length[b] for b in busy)+ v = next(x for x in bd.views if x >= need)+ step_fn, chip_fn = graphs[v]+ pred = (chip_fn if use_chip else step_fn)(tok, pos).tolist()+ bd.stats["view_sum"] += v+ pos.add_(1)+ if rpos is not None: # the input was text: the next RoPE position moves on+ rpos.add_(bd._is_text(tok[:, 0]))+ nt, done = [], []+ for b, p in enumerate(pred):+ r = live[b]+ if r is None:+ nt.append(0)+ continue+ forced = bool(r.forced)+ t = r.forced.popleft() if forced else p+ nt.append(t)+ length[b] += 1+ if r.push(t, forced, ctx, budget, bd.max_len, hook):+ done.append(b)+ tok.copy_(torch.tensor(nt, device=tok.device)[:, None])+ last = nt+ idle = [b for b in range(n) if live[b] is None]+ if idle:+ pos[idle] = 0+ if rpos is not None:+ rpos[idle] = 0+ for b in done:+ on_done(live[b].i, live[b].result())+ admit(b)+++def default_model(cond: str) -> str:+ return str(ROOT / "dist" / "q4-v10") if cond.startswith("merged") else snapshot()+++def run(args) -> None:+ import torch++ cond = args.cond + (f"-{args.tag}" if args.tag else "")+ kind, _ = E.parse_cond(cond)+ out = Path(args.out)+ out.mkdir(parents=True, exist_ok=True)+ todo = []+ for data in args.data:+ path = out / f"{data}.{cond}.jsonl"+ done = set()+ if path.exists():+ recs = [json.loads(x) for x in path.read_text(encoding="utf-8").splitlines()+ if x.strip()]+ # --redo-context: rerun problems that hit the cache length (the newer record is+ # appended; readers keep the last record per id)+ done = {r["id"] for r in recs if not (args.redo_context and r["stop"] == "context")}+ probs = load(data, args.n, args.seed)+ if args.per_data: # the first K of the (sorted, seeded) sample: a subset of the full run+ probs = probs[:args.per_data]+ probs = [p for p in probs if p["id"] not in done]+ print(f"{path.name}: {len(done)} done, {len(probs)} to go", flush=True)+ todo += [(data, p) for p in probs]+ if args.limit:+ todo = todo[:args.limit]+ if not todo:+ return+ mpath = args.model or default_model(cond)+ budget = args.budget+ batch = min(args.batch, len(todo))+ # the cache length is fixed per condition (one compile); prompts here are short+ max_len = args.max_len or (budget + 1024 + (4096 if cond.startswith("merged") else 512))+ max_len = -(-max_len // 1024) * 1024+ tok, _model, ctx, hook, bios, bios_id, banned, bd = setup(+ cond, mpath, batch, max_len, args.fast, not args.no_compile, args.view)+ prompts = [E.build_prompt(tok, p["question"], cond, bios) for _, p in todo]+ assert max(map(len, prompts)) + budget + 512 < max_len, "prompt too long for max_len"+ if kind == "merged" and not args.no_selftest:+ E.selftest(bd, ctx, tok, bios)+ print(f"batch {bd.batch}, max_len {max_len}, budget {budget}, {len(todo)} problems",+ flush=True)+ t0 = time.time()+ counts = {}++ def done(i, r):+ data, p = todo[i]+ try:+ rec = E.score(_judgeable(p), cond, ctx, r, r["seconds"], False)+ rec |= exact_fields(p, rec["text"])+ rec["gold"] = p["gold"]+ except (ArithmeticError, ValueError, TypeError, KeyError, IndexError) as e:+ # never lose the run to one transcript's scoring+ print(f"scoring error {p['id']}: {e!r}", flush=True)+ rec = {"id": p["id"], "cond": cond, "question": p["question"], "gold": p["gold"],+ "pred": None, "source": "score_error", "inline": [], "correct": False,+ "calls": [], "stop": r["stop"], "n_qwen": r["n_qwen"],+ "n_chip": sum(1 for t in r["out"] if ctx.is_chip(t)), "n_out": len(r["out"]),+ "seconds": round(r["seconds"], 2), "text": ctx.show(r["out"]),+ "pred_exact": None, "src_exact": "score_error", "exact": False,+ "score_error": repr(e)}+ rec |= {"data": data, "group": p["group"], "clean": p["clean"],+ "n_prompt": len(prompts[i]), "batch": bd.batch, "model": mpath,+ "bios": bios_id, "banned": banned, "ids": r["out"], "inj_at": r.get("inj_at")}+ if "length" in p:+ rec["length"] = p["length"]+ with (out / f"{data}.{cond}.jsonl").open("a", encoding="utf-8") as f:+ f.write(json.dumps(rec, ensure_ascii=False) + "\n")+ counts[data] = counts.get(data, 0) + 1+ print(json.dumps({k: rec[k] for k in ("id", "gold", "pred_exact", "exact", "stop",+ "n_qwen", "n_chip", "seconds")}+ | {"calls": len(rec["calls"]), "fires": len(rec["inline"])}), flush=True)++ decode_views(bd, ctx, prompts, budget, hook, done)+ wall = time.time() - t0+ with (out / "runs.log").open("a", encoding="utf-8") as f:+ f.write(json.dumps({"cond": cond, "data": counts, "n": len(todo), "batch": bd.batch,+ "wall_s": round(wall, 1), "s_per_problem": round(wall / len(todo), 2),+ "view": args.view, "stats": bd.stats, "model": mpath, "bios": bios_id, "budget": budget,+ "gpu": torch.cuda.get_device_name()}) + "\n")+ print(f"done {len(todo)} in {wall:.0f}s ({wall / len(todo):.1f} s/problem)", flush=True)+++# ---------------------------------------------------------------------------------- checks++def identity(args) -> None:+ """BatchDecoder (the config the runs use) vs HF generate, greedy, on a few problems."""+ import torch++ cond = args.cond+ probs = load("dmx_mul_big", 100)[:2] + load("nupa_mulh", 400)[100:102] + load("bbh_mat2")[:1]+ mpath = args.model or default_model(cond)+ max_len = 1024 + 4096 + args.max_new+ max_len = -(-max_len // 1024) * 1024+ tok, model, ctx, _hook, bios, _, _banned, bd = setup(cond, mpath, args.batch, max_len,+ True, True)+ prompts = [E.build_prompt(tok, p["question"], cond, bios) for p in probs]+ got = {}+ E.decode_all(bd, ctx, prompts, args.max_new, None, lambda i, r: got.__setitem__(i, r["out"]))+ for i, p in enumerate(prompts):+ x = torch.tensor([p], device="cuda")+ with torch.no_grad():+ g = model.generate(x, max_new_tokens=len(got[i]), do_sample=False,+ cache_implementation="static")[0, len(p):].tolist()+ a = got[i]+ k = next((j for j in range(min(len(a), len(g))) if a[j] != g[j]), min(len(a), len(g)))+ n_chip = sum(ctx.is_chip(t) for t in a)+ print(json.dumps({"id": probs[i]["id"], "batch_len": len(a), "gen_len": len(g),+ "identical": a == g[:len(a)], "first_diff": k, "chip_tokens": n_chip,+ "diff_ctx": ctx.show(a[max(0, k - 20):k + 5]) if a != g[:len(a)] else ""}),+ flush=True)+++def tfcheck(args) -> None:+ """Teacher-forced check of stored records against HF's plain forward (eager, batch 1): at+ every generated position, is the stored token the HF model's argmax? Where it is not, the+ top-2 logit gap shows whether it is a bf16 near-tie (kernel rounding) or a real+ disagreement. Positions with a forced token (pytool) are skipped."""+ import torch+ from transformers import AutoModelForImageTextToText, AutoTokenizer++ cond = args.cond+ mpath = args.model or default_model(cond)+ tok = AutoTokenizer.from_pretrained(mpath)+ model = AutoModelForImageTextToText.from_pretrained(mpath, dtype=torch.bfloat16).cuda().eval()+ from chipformer.q35.fast import exact_matmuls++ exact_matmuls()+ kind, flags = E.parse_cond(cond)+ chip, bios = None, []+ meta_path = Path(mpath) / "chipformer.json"+ if meta_path.exists():+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ meta = json.loads(meta_path.read_text())+ chip = (meta["chip_token_offset"], meta["chip_vocab"])+ if kind in E.BIOS_KINDS:+ bios = [t + chip[0] for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ banned = []+ if "nb" in flags or "ne" in flags or kind == "merged_inline":+ gc = Path(mpath) / "generation_config.json"+ bad = json.loads(gc.read_text()).get("bad_words_ids", []) if gc.exists() else []+ # ne: <|endoftext|> too (Qwen otherwise ends a tool-mode turn right after the response)+ banned = sorted({w[0] for w in bad if len(w) == 1} | {E.IM_START}+ | ({getattr(E, "ENDOFTEXT", 248044)} if "ne" in flags else set()))+ out = Path(args.out)+ for data in args.data:+ src = out / f"{data}.{cond}.jsonl"+ recs = [json.loads(x) for x in src.read_text(encoding="utf-8").splitlines() if x.strip()]+ for r in recs[:args.k]:+ prompt = E.build_prompt(tok, r["question"], cond, bios)+ gen = r["ids"] if r.get("ids") is not None else tok(+ r["text"], add_special_tokens=False).input_ids+ ids = prompt + gen+ with torch.no_grad():+ lg = model(torch.tensor([ids], device="cuda")).logits[0].float()+ if banned:+ lg[:, banned] = float("-inf")+ lg = lg[len(prompt) - 1:len(ids) - 1]+ top2 = lg.topk(2, dim=-1)+ want = torch.tensor(gen, device="cuda")+ arg = top2.indices[:, 0]+ miss = (arg != want).nonzero().flatten().tolist()+ gaps = [round(float(lg[p, arg[p]] - lg[p, want[p]]), 3) for p in miss]+ chipmiss = [p for p in miss if chip and chip[0] <= gen[p] < chip[0] + chip[1]]+ print(json.dumps({"id": r["id"], "cond": cond, "n_gen": len(gen),+ "argmax_agree": len(gen) - len(miss), "disagree": len(miss),+ "first": miss[:1], "gap_at_disagree": gaps[:5],+ "max_gap": max(gaps) if gaps else 0.0,+ "chip_token_disagree": len(chipmiss)}), flush=True)+++def replay(args) -> None:+ """Replay every merged record through the circuit IR: at every position the circuit owns,+ the model's token must be the IR's prediction."""+ import numpy as np+ from transformers import AutoTokenizer++ from chipformer.q35 import bridge as B+ from chipformer.q35 import check as Q+ from chipformer.q35 import chip8_q35, ir+ from chipformer.q35.lib import bcd+ from chipformer.q35.spikes_replay import class_map+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ mpath = args.model or default_model("merged")+ meta = json.loads((Path(mpath) / "chipformer.json").read_text())+ off, nchip = meta["chip_token_offset"], meta["chip_vocab"]+ tok = AutoTokenizer.from_pretrained(mpath)+ cmap = class_map(tok, meta)+ circuit = chip8_q35.build()+ bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ out = Path(args.out)+ for data in args.data:+ src = out / f"{data}.{args.cond}.jsonl"+ if not src.exists():+ continue+ dst = out / f"{data}.{args.cond}.replay.jsonl"+ have = set()+ if dst.exists():+ have = {json.loads(x)["id"] for x in dst.read_text(encoding="utf-8").splitlines() if x}+ recs = []+ for x in src.read_text(encoding="utf-8").splitlines():+ try: # the run may be appending the last line right now+ recs.append(json.loads(x))+ except json.JSONDecodeError:+ pass+ if args.shard:+ k, m = map(int, args.shard.split("/"))+ recs = recs[k::m]+ for r in recs:+ if r["id"] in have or r.get("ids") is None:+ continue+ gen = r["ids"]+ if not any(off <= t < off + nchip for t in gen):+ res = {"id": r["id"], "chip_tokens": 0, "owned": 0, "mismatch": 0, "skipped": True}+ else:+ t0 = time.time()+ prompt = E.build_prompt(tok, r["question"], r["cond"], bios)+ ids = prompt + gen+ stream = [t - off if off <= t < off + nchip else cmap.get(t, B.OTHER)+ for t in ids]+ pred, m = Q.predictions(circuit, stream)+ owner = ir.ev(circuit.owner, m)+ rng = range(len(prompt) - 1, len(ids) - 1)+ owned = [p for p in rng if owner[p] == 1]+ bad = [p for p in owned if pred[p] != stream[p + 1]]+ # chip tokens Qwen wrote where the circuit does not own the position+ stray = [p for p in rng if owner[p] != 1 and off <= ids[p + 1] < off + nchip]+ res = {"id": r["id"], "chip_tokens": sum(off <= t < off + nchip for t in gen),+ "owned": len(owned), "mismatch": len(bad), "stray_chip": len(stray),+ "first": [(p - len(prompt), int(stream[p + 1]), int(pred[p]))+ for p in bad[:3]],+ "seconds": round(time.time() - t0, 1)}+ del m+ with dst.open("a", encoding="utf-8") as f:+ f.write(json.dumps(res) + "\n")+ print(data, json.dumps(res), flush=True)+ np.zeros(1)+++# ---------------------------------------------------------------------------------- summary++def wilson(k, n):+ return E.wilson(k, n)+++def paired(ra: dict, rb: dict, key="exact"):+ ids = sorted(set(ra) & set(rb))+ if not ids:+ return None+ xa = [ra[i][key] for i in ids]+ xb = [rb[i][key] for i in ids]+ n = len(ids)+ bo = sum(x and not y for x, y in zip(xa, xb))+ co = sum(y and not x for x, y in zip(xa, xb))+ d = (bo - co) / n+ se = math.sqrt(max(0.0, (bo + co) / n - d * d) / n)+ return n, d, d - 1.96 * se, d + 1.96 * se, bo, co, E.mcnemar(bo, co)+++def machine_stats(rs: list[dict]):+ """Every result the machine printed: tool responses of valid calls and parsed inline fires.+ exact = printed == A op B."""+ tool = [c["exact"] for r in rs for c in r["calls"] if c["valid"] and c["response"] is not None]+ inl = [f["exact"] for r in rs for f in r.get("inline", []) if "exact" in f]+ unparsed = sum("exact" not in f for r in rs for f in r.get("inline", []))+ return tool, inl, unparsed+++def load_all(out: Path):+ res, rep = {}, {}+ for f in sorted(out.glob("*.jsonl")):+ parts = f.name.split(".")+ if len(parts) == 3:+ recs = [json.loads(x) for x in f.read_text(encoding="utf-8").splitlines() if x.strip()]+ res[(parts[0], parts[1])] = {r["id"]: r for r in recs}+ elif len(parts) == 4 and parts[2] == "replay":+ recs = [json.loads(x) for x in f.read_text(encoding="utf-8").splitlines() if x.strip()]+ rep[(parts[0], parts[1])] = {r["id"]: r for r in recs}+ return res, rep+++def _pct(k, n):+ return f"{100 * k / n:.1f}%" if n else "-"+++def _table(res, rep, keys: list[tuple], conds: list[str], title: str, wall: dict) -> list[str]:+ """One table over the pooled records of ``keys`` (datasets), per condition."""+ L = [f"\n### {title}\n",+ ("| cond | N | exact acc | 95% CI | clean acc (n) | vs base (pts, 95% CI, McNemar p) | "+ "vs base_nudge | tool-call rate | inline-fire rate | answer = a printed result | "+ "machine exact (tool / inline) | IR replay mismatches | cutoffs | loops | Qwen tok | "+ "chip tok | s/prob |"),+ "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|"]+ pooled = {}+ for c in conds:+ d = {}+ for k in keys:+ d |= res.get((k, c), {})+ pooled[c] = d+ for c in conds:+ rs = list(pooled[c].values())+ if not rs:+ continue+ n, k = len(rs), sum(r["exact"] for r in rs)+ lo, hi = wilson(k, n)+ cl = [r["exact"] for r in rs if r["clean"]]+ cmp = []+ for b in ("base", "base_nudge"):+ pr = paired(pooled[c], pooled.get(b, {})) if c != b else None+ cmp.append(f"{100 * pr[1]:+.1f} ({100 * pr[2]:+.1f}..{100 * pr[3]:+.1f}), "+ f"p={pr[6]:.2g} [{pr[4]}/{pr[5]}]" if pr else "-")+ tool, inl, unp = machine_stats(rs)+ rp = [rep.get((key, c), {}) for key in keys]+ rpr = [x for dct in rp for i, x in dct.items() if i in pooled[c]]+ rp_s = "-"+ if rpr:+ checked = [x for x in rpr if not x.get("skipped")]+ rp_s = (f"{sum(x['mismatch'] for x in checked)} in "+ f"{sum(x['owned'] for x in checked)} owned pos. ({len(checked)} recs"+ f"{'' if len(rpr) == n else f', {len(rpr)}/{n} done'})")+ ws = [wall[(key, c)] for key in keys if (key, c) in wall]+ L.append(+ f"| {c} | {n} | {_pct(k, n)} | {100 * lo:.0f}-{100 * hi:.0f} | "+ f"{_pct(sum(cl), len(cl))} ({len(cl)}) | {cmp[0]} | {cmp[1]} | "+ f"{_pct(sum(bool(r['calls']) for r in rs), n)} | "+ f"{_pct(sum(bool(r.get('inline')) for r in rs), n)} | "+ f"{_pct(sum(_from_machine(r) for r in rs), n)} | "+ f"{sum(tool)}/{len(tool)} / {sum(inl)}/{len(inl)}{f' (+{unp} unparsed)' if unp else ''} | "+ f"{rp_s} | {sum(r['stop'] != 'eos' for r in rs)} | "+ f"{sum(E._looping(r.get('inline', [])) for r in rs)} | "+ f"{sum(r['n_qwen'] for r in rs) / n:.0f} | {sum(r['n_chip'] for r in rs) / n:.0f} | "+ f"{f'{sum(ws) / len(ws):.1f}' if ws else '-'} |")+ return L+++def _cause(r: dict) -> str:+ """Why a wrong answer is wrong (the first that applies)."""+ tool, inl, _ = machine_stats([r])+ if not all(tool) or not all(inl):+ return "wrong print"+ loop = E._looping(r.get("inline", []))+ if r["stop"] != "eos":+ return "cutoff (looping)" if loop else "cutoff"+ if r.get("pred_exact") is None:+ return "no answer"+ if loop:+ return "looping, then wrong"+ g = str(gold_value(r["gold"]))+ printed = {f.get("true") for f in r.get("inline", []) if f.get("exact")} | {+ (c.get("response") or "").strip() for c in r["calls"] if c.get("exact")}+ if g in printed:+ return "gold was printed, answer differs"+ return "Qwen's own error"+++CAUSES = ["wrong print", "cutoff (looping)", "cutoff", "looping, then wrong", "no answer",+ "gold was printed, answer differs", "Qwen's own error"]+++def _fail_table(res, keys, conds) -> list[str]:+ L = ["\nWrong answers by cause (pooled; loops = 3+ identical inline fires in a row):\n",+ "| cond | wrong | " + " | ".join(CAUSES) + " |", "|---|---|" + "---|" * len(CAUSES)]+ for c in conds:+ rs = [r for k in keys for r in res.get((k, c), {}).values()]+ if not rs:+ continue+ bad = [_cause(r) for r in rs if not r["exact"]]+ L.append(f"| {c} | {len(bad)}/{len(rs)} | " + " | ".join(+ str(bad.count(x)) for x in CAUSES) + " |")+ return L+++def _from_machine(r: dict) -> bool:+ """The final answer is a number the machine (or the Python tool) printed."""+ p = r.get("pred_exact")+ if p is None:+ return False+ return (any(f.get("is_answer") for f in r.get("inline", []))+ or any((c.get("response") or "").strip() == p for c in r["calls"]))+++def _wall(out: Path) -> dict:+ """(data, cond) -> wall seconds per problem of the run(s) that made it (a run that covers+ several datasets shares its wall time by problem count)."""+ tot = {}+ log = out / "runs.log"+ if log.exists():+ for line in log.read_text(encoding="utf-8").splitlines():+ x = json.loads(line)+ for d, n in x["data"].items():+ w, m = tot.get((d, x["cond"]), (0.0, 0))+ tot[(d, x["cond"])] = (w + x["s_per_problem"] * n, m + n)+ return {k: w / n for k, (w, n) in tot.items() if n}+++def _groups(res, keys, conds):+ """Accuracy per group (DM module / NUPA length bucket / Eleuther subset)."""+ groups = {}+ for k in keys:+ for c in conds:+ for r in res.get((k, c), {}).values():+ groups.setdefault(f"{k}:{r['group']}", set())+ L = ["\n| group | " + " | ".join(conds) + " |", "|---|" + "---|" * len(conds)]+ for g in sorted(groups, key=lambda s: (s.split(":")[0], [int(x) if x.isdigit() else x+ for x in re.split(r"(\d+)", s)])):+ k, name = g.split(":", 1)+ cells = []+ for c in conds:+ rs = [r for r in res.get((k, c), {}).values() if r["group"] == name]+ cells.append(f"{sum(r['exact'] for r in rs)}/{len(rs)}" if rs else "-")+ L.append(f"| {g} | " + " | ".join(cells) + " |")+ return L+++BENCH = [("DeepMind Mathematics, extrapolate (big / longer) test modules",+ [k for k in DM_MODULES if k.startswith("dmx") and k not in DM_DIV]),+ (("DeepMind Mathematics, extrapolate big modules, clean problems only (non-negative "+ "integers, no decimals)"), DMXC),+ ("DeepMind Mathematics, extrapolate division modules (div_big, mul_div_multiple_longer, "+ "clean div_big)", DM_DIV),+ ("DeepMind Mathematics, interpolate test modules", [k for k in DM_MODULES if k.startswith("dmi")]),+ ("NUPA integer add / sub / multiply (lengths 3-20)", list(NUPA_TASKS)),+ ("BBH multistep_arithmetic_two", ["bbh_mat2"]),+ ("EleutherAI/arithmetic", [f"eleu_{s}" for s in ELEU])]+++def summary(args) -> None:+ out = Path(args.out)+ res, rep = load_all(out)+ wall = _wall(out)+ conds = [c for c in CONDS if any(cc == c for _, cc in res)] + sorted(+ {cc for _, cc in res if cc not in CONDS})+ L = ["# Arithmetic benchmarks: merged (computer in the weights) vs base Qwen3.5-4B\n",+ PREAMBLE]+ for title, keys in BENCH:+ if not any((k, c) in res for k in keys for c in conds):+ continue+ L += _bench(res, rep, wall, conds, title, keys)+ text = "\n".join(L)+ print(text)+ (out / "SUMMARY.md").write_text(text + "\n", encoding="utf-8")+++PREAMBLE = (+ "Non-thinking, greedy, 2048 Qwen-written tokens per problem (the machine's tokens are not "+ "counted), evals.py's prompt suffix (final answer in \\boxed{}). Problems are seeded "+ "samples, identical across conditions (paired). exact acc = boxed (else last) number equals "+ "the gold exactly as a rational (no tolerance). clean = all numbers in the problem and the "+ "answer are non-negative integers and there is no division. Paired deltas: A - B in points "+ "with a 95% CI, exact McNemar p, [A-only/B-only wins]. machine exact = printed result == "+ "A op B over every valid tool call / parsed inline fire. IR replay = positions owned by the "+ "circuit where the model's token differs from the circuit IR's prediction (spikes_replay). "+ "cutoffs = budget or context stops; loops = 3+ identical inline fires in a row. s/prob = "+ "wall time per problem at the run's batch size (base / base_nudge / pytool / merged_nb: L4, "+ "batch 16 / 16 / 16 / 8; v11 merged_inline*: L40S, batch 32; v13: L4 / A10G, batch 8). "+ "Models: merged_nb = q4-v12 (commit 62ed528); merged_inline* = q4-v11 (afd8230; v12 only "+ "changes tool-call capture); -v13 = q4-v13 (f270d1b: inline A - B < 0 prints its negative "+ "result, inline add/sub need >= 7 digits). _fp = RoPE positions frozen on chip tokens; "+ "_tc = chip tokens invisible to Qwen's layers; _mix = NUDGE_MIX (products inline, big "+ "add/sub through the tools).\n")+++def _bench(res, rep, wall, conds, title, keys) -> list[str]:+ L = [f"\n## {title}"]+ L += _table(res, rep, keys, conds, "all modules pooled" if len(keys) > 1 else title, wall)+ L += _fail_table(res, keys, conds)+ if len(keys) > 1:+ for k in keys:+ if any((k, c) in res for c in conds):+ L += _table(res, rep, [k], conds, k, wall)+ L += _fail_table(res, [k], conds)+ L.append("\nPer group (exact correct / N):")+ L += _groups(res, keys, conds)+ return L+++def _tags(r: dict) -> set:+ fires = r.get("inline", [])+ t = set()+ if any(f.get("true", "0").startswith("-") for f in fires):+ t.add("negative fire")+ if E._looping(fires):+ t.add("loop")+ if r["stop"] != "eos":+ t.add("cutoff")+ return t+++def losses(args) -> None:+ """Paired wins / losses of A vs B on the given datasets, each problem tagged with what+ happened in A's transcript (negative inline fire, loop, cutoff)."""+ res, _ = load_all(Path(args.out))+ L = [f"\n{args.a} vs {args.b}\n",+ ("| data | N | A | B | A-only | B-only (losses) | losses with negative fire | with loop | "+ "with cutoff | none of these | McNemar p |"),+ "|---|---|---|---|---|---|---|---|---|---|---|"]+ for data in [*args.data, "pooled"]:+ keys = args.data if data == "pooled" else [data]+ ra = {i: r for k in keys for i, r in res.get((k, args.a), {}).items()}+ rb = {i: r for k in keys for i, r in res.get((k, args.b), {}).items()}+ ids = sorted(set(ra) & set(rb))+ if not ids:+ continue+ win = [i for i in ids if ra[i]["exact"] and not rb[i]["exact"]]+ lose = [i for i in ids if rb[i]["exact"] and not ra[i]["exact"]]+ tags = [_tags(ra[i]) for i in lose]+ L.append(f"| {data} | {len(ids)} | {sum(ra[i]['exact'] for i in ids)} | "+ f"{sum(rb[i]['exact'] for i in ids)} | {len(win)} | {len(lose)} | "+ f"{sum('negative fire' in t for t in tags)} | {sum('loop' in t for t in tags)} | "+ f"{sum('cutoff' in t for t in tags)} | {sum(not t for t in tags)} | "+ f"{E.mcnemar(len(win), len(lose)):.2g} |")+ print("\n".join(L))+++def main() -> None:+ ap = argparse.ArgumentParser()+ sub = ap.add_subparsers(dest="cmd", required=True)+ r = sub.add_parser("run")+ r.add_argument("cond", type=E.cond_arg)+ r.add_argument("data", nargs="+", choices=ALL)+ r.add_argument("--n", type=int, help="problems per dataset (default per family)")+ r.add_argument("--seed", type=int, default=0)+ r.add_argument("--model")+ r.add_argument("--out", default=str(OUT))+ r.add_argument("--budget", type=int, default=2048)+ r.add_argument("--max-len", type=int)+ r.add_argument("--batch", type=int, default=16)+ r.add_argument("--limit", type=int, help="only the first N problems still to do")+ r.add_argument("--per-data", type=int, help="only the first K problems of each dataset")+ r.add_argument("--redo-context", action="store_true",+ help="rerun problems whose stored record stopped at the cache length")+ r.add_argument("--no-compile", action="store_true")+ r.add_argument("--no-selftest", action="store_true")+ r.add_argument("--fast", action="store_true", default=True)+ r.add_argument("--tag")+ r.add_argument("--view", type=int, default=1024,+ help="attention KV view granularity (bit-identical; 0 = whole cache)")+ i = sub.add_parser("identity")+ i.add_argument("--cond", default="merged_inline")+ i.add_argument("--model")+ i.add_argument("--batch", type=int, default=8)+ i.add_argument("--max-new", type=int, default=1024)+ t = sub.add_parser("tfcheck")+ t.add_argument("cond")+ t.add_argument("data", nargs="+", choices=ALL)+ t.add_argument("--model")+ t.add_argument("--out", default=str(OUT))+ t.add_argument("--k", type=int, default=5)+ p = sub.add_parser("replay")+ p.add_argument("cond")+ p.add_argument("data", nargs="+", choices=ALL)+ p.add_argument("--model")+ p.add_argument("--out", default=str(OUT))+ p.add_argument("--shard")+ lo = sub.add_parser("losses", help="paired wins / losses of A vs B, losses tagged")+ lo.add_argument("a")+ lo.add_argument("b")+ lo.add_argument("data", nargs="+", choices=ALL)+ lo.add_argument("--out", default=str(OUT))+ e = sub.add_parser("rescore-errors")+ e.add_argument("--out", default=str(OUT))+ s = sub.add_parser("summary")+ s.add_argument("--out", default=str(OUT))+ ld = sub.add_parser("show", help="print a few loaded problems")+ ld.add_argument("data", nargs="+", choices=ALL)+ ld.add_argument("--k", type=int, default=3)+ args = ap.parse_args()+ if args.cmd == "show":+ for d in args.data:+ ps = load(d)+ print(d, len(ps), "clean", sum(p["clean"] for p in ps))+ for p in ps[:args.k]:+ print(" ", json.dumps(p))+ return+ {"run": run, "summary": summary, "replay": replay, "identity": identity,+ "tfcheck": tfcheck, "rescore-errors": rescore_errors,+ "losses": losses}[args.cmd](args)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/batch.py b/experimental/riscv/chipformer/q35/batch.pynew file mode 100644index 0000000..97d1614--- /dev/null+++ b/experimental/riscv/chipformer/q35/batch.py@@ -0,0 +1,839 @@+"""Batched greedy decoding of q35 checkpoints with stock modules, token-identical to decoding+each prompt on its own.++ outs = generate_batch(model, tok, prompts, max_new=4500, batch=16)++How it stays equal to ``model.generate(do_sample=False)`` on one prompt:++- **Prefill alone.** Each prompt is prefilled on its own (batch 1, no padding) through the same+ modules ``generate`` uses, so its states and first token are the batch-1 ones. The states are+ then copied into one row of the batched cache. There is no padding anywhere, so nothing+ depends on ``apply_mask_to_padding_states`` or on DeltaNet chunk alignment, and the circuit+ never sees a pad token.+- **Per-row positions.** Every row keeps its own position: full attention writes key/value ``t``+ of row ``b`` at slot ``t`` (like the batch-1 static cache), RoPE gets the row's own position,+ and the mask of row ``b`` admits slots ``<= pos[b]``. Gated DeltaNet rows are independent.+- **Continuous batching.** A row that emits an end token (or reaches ``max_new``) is refilled+ with the next prompt at once, so the batch stays full while the queue lasts. When the queue+ is empty and few rows are left, the live rows are compacted into a smaller batch.++What can still differ from batch 1 is floating-point rounding in Qwen's own text arithmetic+(GEMM kernels for M = 1 and M = B accumulate in different orders); the circuit's arithmetic is+exact either way (``torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction`` must be+False, see ``fast.exact_matmuls``).++Decoding modes (each step is a CUDA graph per batch size, captured once):++- ``compile=True`` (default): Inductor kernels. Fastest; at batch 1 it matches+ ``generate(cache_implementation="static")`` (which also compiles) only while the graph is the+ same as HF's (no compaction), since any change shifts Inductor's fusion and Qwen's rounding.+- ``compile=False, kv_len="generate"``: eager kernels, attending over exactly generate's static+ cache length (``len(prompt) + max_new``; the attention result depends on it). At batch 1 this+ is bit-identical to ``generate(cache_implementation="static", disable_compile=True)``, also+ with ``fast.compact`` and ``skip_qwen`` (dist/spikes/b22_eager.py).++Options: ``skip_qwen`` (chip-token inputs run only the chip layers; Qwen's layers are exact+no-ops there), ``freeze_chip_positions`` (RoPE counts only non-chip tokens), ``run(hook=...)``+(token forcing / custom stops), generation_config ``bad_words_ids`` are honored. Prompts are+prefilled ahead on a second stream (``prefetch``) while the batch decodes.+"""++import time+from collections import deque++import torch+++def _text_model(model):+ inner = model.model+ return getattr(inner, "language_model", inner)+++def _mark(t):+ torch._dynamo.mark_static_address(t)+ return t+++def _fold_sdpa(module, query, key, value, attention_mask, dropout=0.0, scaling=None, **kw):+ """HF's SDPA attention, except that a single-token query attends without ``repeat_kv``:+ the GQA group goes into the query axis (heads ``j*G .. j*G+G-1`` share KV head ``j``), so+ the cache is not copied G times per layer and step. Bit-identical to the repeat_kv path+ (dist/spikes/b14_fold.py); prompts (more than one query) take HF's path unchanged."""+ from transformers.integrations.sdpa_attention import sdpa_attention_forward++ b, h, q, d = query.shape+ kv = key.shape[1]+ if q != 1 or h == kv or dropout:+ return sdpa_attention_forward(module, query, key, value, attention_mask,+ dropout=dropout, scaling=scaling, **kw)+ out = torch.nn.functional.scaled_dot_product_attention(+ query.view(b, kv, h // kv, d), key, value, attn_mask=attention_mask, scale=scaling)+ return out.reshape(b, h, 1, d).transpose(1, 2).contiguous(), None+++def _use_fold(model) -> None:+ from transformers import AttentionInterface++ AttentionInterface.register("q35_fold", _fold_sdpa)+ cfg = getattr(model.config, "text_config", model.config)+ if cfg._attn_implementation in ("sdpa", "q35_fold"):+ cfg._attn_implementation = "q35_fold"+++class _Rows:+ """Batched cache: linear-attention layers (stock ``LinearAttentionLayer``), and full-attention+ layers whose ``update`` writes each row at its own position."""++ def __init__(self, cfg, small, batch: int, max_len: int, pos: torch.Tensor,+ kvpos: torch.Tensor | None = None):+ from transformers.cache_utils import Cache, LinearAttentionLayer, StaticLayer++ self.view = max_len # attention reads slots [0, view): a compile-time constant+ owner = self+ slot = pos if kvpos is None else kvpos # KV slot per row (transparent: text count)++ class RowKV(StaticLayer):+ def update(self, key_states, value_states, *args, **kwargs):+ # decode only: one new position per row+ rows = torch.arange(slot.shape[0], device=slot.device)+ self.keys[rows, :, slot] = key_states[:, :, 0]+ self.values[rows, :, slot] = value_states[:, :, 0]+ return self.keys[:, :, :owner.view], self.values[:, :, :owner.view]++ layers = []+ for kind, s in zip(cfg.layer_types, small.layers):+ if kind == "linear_attention":+ la = LinearAttentionLayer()+ conv, rec = s.conv_states[0], s.recurrent_states[0]+ la.dtype, la.device = conv.dtype, conv.device+ la.conv_states[0] = _mark(conv.new_zeros(batch, *conv.shape[1:]))+ la.recurrent_states[0] = _mark(rec.new_zeros(batch, *rec.shape[1:]))+ la.conv_kernel_size[0] = conv.shape[-1]+ la.is_conv_states_initialized[0] = True+ la.is_recurrent_states_initialized[0] = True+ la.has_previous_state[0] = True+ layers.append(la)+ else:+ kv = RowKV(max_cache_len=max_len)+ kv.dtype, kv.device = s.keys.dtype, s.keys.device+ kv.batch_size, kv.num_heads = batch, s.keys.shape[1]+ kv.k_head_dim, kv.v_head_dim = s.keys.shape[-1], s.values.shape[-1]+ kv.keys = _mark(s.keys.new_zeros(batch, s.keys.shape[1], max_len, s.keys.shape[-1]))+ kv.values = _mark(s.values.new_zeros(batch, s.values.shape[1], max_len,+ s.values.shape[-1]))+ kv.cumulative_length = kv.cumulative_length.to(pos.device)+ kv.is_initialized = True+ layers.append(kv)+ self.cache = Cache(layers=layers)+ self.kinds = list(cfg.layer_types)++ @torch.no_grad()+ def load_row(self, b: int, small, n: int) -> None:+ """Copy a batch-1 prefill cache holding ``n`` positions into row ``b``."""+ for kind, big, s in zip(self.kinds, self.cache.layers, small.layers):+ if kind == "linear_attention":+ big.conv_states[0][b].copy_(s.conv_states[0][0])+ big.recurrent_states[0][b].copy_(s.recurrent_states[0][0])+ else:+ big.keys[b, :, :n].copy_(s.keys[0, :, :n])+ big.values[b, :, :n].copy_(s.values[0, :, :n])++ @torch.no_grad()+ def take(self, idx: torch.Tensor, into: "_Rows") -> None:+ """Copy rows ``idx`` of this cache into rows 0.. of ``into`` (a smaller batch)."""+ for big, small in zip(self.cache.layers, into.cache.layers):+ if hasattr(big, "keys") and big.keys is not None:+ small.keys[: len(idx)].copy_(big.keys[idx])+ small.values[: len(idx)].copy_(big.values[idx])+ else:+ small.conv_states[0][: len(idx)].copy_(big.conv_states[0][idx])+ small.recurrent_states[0][: len(idx)].copy_(big.recurrent_states[0][idx])+++class _Prefetcher:+ """Prefills the queued prompts ahead of time on a second CUDA stream, from a worker thread,+ into a few spare batch-1 caches; the decode loop then only copies a ready cache into a row.+ Same modules and kernels as ``BatchDecoder.prefill``, so the states are the same bits; the+ GPU overlaps the (launch-bound) prefill with the decode steps."""++ def __init__(self, dec: "BatchDecoder", prompts, order, depth: int, lens=None):+ import threading++ from transformers import StaticCache++ self.dec, self.prompts, self.lens = dec, prompts, lens+ self.order = list(order)+ self.ready: dict[int, tuple] = {}+ self.cv = threading.Condition()+ self.stream = torch.cuda.Stream()+ while len(dec.spare) < depth:+ dec.spare.append(StaticCache(config=dec.cfg, max_cache_len=dec.max_len))+ self.free = [(c, None) for c in dec.spare[:depth]] # (cache, event: free after it)+ self.stop = False+ self.error = None+ self.thread = threading.Thread(target=self._work, daemon=True)+ self.thread.start()++ def _work(self):+ try:+ with torch.no_grad(), torch.cuda.stream(self.stream):+ for i in self.order:+ with self.cv:+ while not self.free and not self.stop:+ self.cv.wait()+ if self.stop:+ return+ cache, ev = self.free.pop()+ if ev is not None:+ self.stream.wait_event(ev) # the row copy out of it is done+ if self.lens is not None: # a cache of generate's length for this prompt+ from transformers import StaticCache++ cache = StaticCache(config=self.dec.cfg, max_cache_len=self.lens[i])+ t0 = time.perf_counter()+ t = self.dec._prefill_into(cache, self.prompts[i])+ done = torch.cuda.Event()+ done.record(self.stream)+ with self.cv:+ self.dec.stats["prefills"] += 1+ self.dec.stats["prefill_s"] += time.perf_counter() - t0+ self.ready[i] = (cache, t, done)+ self.cv.notify_all()+ except BaseException as e: # noqa: BLE001 - surfaced in get()+ with self.cv:+ self.error = e+ self.cv.notify_all()++ def get(self, i: int):+ """(cache, first token) for prompt ``i``; call ``release`` after copying from it."""+ with self.cv:+ while i not in self.ready:+ if self.error is not None:+ raise self.error+ self.cv.wait()+ cache, t, done = self.ready.pop(i)+ torch.cuda.current_stream().wait_event(done)+ return cache, t++ def release(self, cache):+ ev = torch.cuda.Event()+ ev.record()+ with self.cv:+ self.free.append((cache, ev))+ self.cv.notify_all()++ def close(self):+ with self.cv:+ self.stop = True+ self.cv.notify_all()+ self.thread.join()+++class BatchDecoder:+ """Greedy decoding of many prompts through one model, ``batch`` rows at a time.++ ``sizes`` are the batch sizes with a compiled step (the largest is the full batch); when the+ queue is empty and the live rows fit a smaller size, they are compacted into it.+ """++ def __init__(self, model, batch: int = 16, max_len: int = 8192, eos=None,+ compile: bool = True, sizes=None, mode: str = "graphs",+ fullgraph: bool = True, skip_qwen: bool = False, chip_per_full: int = 3,+ fold_gqa: bool = False, view: int = 0, prefetch: int = 2,+ freeze_chip_positions: bool = False, kv_len: str | None = None,+ transparent_chip: bool = False):+ from transformers import StaticCache++ from chipformer.q35.fast import _n_chip_layers, chip_meta, exact_matmuls++ exact_matmuls() # batch sizes change cuBLAS's kernel choice; keep the circuit exact+ if fold_gqa:+ _use_fold(model)++ self.model = model+ self.lm = _text_model(model)+ self.head = model.lm_head # an nn.Linear, or fast.SparseLinear after fast.compact+ self.cfg = getattr(model.config, "text_config", model.config)+ self.dev = model.lm_head.weight.device+ self.max_len = max_len+ if eos is None:+ eos = model.generation_config.eos_token_id+ eos = [eos] if isinstance(eos, int) else list(eos or [])+ self.eos = set(eos)+ # generate's bad_words_ids / suppress_tokens (single tokens): never predicted+ gc = model.generation_config+ ban = set(getattr(gc, "suppress_tokens", None) or [])+ for w in getattr(gc, "bad_words_ids", None) or []:+ assert len(w) == 1, f"multi-token bad word {w} is not supported"+ ban.add(w[0])+ self.banned = torch.tensor(sorted(ban), dtype=torch.long, device=self.dev) if ban else None+ self.small = StaticCache(config=self.cfg, max_cache_len=max_len)+ self.sizes = sorted(set(sizes or [batch]) | {batch})+ self.batch = batch+ self.compile = compile+ self.mode, self.fullgraph = mode, fullgraph+ self.levels = {} # batch size -> (rows cache, tok, pos, step, chip step, chip keys)+ self.ready = set()+ self.pool, self.graphs = None, []+ # attention views: KV slots read per step, by the longest live row+ self.views = sorted({*range(view, max_len, view), max_len}) if view else [max_len]+ self.view_graphs = {} # batch size -> {view: [text step, chip step]}+ self.raw = {} # batch size -> compiled (uncaptured) steps+ self.prefetch = prefetch # prompts prefilled ahead, on a second stream+ self.spare = [] # their batch-1 caches+ torch._dynamo.config.cache_size_limit = max(torch._dynamo.config.cache_size_limit, 64)+ self.slots = torch.arange(max_len, device=self.dev)+ self.chip_per_full = chip_per_full+ # kv_len="generate": attention reads exactly generate's static cache length+ # (prompt + max_new) in the prefill and every step. The attention kernel's result+ # depends on that length, so this is what makes batch 1 bit-identical to+ # generate(cache_implementation="static", disable_compile=True). Eager kernels only.+ self.kv_exact = kv_len == "generate"+ assert not (self.kv_exact and compile), "kv_len='generate' needs compile=False"+ meta = chip_meta(model)+ self.rpos = {} # batch size -> RoPE position buffer (``pos`` itself unless frozen)+ self.freeze = (freeze_chip_positions or transparent_chip) and meta is not None+ if meta is not None:+ self.chip_lo = meta["chip_token_offset"]+ self.chip_hi = self.chip_lo + meta["chip_vocab"]+ # transparent_chip (implies freeze_chip_positions): chip tokens are invisible to Qwen's+ # layers. A chip-token input leaves their conv windows as they were (the stock model+ # shifts in a zero), and Qwen's full attention keeps its KV by text position: a chip+ # token's key goes to the slot the next text token overwrites, so text keys are+ # contiguous and no query ever reads a chip key. Qwen's decode steps then compute what+ # they would on the same text with the chip tokens deleted (dist/spikes/b26_*.py).+ self.transparent = transparent_chip and meta is not None+ if meta is not None:+ self.n_chip = _n_chip_layers(model)+ self.skip_qwen = skip_qwen and meta is not None+ if self.skip_qwen:+ self.n_chip, self.chip_dims = _n_chip_layers(model), meta["chip_dims_offset"]+ self.chip_lo = meta["chip_token_offset"]+ self.chip_hi = self.chip_lo + meta["chip_vocab"]+ self.stats = {"view_sum": 0, "chip_steps": 0, "steps": 0, "row_steps": 0, "tokens": 0, "prefills": 0, "prefill_s": 0.0}++ # --- one batch size ---------------------------------------------------------------------++ def _level(self, n: int):+ if n not in self.levels:+ pos = _mark(torch.zeros(n, dtype=torch.long, device=self.dev))+ tok = _mark(torch.zeros(n, 1, dtype=torch.long, device=self.dev))+ # RoPE position per row: the slot, or with freeze_chip_positions the number of+ # non-chip tokens before it (chip layers have no RoPE; Qwen sees contiguous text)+ rpos = _mark(torch.zeros(n, dtype=torch.long, device=self.dev)) if self.freeze else pos+ self.rpos[n] = rpos+ transparent = self.transparent+ kvpos = rpos if transparent else pos # the KV slot each row writes+ rows = _Rows(self.cfg, self.small, n, self.max_len, pos, kvpos)+ cache, slots, lm, head = rows.cache, self.slots, self.lm, self.head+ if transparent:+ kinds = self.cfg.layer_types+ qlin = [cache.layers[i] for i in range(self.n_chip, len(kinds))+ if kinds[i] == "linear_attention"]+ lo, hi = self.chip_lo, self.chip_hi++ def step(tok, pos):+ # slots past the view are masked anyway: reading fewer of them is bit-identical+ # (fully masked key blocks add exact zeros; dist/spikes/b14_fold.py)+ mask = slots[None, :rows.view] <= kvpos[:, None]+ if transparent:+ ischip = (tok[:, 0] >= lo) & (tok[:, 0] < hi)+ saved = [L.conv_states[0].clone() for L in qlin]+ h = lm(input_ids=tok, position_ids=rpos[:, None], past_key_values=cache,+ attention_mask={"full_attention": mask[:, None, None, :],+ "linear_attention": None},+ use_cache=True).last_hidden_state[:, -1]+ if transparent: # chip inputs: Qwen's conv windows stay as they were+ for L, c in zip(qlin, saved):+ L.conv_states[0].copy_(torch.where(ischip[:, None, None], c,+ L.conv_states[0]))+ return self._argmax(head(h))++ chip = kconst = None+ act = _mark(torch.ones(n, dtype=torch.bool, device=self.dev))+ if self.skip_qwen:+ chip, kconst = self._chip_step(rows, n, act, kvpos)+ if self.compile:+ # "graphs": inductor kernels, captured below as one plain CUDA graph per step+ # kind (no cudagraph-tree bookkeeping between the two kinds of step)+ mode = None if self.mode == "graphs" else self.mode+ step = torch.compile(step, mode=mode, fullgraph=self.fullgraph)+ if chip is not None:+ chip = torch.compile(chip, mode=mode, fullgraph=self.fullgraph)+ self.levels[n] = (rows, tok, pos, step, chip, kconst, act)+ return self.levels[n]++ def _chip_step(self, rows, n, act, kvpos):+ """The step for rows whose input is a chip token. Qwen's layers are exact no-ops there+ (muted: residual unchanged, DeltaNet state unchanged, conv window shifts in a zero,+ full attention stores a value of 0 under a fixed key; dist/spikes/b5_skip.py checks all+ of it bitwise), so they are replaced by those cache writes, and the head reads only the+ chip dims (Qwen's dims of the hidden state are exactly 0)."""+ from chipformer.q35.fast import SparseLinear++ lm, cache, kinds = self.lm, rows.cache, self.cfg.layer_types+ n_chip, d0 = self.n_chip, self.chip_dims+ attn = [cache.layers[i] for i in range(n_chip, len(kinds)) if kinds[i] == "full_attention"]+ lin = [cache.layers[i] for i in range(n_chip, len(kinds))+ if kinds[i] == "linear_attention"]+ kconst = [_mark(L.keys.new_zeros(L.keys.shape[1], L.keys.shape[-1])) for L in attn]+ w = self.head.weight+ c0 = self.head.c0 if isinstance(self.head, SparseLinear) else 0+ c1 = c0 + w.shape[1]+ if not isinstance(self.head, SparseLinear):+ from chipformer.q35.fast import _hull, nonzero_cols++ c1 = _hull(nonzero_cols(w))[1]+ block = w[:, d0 - c0:c1 - c0].contiguous()+ chip_layers = lm.layers[:n_chip]+ chip_lin = [cache.layers[i] for i in range(n_chip) if kinds[i] == "linear_attention"]+ transparent = self.transparent+ def chip(tok, pos):+ # rows with act False (text rows waiting for a full step) come out untouched;+ # ``act`` is the level's buffer, all True unless the caller sets it+ saved = [(L.conv_states[0].clone(), L.recurrent_states[0].clone()) for L in chip_lin]+ h = lm.embed_tokens(tok)+ for layer in chip_layers:+ h = layer(h, position_embeddings=None, past_key_values=cache, use_cache=True)+ a3 = act[:, None, None]+ for L, (c, r) in zip(chip_lin, saved):+ L.conv_states[0].copy_(torch.where(a3, L.conv_states[0], c))+ L.recurrent_states[0].copy_(torch.where(act[:, None, None, None],+ L.recurrent_states[0], r))+ idx = torch.arange(pos.shape[0], device=pos.device)+ for L, k in zip(attn, kconst):+ L.keys[idx, :, kvpos] = torch.where(a3, k, L.keys[idx, :, kvpos])+ L.values[idx, :, kvpos] = torch.where(a3, 0.0, L.values[idx, :, kvpos])+ if not transparent: # transparent: Qwen's conv windows stay as they were+ for L in lin:+ c = L.conv_states[0]+ c.copy_(torch.where(a3, torch.nn.functional.pad(c[..., 1:], (0, 1)), c))+ h = lm.norm(h)[:, -1, d0:c1]+ return self._argmax(torch.nn.functional.linear(h, block))++ return chip, kconst++ def _argmax(self, logits):+ if self.banned is not None: # as generate's logits processors do: score -inf+ logits = logits.index_fill(-1, self.banned, float("-inf"))+ return logits.argmax(-1)++ @torch.no_grad()+ def prefill(self, ids: list[int]) -> int:+ """Batch-1 prefill into ``self.small`` (as ``generate`` does it); returns the first token."""+ t0 = time.perf_counter()+ t = self._prefill_into(self.small, ids)+ self.stats["prefills"] += 1+ self.stats["prefill_s"] += time.perf_counter() - t0+ return t++ def _rope_positions(self, ids: list[int]) -> torch.Tensor:+ if not self.freeze:+ return torch.arange(len(ids), device=self.dev)+ lo, hi = self.chip_lo, self.chip_hi+ p, out = 0, []+ for t in ids:+ out.append(p)+ p += not lo <= t < hi+ return torch.tensor(out, device=self.dev)++ def _is_text(self, tok):+ return (tok < self.chip_lo) | (tok >= self.chip_hi)++ def chip_mask(self, ids: list[int]) -> torch.Tensor:+ """[len(ids)] bool on the device: which tokens are chip tokens."""+ x = torch.tensor(ids, device=self.dev)+ return (x >= self.chip_lo) & (x < self.chip_hi)++ def mark_prompt(self, n: int, b: int, ids: list[int]) -> None:+ """Call after loading a prefilled prompt into row ``b`` of the batch-``n`` level. With+ transparent_chip it moves the row's full-attention KV to text positions (the prefill+ wrote one slot per token); a no-op otherwise."""+ if not self.transparent:+ return+ text = (~self.chip_mask(ids)).nonzero().flatten()+ if len(text) == len(ids):+ return+ for L in self.levels[n][0].cache.layers:+ if getattr(L, "keys", None) is not None:+ L.keys[b, :, :len(text)] = L.keys[b, :, text]+ L.values[b, :, :len(text)] = L.values[b, :, text]++ def _prefill_into(self, cache, ids: list[int]) -> int:+ cache.reset()+ x = torch.tensor([ids], device=self.dev)+ mask = torch.ones_like(x)+ if self.transparent:+ ch = self.chip_mask(ids)+ if bool(ch.any()):+ # chip keys are invisible (a chip query keeps its own). A leading run of chip+ # tokens (the BIOS) also matches Qwen's zero-padded conv window exactly; chip+ # tokens later in a prompt still shift zeros into the conv windows.+ n = len(ids)+ size = next(L.max_cache_len for L in cache.layers if hasattr(L, "max_cache_len"))+ q = torch.arange(n, device=self.dev)[:, None]+ k = torch.arange(size, device=self.dev)[None, :]+ vis = (k <= q) & (~torch.nn.functional.pad(ch, (0, size - n))[None, :] | (k == q))+ mask = {"full_attention": vis[None, None], "linear_attention": None}+ h = self.lm(input_ids=x, position_ids=self._rope_positions(ids)[None],+ attention_mask=mask, past_key_values=cache,+ use_cache=True).last_hidden_state[:, -1:]+ return int(self._argmax(self.head(h)[0, -1]))++ @torch.no_grad()+ def _prepare(self, n: int):+ """The level of batch size ``n``, compiled and warmed; for the chip step, its fixed+ keys are taken from a real full step on a chip-token input."""+ level = self._level(n)+ if n in self.ready:+ return level+ rows, tok, pos, step, chip, kconst, _act = level+ tok.zero_()+ pos.zero_()+ for _ in range(3):+ step(tok, pos)+ if chip is not None:+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ boot = [t + self.chip_lo for t in boot_tokens(Chip8(bytes([0x12, 0x00])).cells)]+ k = len(boot) - 1+ self.prefill(boot[:k])+ rows.load_row(0, self.small, k)+ tok[0, 0], pos[0] = boot[k], k+ step(tok, pos)+ kinds = self.cfg.layer_types+ attn = [rows.cache.layers[i] for i in range(self.n_chip, len(kinds))+ if kinds[i] == "full_attention"]+ for L, kc in zip(attn, kconst):+ kc.copy_(L.keys[0, :, int((self.rpos[n] if self.transparent else pos)[0])])+ tok.fill_(boot[k])+ pos.zero_()+ for _ in range(3):+ chip(tok, pos)+ tok.zero_()+ pos.zero_()+ if self.mode == "graphs": # compiled or eager kernels, captured as CUDA graphs+ # the compiled functions own buffers their kernels read (e.g. inductor's copies of+ # closure tensors): they must outlive the graphs, or replays read freed memory+ self.raw[n] = (step, chip)+ graphs = {}+ for view in self.views:+ rows.view = view+ tok.zero_()+ pos.zero_()+ for _ in range(2):+ step(tok, pos)+ g = [self._capture(step, tok, pos), None]+ if chip is not None: # the chip step reads no keys: one graph serves all views+ if not graphs:+ tok.fill_(self.chip_lo)+ for _ in range(2):+ chip(tok, pos)+ g[1] = self._capture(chip, tok, pos)+ else:+ g[1] = next(iter(graphs.values()))[1]+ graphs[view] = g+ rows.view = self.max_len+ tok.zero_()+ pos.zero_()+ self.view_graphs[n] = graphs+ level = self.levels[n] = (rows, tok, pos, *graphs[self.max_len], kconst, _act)+ torch.cuda.synchronize()+ self.ready.add(n)+ return level++ def _view_graph(self, n: int, v: int):+ """[text step, chip step] graphs of batch size ``n`` whose attention reads ``v`` slots."""+ graphs = self.view_graphs[n]+ if v not in graphs:+ rows, tok, pos = self.levels[n][:3]+ rows.view = v+ g = self._capture(self.raw[n][0], tok, pos, warm=False)+ rows.view = self.max_len+ graphs[v] = [g, graphs[self.max_len][1]]+ return graphs[v]++ def _capture(self, fn, tok, pos, warm: bool = True):+ """``fn(tok, pos)`` as a CUDA graph over the level's static buffers; the returned+ callable takes the same (ignored: always those buffers) arguments. ``warm=False``+ records without running it first (safe mid-run: recording runs no kernel)."""+ torch.cuda.synchronize()+ s = torch.cuda.Stream()+ s.wait_stream(torch.cuda.current_stream())+ with torch.cuda.stream(s): # warm up on the capture stream (lazy cuBLAS state)+ for _ in range(2 if warm else 0):+ fn(tok, pos)+ torch.cuda.current_stream().wait_stream(s)+ torch.cuda.synchronize()+ g = torch.cuda.CUDAGraph()+ # thread_local: the prefetch thread may be running a prefill on its own stream+ with torch.cuda.graph(g, pool=self.pool, stream=s, capture_error_mode="thread_local"):+ out = fn(tok, pos)+ self.pool = g.pool()+ self.graphs.append(g)++ def replay(tok_=None, pos_=None):+ g.replay()+ return out++ return replay++ @torch.no_grad()+ def warmup(self, prompt: list[int] | None = None) -> None:+ """Compile (and record the CUDA graphs of) every batch size's steps ahead of time."""+ if prompt is not None:+ self.prefill(prompt)+ for n in self.sizes:+ self._prepare(n)++ # --- the loop -----------------------------------------------------------------------------++ @torch.no_grad()+ def run(self, prompts: list[list[int]], max_new: int, on_done=None,+ hook=None) -> list[list[int]]:+ """Greedy continuations (each ending with its end token, or ``max_new`` long).++ ``hook(i, t) -> (t_used, stop)`` sees every token of prompt ``i`` as it is produced: it+ may replace it (token forcing: ``t_used`` is what is kept and fed back) or end the row+ (``stop``). With a hook the loop reads every step before queueing the next one.+ ``on_done(i, tokens)`` is called as each prompt finishes."""+ for p in prompts:+ assert len(p) + max_new <= self.max_len, (len(p), max_new, self.max_len)+ if not prompts:+ return []+ if not self.small.layers[0].is_conv_states_initialized[0]:+ self.prefill(prompts[0]) # the batched cache copies a prefilled cache's shapes+ self._prepare(self.batch)+ lens = [len(p) + max_new for p in prompts] if self.kv_exact else None+ # record every graph now (all batch sizes, and generate-length views): capturing+ # mid-run would race the prefetch thread's allocations+ for m in self.sizes:+ self._prepare(m)+ for v in sorted(set(lens or [])) if m in self.view_graphs else []:+ self._view_graph(m, min(v, self.max_len))+ pre = _Prefetcher(self, prompts, range(len(prompts)), max(1, self.prefetch), lens)+ try:+ return self._run(prompts, max_new, on_done, pre, hook)+ finally:+ pre.close()++ def _run(self, prompts, max_new, on_done, pre, hook=None):+ out: list[list[int] | None] = [None] * len(prompts)+ queue = deque(range(len(prompts)))+ n = self.batch+ rows, tok, pos, step, chip, _, act = self._prepare(n)+ rpos = self.rpos[n]+ live: list[int | None] = [None] * n # prompt index per row+ gen: list[list[int]] = [[] for _ in range(n)]+ plen = [0] * n # prompt length per row++ def finish(b):+ i = live[b]+ out[i] = gen[b]+ live[b] = None+ if on_done is not None:+ on_done(i, gen[b])++ def admit(b):+ while queue:+ i = queue.popleft()+ cache, t = pre.get(i)+ stop = False+ if hook is not None:+ t, stop = hook(i, t)+ live[b], gen[b] = i, [t]+ if stop or t in self.eos or max_new <= 1:+ pre.release(cache)+ finish(b)+ continue+ rows.load_row(b, cache, len(prompts[i]))+ pre.release(cache)+ tok[b, 0] = t+ pos[b] = len(prompts[i])+ plen[b] = len(prompts[i])+ if self.freeze:+ rpos[b] = sum(not self.chip_lo <= x < self.chip_hi for x in prompts[i])+ self.mark_prompt(n, b, prompts[i])+ return+ tok[b, 0] = 0 # an idle row decodes garbage at slot 0 until it is reused+ pos[b] = 0++ for b in range(n):+ admit(b)+ # The host runs one step ahead: step t+1 is queued before the tokens of step t are read,+ # so the GPU never waits for Python. A row that finishes at step t has its (stale) step+ # t+1 token dropped; a row refilled then starts with step t+2.+ host = [torch.zeros(self.batch, dtype=torch.long, pin_memory=True) for _ in range(2)]+ skip = [0] * n+ flip = 0++ lo, hi = (self.chip_lo, self.chip_hi) if chip is not None else (0, 0)+ budget = 0 # chip steps left before the waiting text rows get a full step++ def launch():+ """One step. With skip_qwen, rows whose input is a chip token take the cheap chip+ step while text rows wait (up to ``chip_per_full`` chip steps per full step); a full+ step advances every row (it is exact for chip rows too)."""+ nonlocal flip, budget+ on = None+ if chip is not None:+ on = [live[b] is not None and lo <= gen[b][-1] < hi for b in range(n)]+ text = any(live[b] is not None and not on[b] for b in range(n))+ if any(on) and (not text or budget > 0):+ budget -= 1+ else:+ on = None+ budget = self.chip_per_full+ step_fn, chip_fn = step, chip+ if n in self.view_graphs and self.kv_exact: # generate's cache length+ v = max((plen[b] + max_new for b in range(n) if live[b] is not None), default=0)+ step_fn, chip_fn = self._view_graph(n, min(max(v, 1), self.max_len))+ elif n in self.view_graphs: # the smallest attention view covering every live row+ need = 1 + max((plen[b] + len(gen[b]) for b in range(n) if live[b] is not None),+ default=0)+ v = next(v for v in self.views if v >= need)+ step_fn, chip_fn = self.view_graphs[n][v]+ self.stats["view_sum"] += v+ if on is not None:+ act.copy_(torch.tensor(on), non_blocking=True)+ nxt = chip_fn(tok, pos)+ tok.copy_(torch.where(act[:, None], nxt[:, None], tok))+ pos.add_(act)+ self.stats["chip_steps"] += 1+ else:+ nxt = step_fn(tok, pos)+ if self.freeze: # the input was text: the next RoPE position moves on+ rpos.add_(self._is_text(tok[:, 0]))+ tok.copy_(nxt[:, None])+ pos.add_(1)+ pos.clamp_(max=self.max_len - 1) # idle rows count up; live rows never get there+ buf = host[flip][:n]+ flip ^= 1+ buf.copy_(nxt, non_blocking=True)+ ev = torch.cuda.Event()+ ev.record()+ self.stats["steps"] += 1+ self.stats["row_steps"] += n+ return ev, buf, on++ def collect(res, inflight: bool):+ ev, buf, on = res+ ev.synchronize()+ for b, t in enumerate(buf.tolist()):+ if skip[b]:+ skip[b] -= 1+ continue+ if live[b] is None or (on is not None and not on[b]):+ continue+ stop = False+ if hook is not None:+ t2, stop = hook(live[b], t)+ if t2 != t: # forced: feed the forced token instead+ tok[b, 0] = t2+ t = t2+ gen[b].append(t)+ self.stats["tokens"] += 1+ if stop or t in self.eos or len(gen[b]) >= max_new:+ finish(b)+ skip[b] = int(inflight)+ admit(b)++ pending = None+ while any(i is not None for i in live):+ cur = launch()+ if chip is not None or hook is not None: # the next step depends on them: no lag+ collect(cur, False)+ cur = None+ elif pending is not None:+ collect(pending, True)+ pending = cur+ if not queue:+ alive = [b for b in range(n) if live[b] is not None]+ smaller = [s for s in self.sizes if s < n and s >= len(alive)]+ if smaller:+ if pending is not None:+ collect(pending, False)+ pending = None+ alive = [b for b in range(n) if live[b] is not None]+ if not alive:+ break+ m = smaller[0]+ rows2, tok2, pos2, step2, chip2, _, act2 = self._prepare(m)+ idx = torch.tensor(alive, device=self.dev)+ rows.take(idx, rows2)+ tok2[: len(alive)].copy_(tok[idx])+ if self.freeze:+ self.rpos[m][: len(alive)].copy_(rpos[idx])+ self.rpos[m][len(alive):] = 0+ pos2[: len(alive)].copy_(pos[idx])+ pos2[len(alive):] = 0+ tok2[len(alive):] = 0+ live = [live[b] for b in alive] + [None] * (m - len(alive))+ gen = [gen[b] for b in alive] + [[] for _ in range(m - len(alive))]+ plen = [plen[b] for b in alive] + [0] * (m - len(alive))+ skip = [0] * m+ n, rows, tok, pos, step, chip, act = m, rows2, tok2, pos2, step2, chip2, act2+ rpos = self.rpos[m]+ if pending is not None:+ collect(pending, False)+ return out+++_DECODERS: dict = {}+++def generate_batch(model, tok, prompts: list[list[int]], max_new: int, batch: int | None = None,+ max_len: int | None = None, **kw) -> list[list[int]]:+ """Greedy continuations of ``prompts`` (token id lists), equal to running each alone.++ ``tok`` is unused (end tokens come from ``model.generation_config``); it is accepted so the+ call reads like the usual ``generate`` helpers. The decoder (and its compiled steps) is+ reused by later calls with the same model, batch, ``max_len`` and options; a call with+ different ones replaces it (one decoder per model, to bound memory).+ """+ del tok+ if not prompts:+ return []+ batch = batch or min(16, len(prompts))+ if max_len is None:+ need = max(len(p) for p in prompts) + max_new+ max_len = -(-need // 1024) * 1024+ sizes = kw.pop("sizes", None) or sorted({batch, *(s for s in (1, 2, 4, 8, 16, 32)+ if s < batch)})+ key = (batch, max_len, tuple(sizes), tuple(sorted(kw.items())))+ old = _DECODERS.get(id(model))+ if old is None or old[0] != key:+ _DECODERS.pop(id(model), None)+ torch.cuda.empty_cache()+ _DECODERS[id(model)] = (key, BatchDecoder(model, batch=batch, max_len=max_len,+ sizes=sizes, **kw))+ return _DECODERS[id(model)][1].run(prompts, max_new)+++def generate_fast(model, prompts: list[list[int]], max_new: int, batch: int | None = None,+ compact_qwen: bool = False, skip_qwen: bool = True, **kw) -> list[list[int]]:+ """Fast greedy decoding of a merged q35 checkpoint: the drop-in for++ [model.generate(torch.tensor([p]), do_sample=False, max_new_tokens=max_new)[0, len(p):]+ .tolist() for p in prompts]++ Each continuation ends with its end token (generation_config.eos_token_id) or is+ ``max_new`` long. On first use it compacts the chip layers in place (``fast.compact``: only+ their nonzero weight blocks are read, every logit stays bit-identical), and with+ ``skip_qwen`` chip-token inputs run only the chip layers (Qwen's layers are exact no-ops+ there). ``compact_qwen=True`` also compacts Qwen's layers and the head: faster at text+ steps, but Qwen's sums round differently (1 of 12 demo answers changed its final period). Prompts are+ decoded ``batch`` at a time with continuous batching (``batch=1`` is the latency setting).+ The compacted model still works with ``generate`` and plain forwards.+ """+ from chipformer.q35.fast import compact++ done = getattr(model, "_q35_compact", None)+ if done is None:+ compact(model, qwen=compact_qwen)+ model._q35_compact = compact_qwen+ elif compact_qwen and not done:+ compact(model, qwen=True)+ model._q35_compact = True+ return generate_batch(model, None, prompts, max_new, batch=batch, skip_qwen=skip_qwen, **kw)diff --git a/experimental/riscv/chipformer/q35/bridge.py b/experimental/riscv/chipformer/q35/bridge.pynew file mode 100644index 0000000..bd4787f--- /dev/null+++ b/experimental/riscv/chipformer/q35/bridge.py@@ -0,0 +1,250 @@+"""The read bridge: the computer prints text that Qwen reads, inside one token stream.++Token space of the v2 circuit: the 8708 chip tokens, then a few *text classes* standing for+Qwen's own text tokens (every other text token is OTHER). In a merged model each Qwen text token+row carries the features of its class, so the circuit sees text but only acts on these.++SCHIP's 00FD ("exit") is redefined as print-and-halt: the record that would execute it prints++ \\n <tool_response> \\n h(mem[I]) h(mem[I+1]) ... h(mem[I+V0-1]) \\n </tool_response> \\n++as real text tokens (h = the low nibble as a hex digit, so BCD digits print as decimals), then+the machine stops and Qwen continues. Nothing in the machine state changes.+"""++from chipformer import spec+from chipformer.ref.emu import fetch+from chipformer.trace import Machine, encode_writes++TEXT_BASE = spec.VOCAB_SIZE # 8708+HEX = list(range(TEXT_BASE, TEXT_BASE + 16)) # '0'..'9', 'a'..'f'+NL, TR_OPEN, TR_CLOSE, TC_OPEN, TC_CLOSE, OTHER, FN_ADD, FN_MUL, FN_SUB = range(+ TEXT_BASE + 16, TEXT_BASE + 25)+# inline arithmetic ("37 \\times 43 =", see inline_scan)+SPACE, EQ, PLUS, MINUS, TIMES, BSL, TIMESW, COMMA, DOLLAR, LPAREN, WS = range(TEXT_BASE + 25,+ TEXT_BASE + 36)+# "$" splits by spelling: the circuit prints "$", "$$" and "." to close an inline result, so those+# classes may have no other spellings; " $", " $$" and "\$" (currency) are their own classes+SDOLLAR, DDOLLAR, SDDOLLAR, CURR, PERIOD = range(TEXT_BASE + 36, TEXT_BASE + 41)+DOLLARS = (DOLLAR, SDOLLAR, DDOLLAR, SDDOLLAR, CURR) # all neutral, all may precede a number+# " -" starts a negative inline result, so its other spellings ("-", "−") are their own class+MINUS2 = TEXT_BASE + 41+# division: "/" (" /", "÷"), the tool name "div" (also "\div"), and "..." after an inexact quotient+DIV, FN_DIV, ELLIPSIS = range(TEXT_BASE + 42, TEXT_BASE + 45)+N_TEXT = 45+FUNCS = {"add": FN_ADD, "mul": FN_MUL, "sub": FN_SUB, "div": FN_DIV} # tool names it answers+PRINT_OP = 0x00FD+MAX_PRINT = 58 # digits per print (the write counter tops out at 64)+MAX_ARG = 24 # digits per tool-call argument++# Qwen3.5 token ids of the classes (single tokens in its tokenizer)+QWEN_TEXT = {**{TEXT_BASE + i: c for i, c in enumerate("0123456789abcdef")},+ NL: "\n", TR_OPEN: "<tool_response>", TR_CLOSE: "</tool_response>",+ TC_OPEN: "<tool_call>", TC_CLOSE: "</tool_call>", FN_ADD: "add", FN_MUL: "mul",+ FN_SUB: "sub", SPACE: " ", EQ: " =", PLUS: " +", MINUS: " -", TIMES: " \u00d7",+ BSL: " \\", TIMESW: "times", COMMA: ",", DOLLAR: "$", LPAREN: "(",+ WS: "\n\n", SDOLLAR: " $", DDOLLAR: "$$", SDDOLLAR: " $$", CURR: " \\$",+ PERIOD: ".", MINUS2: "-", DIV: "/", FN_DIV: "div", ELLIPSIS: "..."}+# other single tokens Qwen generates for the same thing: it writes <function=add> as "=add" and+# <function=mul> as "=m" "ul", not the way its tokenizer splits the text+# (never for a class the circuit prints: the tied lm_head rows would tie)+EXTRA_TEXT = {FN_ADD: ["=add"], FN_MUL: ["ul"], FN_SUB: ["=sub", "ub"], EQ: ["="], PLUS: ["+"],+ MINUS2: [" \u2212", "\u2212"], DIV: [" /", "\u00f7"], TIMES: ["\u00d7", " *", "*"], BSL: ["\\"],+ TIMESW: ["cdot"], CURR: ["\\$"], LPAREN: [" ("],+ WS: ["\n\n\n", "\n\n\n\n", " ", " ", " ", " ", " \n", " \n\n"]}+INLINE_OPS = {PLUS: "add", MINUS: "sub", MINUS2: "sub", TIMES: "mul", DIV: "div"}+# and after a backslash: TIMESW ("\times", "\cdot") mul, FN_DIV ("\div") div+BSL_OPS = {TIMESW: "mul", FN_DIV: "div"}+# Inline calls need at least this many digits in A and B together. Qwen gets small ones right,+# and right after a trace its conv window holds chip tokens instead of the "=", so a short+# printed result can read to it like the start of a new expression. Qwen also adds and+# subtracts numbers of a few digits reliably, so those stay its own.+INLINE_MIN_DIGITS = {"mul": 4, "add": 7, "sub": 7, "div": 4}+++def is_text(t: int) -> bool:+ return t >= TEXT_BASE+++def readout(cells: list[int]) -> list[int]:+ from chipformer.q35.lib import bcd++ i, n = cells[spec.I], min(cells[spec.V], MAX_PRINT)+ # 10 and 11 print as "." and "..." (a quotient's decimals), so no hex digits a / b+ shown = {**dict(enumerate(HEX)), 10: PERIOD, 11: ELLIPSIS}+ digits = [shown[cells[(i + k) & 0xFFF] & 15] for k in range(n)]+ mode = cells[bcd.MODE]+ if mode & 1: # an inline call: " digits" (" -digits" below 0) and a closer (the math's+ close = DDOLLAR if mode & 4 else DOLLAR if mode & 2 else PERIOD # "$"/"$$", else ".")+ return [MINUS if mode & 8 else SPACE, *digits, close] if digits else [SPACE]+ return [NL, TR_OPEN, NL, *digits, NL, TR_CLOSE, NL]+++class InlineScan:+ """Which "A op B =" the circuit answers: the reference for its capture logic.++ Text since the last *breaker* token (anything but digits, spaces, operators, "=", "$", "(",+ a backslash, or a comma right after a digit) must be exactly: digits, one operator, digits,+ then "=", with at least INLINE_MIN_DIGITS[op] digits in the two numbers together. A comma+ between digits joins them (1,234). The first number must start after a+ space, newline, other whitespace, "$", "(" or a chip token, so "2.5 * 3 =" and "x^2 + 1 =" are left alone.+ Chip tokens are breakers too. Fed every token of the stream, in order.+ """++ def __init__(self):+ self.prev = self.prev2 = None # None: no token yet (treated like a chip token)+ self.p = self.nops = self.d = 0+ self.lens = {1: 0, 2: 0}+ self.digits = {}+ self.op, self.op_ok, self.a_safe, self.in_call = None, False, False, False+ self.opener = 0 # what the first number follows: 1 "$", 2 "$$" (maybe with a space)++ def mode(self) -> int:+ """The CALL record's MODE cell for an inline call: 1 | 2 inside "$" | 4 inside "$$"."""+ return 1 | (2 if self.opener else 0) | (4 if self.opener == 2 else 0)++ def feed(self, t: int):+ """Returns (op, a, b) if the circuit takes over after token t, else None."""+ prev, prev2 = self.prev, self.prev2+ self.prev, self.prev2 = t, prev+ text = is_text(t)+ dig = text and t in HEX[:10]++ def isd(x):+ return x is not None and x in HEX[:10]++ if t in (TC_OPEN, TC_CLOSE) or t in (spec.STEP, spec.BOOT):+ self.in_call = t == TC_OPEN+ if t == TC_OPEN:+ self.op = None # a tool call needs its own function name+ cont = dig and (isd(prev) or (prev == COMMA and isd(prev2)))+ run_start = dig and not cont+ tw_op = t in BSL_OPS and prev == BSL+ op_tok = t in INLINE_OPS or tw_op+ neutral = text and (dig or op_tok or t in (SPACE, LPAREN, BSL, EQ, *DOLLARS)+ or (t == COMMA and isd(prev)))+ brk = not neutral+ after_eq = prev == EQ and not brk and not self.in_call # a fresh window after "="+ if t == TC_OPEN or (brk and not self.in_call) or after_eq:+ self.p = 0+ elif run_start:+ self.p += 1+ if self.p == 1:+ self.a_safe = prev is not None and (not is_text(prev) or prev in (+ SPACE, NL, LPAREN, WS, *DOLLARS))++ def kind(x):+ return 1 if x in (DOLLAR, SDOLLAR) else 2 if x in (DDOLLAR, SDDOLLAR) else 0++ self.opener = kind(prev) or (kind(prev2) if prev == SPACE else 0)+ if run_start:+ self.d = 0+ elif cont:+ self.d += 1+ if brk or after_eq:+ self.nops = 0+ elif op_tok:+ self.nops += 1+ if dig and self.p in (1, 2):+ self.lens[self.p] = self.d + 1 if self.d < MAX_ARG else 0+ if self.d < MAX_ARG:+ self.digits[self.p, self.d] = HEX.index(t)+ fn = {cls: name for name, cls in FUNCS.items()}.get(t)+ if fn and self.in_call:+ self.op, self.op_ok = fn, self.p == 1+ if op_tok and not self.in_call:+ self.op, self.op_ok = (BSL_OPS[t] if tw_op else INLINE_OPS[t]), self.p == 1+ if (t == EQ and not self.in_call and self.p == 2 and self.nops == 1 and self.op_ok+ and self.a_safe and self.lens[1] and self.lens[2] and self.op+ and self.lens[1] + self.lens[2] >= INLINE_MIN_DIGITS[self.op]):+ a = "".join(str(self.digits[1, w]) for w in range(self.lens[1]))+ b = "".join(str(self.digits[2, w]) for w in range(self.lens[2]))+ return self.op, a, b+ return None+++class BridgeMachine(Machine):+ """Reference trace for the v2 circuit: machine records, prints, and Qwen's text."""++ def __init__(self, rom: bytes, quirks=None):+ super().__init__(rom, quirks)+ self.halted = False+ self.scan = InlineScan()+ for t in self.tokens:+ self.scan.feed(t)++ def _emit(self, tokens: list[int], generated: bool) -> None:+ self.fired = None+ for t in tokens:+ self.fired = self.scan.feed(t)+ super()._emit(tokens, generated)++ def qwen(self, classes: list[int]) -> list[tuple[str, str, str]]:+ """Qwen writes these text tokens one at a time; wherever the circuit answers an inline+ "A op B =", it takes over (STEP, the CALL record, the routine, the print)."""+ calls = []+ for t in classes:+ self._emit([t], False)+ if self.fired:+ calls.append(self.fired)+ self._call(*self.fired, mode=self.scan.mode())+ while not self.halted:+ self.step()+ return calls++ def step(self):+ if self.halted:+ raise RuntimeError("machine halted")+ op = fetch(self.emu.cells)+ if op == PRINT_OP:+ self._emit(readout(self.emu.cells), True)+ self.halted = True+ return []+ from chipformer.q35.lib.macros import macro_writes++ w = macro_writes(self.emu.cells, op)+ if w is not None:+ self.emu.apply(w)+ self._emit(encode_writes(w) + [spec.STEP], True)+ return w+ return super().step()++ def text(self, n: int, cls: int = OTHER) -> None:+ """Qwen writes n text tokens (the circuit must hand over)."""+ self._emit([cls] * n, False)++ def tool_call(self, op: str, a: str, b: str) -> None:+ """Qwen calls a library routine with its native syntax; the circuit then writes the+ CALL record (the arguments into RAM, PC = the routine) and the machine runs it."""++ def lit(n):+ return [OTHER] * n++ da, db = [HEX[int(ch)] for ch in a], [HEX[int(ch)] for ch in b]+ text = ([TC_OPEN, NL] + lit(2) + [FUNCS[op]] + lit(1) + [NL] + lit(3) + [NL] + da+ + [NL] + lit(3) + [NL] + lit(3) + [NL] + db + [NL] + lit(3) + [NL] + lit(3)+ + [NL, TC_CLOSE])+ self._emit(text, False)+ self._call(op, a, b, mode=0)++ def _call(self, op: str, a: str, b: str, mode: int) -> None:+ from chipformer.q35.lib import bcd++ w = bcd.call_writes(op, a, b, mode)+ self.emu.apply(w)+ self._emit([spec.STEP] * bool(mode) + encode_writes(w) + [spec.STEP], True)+ self.halted = False+++def owner_expected(m: BridgeMachine) -> list[int | None]:+ """1 where the circuit must predict the next token, 0 where Qwen does, None for host-only."""+ out = []+ toks, gen = m.tokens, m.generated+ for p, t in enumerate(toks):+ if not is_text(t):+ out.append(1)+ elif p + 1 < len(toks) and gen[p + 1]:+ out.append(1) # inside a print+ else:+ out.append(0)+ return outdiff --git a/experimental/riscv/chipformer/q35/check.py b/experimental/riscv/chipformer/q35/check.pynew file mode 100644index 0000000..050294d--- /dev/null+++ b/experimental/riscv/chipformer/q35/check.py@@ -0,0 +1,51 @@+"""Differential testing of the q35 circuit against the reference trace, token by token."""++import numpy as np++from chipformer import spec+from chipformer.check import Mismatch+from chipformer.q35 import ir+from chipformer.q35.chip8_q35 import Chip8Q35, decode_code+from chipformer.trace import Machine+++def predictions(c: Chip8Q35, tokens: list[int], check: bool = True) -> np.ndarray:+ """Next-token prediction at every position, from the circuit's output bits."""+ m = ir.evaluate(c, tokens, check)+ bits = np.stack([ir.ev(b, m) for b in c.next_bits], 1)+ assert np.isin(bits, [0.0, 1.0]).all(), "output bits must be 0/1"+ return np.array([decode_code(row) for row in bits.astype(int).tolist()]), m+++def check(c: Chip8Q35, m: Machine, limit: int = 10, exact: bool = True):+ """(number of checked predictions, first mismatches)."""+ pred, _ = predictions(c, m.tokens, exact)+ want = np.array(m.tokens[1:] + [-1])+ gen = np.array(m.generated[1:] + [False])+ bad = np.nonzero(gen & (pred != want))[0]+ starts = [p for p, _ in getattr(m, "ops", [])]+ out = []+ for p in bad[:limit]:+ i = np.searchsorted(starts, p + 1, side="right") - 1+ op = m.ops[i][1] if i >= 0 and starts else "?"+ out.append(Mismatch(int(p), int(want[p]), float(pred[p]), op))+ return int(gen.sum()), out+++def describe(mm: list[Mismatch]) -> str:+ return "\n".join(f" {x}" for x in mm) or " none"+++__all__ = ["check", "describe", "predictions", "spec"]+++def check_bridge(c: Chip8Q35, tokens: list[int], generated: list[bool], owner_want, limit=10):+ """Predictions at generated positions, and the owner bit everywhere it is defined."""+ pred, m = predictions(c, tokens)+ owner = ir.ev(c.owner, m)+ want = np.array(tokens[1:] + [-1])+ gen = np.array(generated[1:] + [False])+ bad = [(int(p), int(want[p]), int(pred[p])) for p in np.nonzero(gen & (pred != want))[0]]+ bad_owner = [(p, float(owner[p]), w) for p, w in enumerate(owner_want)+ if w is not None and owner[p] != w]+ return int(gen.sum()), bad[:limit], bad_owner[:limit]diff --git a/experimental/riscv/chipformer/q35/chip8_q35.py b/experimental/riscv/chipformer/q35/chip8_q35.pynew file mode 100644index 0000000..e251ccb--- /dev/null+++ b/experimental/riscv/chipformer/q35/chip8_q35.py@@ -0,0 +1,1074 @@+"""CHIP-8 as a q35 circuit: the same machine as chipformer/chip8.py, built for Qwen3.5 layers.++Same structure as the original: every position computes the instruction context from the state+it reads (meaningful at STEP/BOOT positions), a DeltaNet slot carries that context from the+latest record start to the rest of the record, and a per-position micro-step picks the output.+What changes is how state is read:++ memory the delta rule with one-hot keys overwrites a state row, so a cell's row always+ holds its latest write; a read is a one-hot query. RAM is two layers of 16 heads+ (head = address bits 3:0, so PC and PC+1 always land in different heads).+ registers one head, one row per register in its own column block, read all at once.+ positions none: the write index j comes from a head whose row halves at every VAL token+ and resets at STEP/BOOT (x = 2^-j), and "latest STEP/TICK" style lookups are+ rows overwritten at those tokens.++All arithmetic is on bits, one-hot nibbles and small integers, so it is exact in bf16.+"""++from fractions import Fraction++from chipformer import spec+from chipformer.dsl.graph import Expr+from chipformer.q35 import bridge as B+from chipformer.q35.ir import Chan, Target+from chipformer.q35.lib import bcd as bios+from chipformer.q35.prims import Circ, cat, from_bits, onehot_bits++JMAX = 64 # write indices above this all read as JMAX + 1+NCODE = 15 # output code: [is_addr, is_ctrl, payload bits 0..12]+CODE_AMP = 128+KAPPA = 2**-20 # output code after the exit: tiny, so it never disturbs Qwen's norms+TAU_TEXT = 2.0**-40+V, VF = spec.V, spec.V + 0xF+TEXT_CODE = 16 # text classes are ctrl codes with payload 16 + class (STEP/TICK/BOOT: 0..2)+++def token_code(t: int) -> list[int]:+ """0/1 code of a token; the embedding stores CODE_AMP * (2 * bit - 1)."""+ if B.is_text(t):+ kind, payload = (0, 1), TEXT_CODE + t - B.TEXT_BASE+ elif spec.is_val(t):+ kind, payload = (0, 0), t - spec.VAL_BASE+ elif spec.is_addr(t):+ kind, payload = (1, 0), t - spec.ADDR_BASE+ else:+ kind, payload = (0, 1), t - spec.STEP+ return [*kind] + [payload >> i & 1 for i in range(13)]+++def decode_code(bits: list[int]) -> int:+ payload = sum(b << i for i, b in enumerate(bits[2:]))+ if bits[1]:+ return B.TEXT_BASE + payload - TEXT_CODE if payload >= TEXT_CODE else spec.STEP + payload+ return (spec.ADDR_BASE if bits[0] else spec.VAL_BASE) + payload+++class Chip8Q35(Circ):+ def __init__(self, target: Target | None = None) -> None:+ super().__init__(target or Target())+ f = self.token_feature+ isv, isa = spec.is_val, spec.is_addr+ self.is_val = f("is_val", lambda t: int(isv(t)))+ self.is_addr = f("is_addr", lambda t: int(isa(t)))+ self.is_step = f("is_step", lambda t: int(t == spec.STEP))+ self.is_tick = f("is_tick", lambda t: int(t == spec.TICK))+ self.is_boot = f("is_boot", lambda t: int(t == spec.BOOT))+ self.vb = [f(f"vb{i}", lambda t, i=i: (t - spec.VAL_BASE) >> i & 1 if isv(t) else 0)+ for i in range(12)]+ self.an = [[f(f"a{k}[{x}]",+ lambda t, k=k, x=x: int(isa(t) and (t - spec.ADDR_BASE) >> 4 * k & 15 == x))+ for x in range(16)] for k in range(3)]+ self.a12 = f("a12", lambda t: int(isa(t) and (t - spec.ADDR_BASE) >> 12 & 1))+ self.is_cls_addr = f("is_cls_addr", lambda t: int(t == spec.addr_token(spec.CLS)))+ self.is_text = f("is_text", lambda t: int(B.is_text(t)))+ self.mu = f("mu", lambda t: int(not B.is_text(t))) # chip token: Qwen's layers mute on it+ self.text = {cls: f(f"text{cls - B.TEXT_BASE}", lambda t, cls=cls: int(t == cls))+ for cls in range(B.TEXT_BASE, B.TEXT_BASE + B.N_TEXT)}+ self.code = [f(f"code{i}", lambda t, i=i: 0 if t == B.OTHER else+ CODE_AMP * (2 * token_code(t)[i] - 1)) for i in range(NCODE)]+ # tau: -1 on chip tokens (cancelled by the last layer), a tiny value on text tokens.+ # The final norm amplifies it, pushing the chip tokens' logits far down wherever Qwen+ # speaks.+ self.tau = f("tau", lambda t: TAU_TEXT if B.is_text(t) else -1)+ self.rs = self.is_step + self.is_boot # record start++ def addr_lits(self, a: int) -> list:+ """0/1 literals that are all 1 exactly at an ADDR a token (one-hot, no constants)."""+ lits = [self.an[k][a >> 4 * k & 15] for k in range(3)]+ return lits + [self.a12] if a >> 12 else lits++ def write_key(self, a: int) -> Chan:+ """Key pre-activation (AND units) for a write to cell a, seen at its VAL token."""+ lits = self.addr_lits(a)+ if not a >> 12:+ lits = lits + [self.one - self.a12]+ return Chan(sum(lits, Expr()) - (len(lits) - 1), (0, 1, 0, 0))++ def bank(self, name: str, cells: dict[int, int], query: dict, nbits: int, a=0) -> list:+ """One head of trace-written cells (cell -> slot); returns the read bits."""+ keys = {s: self.write_key(c) for c, s in cells.items()}+ data = {2 + i: self.vb[i] for i in range(nbits)}+ return self.slot_head(name, keys, query, data, self.one, a)+++def build(target: Target | None = None) -> Chip8Q35:+ c = Chip8Q35(target)+ one, rs = c.one, c.rs++ # ------------------------------------------------------------ registers (one head)+ regs = [("PC", spec.PC, 12), ("I", spec.I, 12), ("SP", spec.SP, 4), ("DT", spec.DT, 8),+ ("ST", spec.ST, 8), ("RNG", spec.RNG, 12), ("V0", V, 8)]+ regs += [(f"key{k:X}", spec.KEYS + k, 1) for k in range(16)]+ regs += [(f"q.{q}", spec.QUIRKS + i, 1) for i, q in enumerate(spec.QUIRK_NAMES)]+ regs += [("LA", bios.LEN, 5), ("LB", bios.LEN + 1, 5)] # bignum argument lengths+ regs += [("MODE", bios.MODE, 4)] # inline call, inside "$", inside "$$", negative+ keys, data, col, blocks = {127: Chan(c.is_boot)}, {}, 2, {}+ for slot, (name, cell, nb) in enumerate(regs):+ keys[slot] = c.write_key(cell)+ lits = c.addr_lits(cell)+ blocks[name] = list(range(col, col + nb))+ for i in range(nb):+ # the VAL bit (this token) AND the address (previous token), over both conv taps+ data[col] = Chan(c.vb[i] + sum(lits, Expr()) - one * Fraction(len(lits), 2), (1, 1, 0, 0))+ col += 1+ out = c.slot_head("regs", keys, {s: one for s in keys}, data, Chan(c.is_boot))+ by_col = dict(zip(sorted(data), out))+ R = {name: [by_col[k] for k in cols] for name, cols in blocks.items()}+ pcb, ib, spb, dtb, stb, rngb, v0b = (R[k] for k in ("PC", "I", "SP", "DT", "ST", "RNG", "V0"))+ keyb = [R[f"key{k:X}"][0] for k in range(16)]+ lab, lbb = R["LA"], R["LB"]+ mode, mode_d, mode_dd, mode_neg = R["MODE"]+ q = {name: R[f"q.{name}"][0] for name in spec.QUIRK_NAMES}++ # ------------------------------------------------------------ micro-step counter+ # j = VAL tokens (inside a record) or text tokens (inside a print) since the record start;+ # a CALL record starts at the </tool_call> token+ tco, tcc = c.text[B.TC_OPEN], c.text[B.TC_CLOSE]+ rsc = rs + tcc+ (x,) = c.slot_head("counter", {0: rsc, 1: one - rsc}, {0: one - rsc, 1: one - rsc},+ {2: rsc}, one - rsc, 110 * (c.is_val + c.is_text - tcc) + 238 * rsc,+ tiny=2.0**-90)+ cnt = [c.node([(1, one - rsc, 7 * one - 8 * 2**m * x),+ (-1, one - rsc, 6 * one - 8 * 2**m * x)], f"j>{m}")+ for m in range(JMAX + 1)] # cnt[m] = [j > m]+ j = sum(cnt, Expr())++ def jeq(v): # [j == v], a 0/1 linear form+ return (one if v == 0 else cnt[v - 1]) - (cnt[v] if v <= JMAX else 0)++ (in_tick,) = c.slot_head("in_tick", {0: rs + c.is_tick}, {0: one}, {2: c.is_tick},+ rs + c.is_tick)++ # ------------------------------------------------------------ argument capture+ # Tool calls: inside <tool_call> ... </tool_call> the circuit keeps the routine's name and+ # the digits of each argument (runs of digit tokens, in order) for the CALL record.+ # Inline: outside tool calls it keeps the same for "A op B" since the last breaker token, and+ # answers at "=" (bridge.InlineScan is the reference).+ T = c.text+ is_dig = sum((T[B.HEX[v]] for v in range(10)), Expr())+ dvb = [sum((T[B.HEX[v]] for v in range(10) if v >> i & 1), Expr()) for i in range(4)]+ fns = {op: T[cls] for op, cls in B.FUNCS.items()}+ fn_any = sum(fns.values(), Expr())+ (in_call,) = c.slot_head("in_call", {0: tco + tcc + rs}, {0: one}, {2: tco}, tco + tcc + rs)+ dollars = sum((T[d_] for d_ in B.DOLLARS), Expr())+ safe = (T[B.SPACE] + T[B.NL] + T[B.WS] + dollars + T[B.LPAREN]+ + c.mu) # may precede a first number+ open1, open2 = T[B.DOLLAR] + T[B.SDOLLAR], T[B.DDOLLAR] + T[B.SDDOLLAR] # "$", "$$"+ prev_cols = {2: Chan(is_dig, (0, 1, 0, 0)), 3: Chan(is_dig, (0, 0, 1, 0)),+ 4: Chan(T[B.COMMA], (0, 1, 0, 0)), 5: Chan(T[B.BSL], (0, 1, 0, 0)),+ 6: Chan(safe, (0, 1, 0, 0)), 7: Chan(c.is_text, (0, 1, 0, 0)),+ 8: Chan(T[B.EQ], (0, 1, 0, 0)), 9: Chan(open1, (0, 1, 0, 0)),+ 10: Chan(open2, (0, 1, 0, 0)), 11: Chan(open1, (0, 0, 1, 0)),+ 12: Chan(open2, (0, 0, 1, 0)), 13: Chan(T[B.SPACE], (0, 1, 0, 0))}+ (prev_dig, prev2_dig, prev_cm, prev_bs, prev_safe, prev_text, prev_eq, prev_o1, prev_o2,+ prev2_o1, prev2_o2, prev_sp) = c.slot_head("prev", {0: one}, {0: one}, prev_cols, one)+ # the math a first number sits in: right after "$" / "$$", or after it and a space+ in_o1 = c.node(c.t_and([prev_o1]) + c.t_and([prev_sp, prev2_o1]), "open.$")+ in_o2 = c.node(c.t_and([prev_o2]) + c.t_and([prev_sp, prev2_o2]), "open.$$")+ cont = c.node(c.t_and([is_dig, prev_dig]) + c.t_and([is_dig, prev_cm, prev2_dig]), "arg.cont")+ run_start = c.node(c.t_and([is_dig, one - prev_dig, one - prev_cm])+ + c.t_and([is_dig, prev_cm, one - prev2_dig]), "arg.start")+ tw_op = c.and_([T[B.TIMESW], prev_bs], "times.op") # \\times, \\cdot+ dv_op = c.and_([T[B.FN_DIV], prev_bs], "div.op") # \\div+ inl = {"add": T[B.PLUS], "sub": T[B.MINUS] + T[B.MINUS2], "mul": T[B.TIMES] + tw_op,+ "div": T[B.DIV] + dv_op}+ op_tok = sum(inl.values(), Expr())+ comma_join = c.and_([T[B.COMMA], prev_dig], "comma.join")+ neutral = (is_dig + T[B.SPACE] + dollars + T[B.LPAREN] + T[B.BSL] + T[B.EQ] + op_tok+ + comma_join)+ brk = one - neutral # chip tokens included+ # the window also starts afresh right after "=" (so "x^2 = A op B =" still counts)+ after_eq = [prev_eq, one - brk, one - in_call]+ reset = c.node(c.t_and([tco]) + c.t_and([brk - tco, one - in_call]) + c.t_and(after_eq),+ "arg.reset")+ op_reset = c.node(c.t_and([brk]) + c.t_and(after_eq), "op.reset")+ # a digit or operator right after "=" is reset, not counted (the decay must be exactly 238)+ both_d = c.and_([is_dig, prev_eq, one - in_call], "after_eq.digit")+ both_o = c.and_([op_tok, prev_eq, one - in_call], "after_eq.op")+ (xp,) = c.slot_head("arg.count", {0: reset, 1: one - reset}, {0: one - reset, 1: one - reset},+ {2: reset}, one - reset, 110 * (run_start - both_d) + 238 * reset,+ tiny=2.0**-90)+ (xd,) = c.slot_head("digit.count", {0: run_start, 1: one - run_start},+ {0: one - run_start, 1: one - run_start}, {2: run_start},+ one - run_start, 110 * cont + 238 * run_start, tiny=2.0**-90)+ (xo,) = c.slot_head("op.count", {0: op_reset, 1: one - op_reset},+ {0: one - op_reset, 1: one - op_reset}, {2: op_reset}, one - op_reset,+ 110 * (op_tok - both_o) + 238 * op_reset, tiny=2.0**-90)++ def thermo(xv, gate, n, name): # [count > m] for m < n, from x = 2^-count+ return [c.node([(1, one - gate, 7 * one - 8 * 2**m * xv),+ (-1, one - gate, 6 * one - 8 * 2**m * xv)], f"{name}>{m}")+ for m in range(n)]++ cp = thermo(xp, reset, 3, "p") # p = argument number (1, 2)+ cd = thermo(xd, run_start, B.MAX_ARG + 1, "d") # d = digit index within the argument+ co = thermo(xo, op_reset, 2, "ops") # operators since the last breaker++ def peq(v):+ return cp[v - 1] - cp[v]++ def deq(w):+ return (one if w == 0 else cd[w - 1]) - cd[w]++ # lengths and the routine: slots 0/1 = length of argument 1/2 (rewritten at every digit),+ # slot 2 = the routine (at its name token, or the inline operator) and whether it came right+ # after the first argument, slot 127 = ballast+ len_bits = [sum((deq(w) for w in range(B.MAX_ARG) if (w + 1) >> i & 1), Expr())+ for i in range(5)]+ data = {}+ for a_ in (1, 2):+ for i in range(5):+ data[2 + 5 * (a_ - 1) + i] = is_dig + peq(a_) + len_bits[i] - 2+ ops = list(B.FUNCS)+ for i, op in enumerate(ops):+ data[12 + i] = fns[op] + inl[op] - (dv_op if op == "div" else 0) # "\div" counts once+ at_fn = c.node(c.t_and([fn_any, in_call]) + c.t_and([op_tok, one - in_call]), "op.at")+ data[12 + len(ops)] = at_fn + peq(1) - 1+ # <tool_call> clears the routine (a row of zeros), so a tool call needs a name inside it: an+ # inline operator in earlier text must not make the template's "<tool_call></tool_call>" fire+ lk = {0: is_dig + peq(1) - 1, 1: is_dig + peq(2) - 1, 2: at_fn + tco,+ 127: brk - fn_any - tco}+ out = c.slot_head("call.args", lk, {0: one, 1: one, 2: one, 127: one}, data,+ brk - fn_any - tco)+ la_bits, lb_bits, op_oh = out[0:5], out[5:10], out[10:10 + len(ops)]+ op_ok = out[10 + len(ops)]+ a_safe, a_o1, a_o2 = c.slot_head("a.safe", {0: run_start + peq(1) - 1}, {0: one},+ {2: prev_safe, 3: in_o1, 4: in_o2}, run_start + peq(1) - 1)+ la_, lb_ = from_bits(la_bits), from_bits(lb_bits)+ trig = c.and_([T[B.EQ], one - in_call, peq(2), co[0] - co[1], op_ok, a_safe,+ c.node(c.t_ge(la_, 1, 0, 31), "la>0"), c.node(c.t_ge(lb_, 1, 0, 31), "lb>0"),+ c.node(cat(c.t_ge(la_ + lb_, B.INLINE_MIN_DIGITS[op], 0, 62,+ lits=[op_oh[ops.index(op)]]) for op in ops), "inline.big"),+ sum(op_oh, Expr())], "inline.trig")+ # 1 from an inline trigger until the next text token: the mode of the CALL record+ (inline_mode,) = c.slot_head("inline.mode", {0: c.is_text}, {0: one}, {2: trig}, c.is_text)++ # ------------------------------------------------------------ display bytes ("old")+ disp_q = {}+ old = [Expr() for _ in range(8)]+ for h in range(2):+ cells = {spec.DISPLAY + (h << 7) + s: s for s in range(128)}+ query = {s: sum(c.addr_lits(cell), Expr()) - 3 for cell, s in cells.items()}+ disp_q[h] = query+ bits = c.bank(f"display{h}", cells, query, 8, 238 * c.is_cls_addr)+ old = [o + b for o, b in zip(old, bits)]++ # ------------------------------------------------------------ PC one-hots, opcode fetch+ oh = [c.onehot_of_bits(pcb[4 * k:4 * k + 4], f"pc.n{k}") for k in range(3)]+ c4 = pcb[0:4]+ c8 = pcb[0:8]++ def nib_lits(bits, v):+ return [b if v >> i & 1 else one - b for i, b in enumerate(bits)]++ def carry_onehot(bits, carry_bits, name):+ out = []+ for v in range(16):+ here, below = nib_lits(bits, v), nib_lits(bits, (v - 1) % 16)+ terms = (c.t_and(here) + c.t_and(here + carry_bits, -1)+ + c.t_and(below + carry_bits))+ out.append(c.node(terms, f"{name}[{v}]"))+ return out++ oh1_pc1 = carry_onehot(pcb[4:8], c4, "pc1.n1")+ oh2_pc1 = carry_onehot(pcb[8:12], c8, "pc1.n2")+ oh0_pc1 = [oh[0][(v - 1) % 16] for v in range(16)]+ even, odd = [Expr() for _ in range(8)], [Expr() for _ in range(8)]+ for half in range(2):+ for h in range(16):+ n1, n2 = (oh1_pc1, oh2_pc1) if h == 0 else (oh[1], oh[2])+ cells = {(half << 11) | (s >> 4 << 8) | ((s & 15) << 4) | h: s for s in range(128)}+ query = {s: oh[0][h] + oh0_pc1[h] + n1[s & 15] + n2[(s >> 4) + 8 * half] - 2+ for s in range(128)}+ bits = c.bank(f"ramA{half}.{h}", cells, query, 8)+ acc = even if h % 2 == 0 else odd+ for i in range(8):+ acc[i] = acc[i] + bits[i]+ pc0 = pcb[0]+ b1 = [c.node(c.t_and([pc0, odd[i]]) + c.t_and([one - pc0, even[i]]), f"op_hi.b{i}")+ for i in range(8)]+ b2 = [c.node(c.t_and([pc0, even[i]]) + c.t_and([one - pc0, odd[i]]), f"op_lo.b{i}")+ for i in range(8)]++ # ------------------------------------------------------------ decode+ hib, xb, yb, nb = b1[4:], b1[:4], b2[4:], b2[:4]+ ind = {}+ for h, name in [(1, "1NNN"), (2, "2NNN"), (3, "3XNN"), (4, "4XNN"), (6, "6XNN"),+ (7, "7XNN"), (0xA, "ANNN"), (0xB, "BNNN"), (0xC, "CXNN"), (0xD, "DXYN")]:+ ind[name] = c.and_(nib_lits(hib, h), name)+ ind["00E0"] = c.and_(nib_lits(b1[:4], 0) + nib_lits(b1[4:], 0) + nib_lits(b2[:4], 0)+ + nib_lits(b2[4:], 0xE), "00E0")+ ind["00FD"] = c.and_(nib_lits(b1[:4], 0) + nib_lits(b1[4:], 0) + nib_lits(b2[:4], 0xD)+ + nib_lits(b2[4:], 0xF), "00FD") # print and halt (bridge.py)+ ind["00EE"] = c.and_(nib_lits(b1[:4], 0) + nib_lits(b1[4:], 0) + nib_lits(b2[:4], 0xE)+ + nib_lits(b2[4:], 0xE), "00EE")+ for h, name in [(5, "5XY0"), (9, "9XY0")]:+ ind[name] = c.and_(nib_lits(hib, h) + nib_lits(nb, 0), name)+ for m, name in [(1, "BADD"), (2, "BZERO"), (3, "BMUL"), (4, "BSUB"), (5, "BDIV"),+ (6, "BADB"), (7, "BRSUB"), (8, "BTRIM")]: # lib/macros.py+ ind[name] = c.and_(nib_lits(hib, 5) + nib_lits(nb, m), name)+ for m in [0, 1, 2, 3, 4, 5, 6, 7, 0xE]:+ ind[f"8XY{m:X}"] = c.and_(nib_lits(hib, 8) + nib_lits(nb, m), f"8XY{m:X}")+ for h, code in [(0xE, 0x9E), (0xE, 0xA1), (0xF, 0x07), (0xF, 0x0A), (0xF, 0x15),+ (0xF, 0x18), (0xF, 0x1E), (0xF, 0x29), (0xF, 0x33), (0xF, 0x55), (0xF, 0x65)]:+ name = f"{h:X}X{code:02X}"+ ind[name] = c.and_(nib_lits(hib, h) + nib_lits(b2[:4], code & 15)+ + nib_lits(b2[4:], code >> 4), name)+ x_oh = c.onehot_of_bits(xb, "x")+ y_oh = c.onehot_of_bits(yb, "y")+ n_oh = c.onehot_of_bits(nb, "n")++ # ------------------------------------------------------------ V registers, stack+ vcells = {V + r: r for r in range(16)}+ vxb = c.bank("VX", vcells, {r: x_oh[r] for r in range(16)}, 8)+ vyb = c.bank("VY", vcells, {r: y_oh[r] for r in range(16)}, 8)+ sp_oh = c.onehot_of_bits(spb, "sp")+ stack_top = c.bank("stack", {spec.STACK + r: r for r in range(16)},+ {r: sp_oh[(r + 1) % 16] for r in range(16)}, 12)++ # ------------------------------------------------------------ ALU+ vx, vy, nn = from_bits(vxb), from_bits(vyb), from_bits(b2)+ def lo4(bits):+ return from_bits(bits[:4])+++ def nibbles8(s0, s0_lo, s0_hi, z, z_lo, z_hi, name):+ """Bits of z mod 256, given s0 == z (mod 16) over a small range."""+ r0 = []+ for v in range(16):+ terms = []+ for k in range(-2, 3):+ terms += c.t_eq(s0, v + 16 * k, s0_lo, s0_hi)+ r0.append(c.node(terms, f"{name}.n0[{v}]"))+ off = -(z_lo // 256) * 256 if z_lo < 0 else 0+ r1 = c.nib_onehot(z + off, z_lo + off, z_hi + off, f"{name}.n1", div=16)+ return onehot_bits(r0, 4) + onehot_bits(r1, 4)++ add_xy = nibbles8(lo4(vxb) + lo4(vyb), 0, 30, vx + vy, 0, 510, "vx+vy")+ add_xn = nibbles8(lo4(vxb) + lo4(b2), 0, 30, vx + nn, 0, 510, "vx+nn")+ sub_xy = nibbles8(lo4(vxb) - lo4(vyb), -15, 15, vx - vy, -255, 255, "vx-vy")+ sub_yx = nibbles8(lo4(vyb) - lo4(vxb), -15, 15, vy - vx, -255, 255, "vy-vx")+ carry = c.node(c.t_ge(vx + vy, 256, 0, 510), "carry")+ ge_xy = c.node(c.t_ge(vx - vy, 0, -255, 255), "vx>=vy")+ ge_yx = c.node(c.t_ge(vy - vx, 0, -255, 255), "vy>=vx")+ eq_x_nn = c.node(c.t_eq(vx - nn, 0, -255, 255), "vx==nn")+ eq_x_y = c.node(c.t_eq(vx - vy, 0, -255, 255), "vx==vy")+ and_xy = [c.and_([xa, ya], f"vx&vy.b{i}") for i, (xa, ya) in enumerate(zip(vxb, vyb))]+ or_xy = [xa + ya - a for xa, ya, a in zip(vxb, vyb, and_xy)]+ xor_xy = [xa + ya - 2 * a for xa, ya, a in zip(vxb, vyb, and_xy)]+ src = [c.node(c.t_and([q["shift"], xa]) + c.t_and([one - q["shift"], ya]), f"src.b{i}")+ for i, (xa, ya) in enumerate(zip(vxb, vyb))]+ shr, shl = src[1:] + [0], [0] + src[:7]++ # BCD of VX+ hund = c.node(c.t_ge(vx, 100, 0, 255) + c.t_ge(vx, 200, 0, 255), "vx/100")+ tens_all = c.node(cat(c.t_ge(vx, 10 * k, 0, 255) for k in range(1, 26)), "vx/10")+ ones_d = vx - 10 * tens_all+ tens_d = tens_all - 10 * hund++ def small_bits(z, hi, name):+ oh_ = [c.node(c.t_eq(z, v, 0, hi), f"{name}=={v}") for v in range(hi + 1)]+ return onehot_bits(oh_, 4)++ bcd = [small_bits(hund, 2, "hund"), small_bits(tens_d, 9, "tens"),+ small_bits(ones_d, 9, "ones")]++ # keys+ def key_is(k): # literals for VX & 0xF == k+ return nib_lits(vxb[:4], k)++ key_vx = c.node(cat(c.t_and(key_is(k) + [keyb[k]]) for k in range(16)), "key[vx]")+ none_held = [one - kb for kb in keyb]+ fx0a_val = []+ for i in range(8):+ terms = [t for k in range(16) if k >> i & 1+ for t in c.t_and([keyb[k]] + none_held[:k])]+ terms += c.t_and(none_held + [vxb[i]])+ fx0a_val.append(c.node(terms, f"fx0a.b{i}"))+ any_key = c.node(c.t_const(1) + c.t_and(none_held, -1), "any_key")++ # CXNN: one Galois LFSR step, then the low byte AND NN+ shifted = rngb[1:] + [0]+ rng_next, rnd = [], []+ for i in range(12):+ if spec.RNG_MASK >> i & 1:+ a_, b_ = shifted[i], rngb[0]+ if isinstance(a_, int):+ bit = b_+ else:+ bit = c.node(c.t_and([a_, one - b_]) + c.t_and([one - a_, b_]), f"lfsr{i}")+ rng_next.append(bit)+ if i < 8:+ rnd.append(c.node(c.t_and([b2[i], a_, one - b_]) + c.t_and([b2[i], one - a_, b_])+ if not isinstance(a_, int) else c.t_and([b2[i], b_]),+ f"rnd.b{i}"))+ else:+ rng_next.append(shifted[i])+ if i < 8:+ rnd.append(c.and_([b2[i], shifted[i]], f"rnd.b{i}"))++ # FX29: I = FONT + 5 * (VX & 0xF)+ font_i = []+ for i in range(12):+ terms = [t for v in range(16) if (spec.FONT_BASE + 5 * v) >> i & 1+ for t in c.t_and(key_is(v))]+ font_i.append(c.node(terms, f"font.b{i}"))++ def dec8(bits, name):+ """Bits of max(v - 1, 0)."""+ out = []+ for i in range(len(bits)):+ neg = [one - b for b in bits[:i]] # x - 1 flips bit i iff all lower bits are 0+ terms = [(1, one, bits[i])] + (c.t_and(neg) if neg else c.t_const(1))+ terms += c.t_and([bits[i], *neg], -2) + c.t_and([one - b for b in bits], -1)+ out.append(c.node(terms, f"{name}.b{i}"))+ return out++ dt_dec, st_dec = dec8(dtb, "dt-1"), dec8(stb, "st-1")+ sp_dec = onehot_bits([sp_oh[(v + 1) % 16] for v in range(16)], 4)+ sp_inc = onehot_bits([sp_oh[(v - 1) % 16] for v in range(16)], 4)++ # 12-bit sums+ ihi = c.onehot_of_bits(ib[8:12], "I.n2")+ i_lo = from_bits(ib[:8])++ def bits12(nibs):+ return [b for n_ in nibs for b in onehot_bits(n_, 4)]++ pc_lo = from_bits(pcb[:8])+ pc2 = bits12(c.add12(pc_lo, oh[2], 2, 2, "pc+2"))+ pc4 = bits12(c.add12(pc_lo, oh[2], 4, 4, "pc+4"))+ i_vx = bits12(c.add12(i_lo, ihi, vx, 255, "I+vx"))+ i_p = [ib, bits12(c.add12(i_lo, ihi, 1, 1, "I+1")), bits12(c.add12(i_lo, ihi, 2, 2, "I+2"))]+ xint = from_bits(xb)+ i_next = bits12(c.add12(i_lo, ihi, xint + 1, 16, "I+x+1"))+ jump_reg = [c.node(c.t_and([q["jump"], xa]) + c.t_and([one - q["jump"], va]), f"jreg.b{i}")+ for i, (xa, va) in enumerate(zip(vxb, v0b))]+ bnnn = bits12(c.add12(nn, x_oh, from_bits(jump_reg), 255, "bnnn"))+ nnn = list(b2) + list(xb)++ # (before the BTRIM nodes: the first FFN sublayer is nearly full and placement is greedy)+ la_i, lb_i = from_bits(lab), from_bits(lbb)+ mx = c.node([(1, one, la_i), (1, one, lb_i - la_i)], "max(LA,LB)") # la + relu(lb - la)+ # BTRIM: is mem[I] a leading 0 of the result (I inside it, V0 >= 2)? Read at decode from a+ # bank of the result cells queried by I, so it is a plain conditional instruction.+ # (the queries are AND-forms of I's bits, no FFN units)+ r_cells = 2 * bios.MAX_DIGITS+ r_end = bios.R_ADDR + r_cells+ assert bios.R_ADDR >> 8 == (r_end - 1) >> 8+ trim_q = {v: sum(nib_lits(ib[0:4], (bios.R_ADDR + v) & 15)+ + nib_lits(ib[4:8], (bios.R_ADDR + v) >> 4 & 15)+ + nib_lits(ib[8:12], (bios.R_ADDR + v) >> 8), Expr()) - 11+ for v in range(r_cells)}+ trim_d = c.bank("trim.R", {bios.R_ADDR + v: v for v in range(r_cells)}, trim_q, 4)+ trim_in = c.node(c.t_in(from_bits(ib[:8]), bios.R_ADDR & 255, (r_end - 1 & 255) + 1, 0, 255,+ lits=nib_lits(ib[8:12], bios.R_ADDR >> 8)), "trim.in")+ v0_ge2 = c.node(c.t_ge(from_bits(v0b), 2, 0, 255), "v0>=2")+ trim_z = c.and_([one - b_ for b_ in trim_d] + [v0_ge2, trim_in], "trim.z")++ # ------------------------------------------------------------ fixed writes+ vxa = list(xb) + [0] * 8 + [1] # V + x: 0x1000 | x+ stack_sp = list(spb) + [0, 1, 0, 0, 0, 0, 0, 0, 1] # 0x1020 | sp+ i_a = [list(b) + [0] for b in i_p]+ fixed = [+ ([ind["00E0"]], [(spec.CLS, 0)]),+ ([ind["00EE"]], [(spec.SP, sp_dec)]),+ ([ind["2NNN"]], [(stack_sp, pc2), (spec.SP, sp_inc)]),+ ([ind["6XNN"]], [(vxa, list(b2))]),+ ([ind["7XNN"]], [(vxa, add_xn)]),+ ([ind["8XY0"]], [(vxa, list(vyb))]),+ ([ind["ANNN"]], [(spec.I, nnn)]),+ ([ind["CXNN"]], [(vxa, rnd), (spec.RNG, rng_next)]),+ ([ind["FX07"]], [(vxa, list(dtb))]),+ ([ind["FX0A"]], [(vxa, fx0a_val)]),+ ([ind["FX15"]], [(spec.DT, list(vxb))]),+ ([ind["FX18"]], [(spec.ST, list(vxb))]),+ ([ind["FX1E"]], [(spec.I, i_vx)]),+ ([ind["FX29"]], [(spec.I, font_i)]),+ ([ind["FX33"]], [(i_a[0], bcd[0]), (i_a[1], bcd[1]), (i_a[2], bcd[2])]),+ ([ind["BTRIM"], trim_z], [(V, dec8(v0b, "v0-1")), (spec.I, i_p[1])]),+ ]+ for op, result in [("8XY1", or_xy), ("8XY2", and_xy), ("8XY3", xor_xy)]:+ fixed.append(([ind[op], one - q["logic"]], [(vxa, result)]))+ fixed.append(([ind[op], q["logic"]], [(vxa, result), (VF, 0)]))+ for op, result, flag in [("8XY4", add_xy, carry), ("8XY5", sub_xy, ge_xy),+ ("8XY6", shr, src[0]), ("8XY7", sub_yx, ge_yx),+ ("8XYE", shl, src[7])]:+ fixed.append(([ind[op], one - q["vforder"]], [(vxa, result), (VF, [flag])]))+ fixed.append(([ind[op], q["vforder"]], [(VF, [flag]), (vxa, result)]))+ fa = [c.select_bits([(lits, w[m][0]) for lits, w in fixed if len(w) > m], 13, f"A{m}")+ for m in range(3)]+ fv = [c.select_bits([(lits, w[m][1]) for lits, w in fixed if len(w) > m], 12, f"W{m}")+ for m in range(3)]+ has = [c.node(cat(c.t_and(lits) for lits, w in fixed if len(w) > m), f"has{m}")+ for m in range(3)]+ is_vec = ind["FX55"] + ind["FX65"] + ind["DXYN"]+ ls_extra = c.node(c.t_and([ind["FX55"], one - q["loadstore"]])+ + c.t_and([ind["FX65"], one - q["loadstore"]]), "ls_extra")++ # BMUL: VX = rows done (row LB - 1 - VX); 5X13 is the first row (R read as 0: no BZERO),+ # the last row ends the loop+ vx5 = from_bits(vxb[:5])+ mul_first = c.and_(nib_lits(yb, 1), "mul.first")+ mul_last = c.node(c.t_eq(vx5 + 1 - lb_i, 0, -31, 32), "mul.last")+ vx_inc = [one - vxb[0]] + [c.node(c.t_and([vxb[i]]) + c.t_and(vxb[:i])+ + c.t_and([vxb[i], *vxb[:i]], -2), f"vx+1.b{i}")+ for i in range(1, 5)]+ nw_oh = []+ NW = 2 * bios.MAX_DIGITS + 2 # writes before PC: up to LA + LB + 1 for BZERO+ for v in range(1, NW + 1):+ terms = [t for lits, w in fixed if len(w) == v for t in c.t_and(lits)]+ for op in ("FX55", "FX65"):+ if 0 <= v - 1 < 16:+ terms += c.t_and([ind[op], q["loadstore"], *nib_lits(xb, v - 1)])+ if 0 <= v - 2 < 16:+ terms += c.t_and([ind[op], one - q["loadstore"], *nib_lits(xb, v - 2)])+ if v % 2 == 1 and (v - 1) // 2 < 16:+ terms += c.t_and([ind["DXYN"], *nib_lits(nb, (v - 1) // 2)])+ # n = max + 1 digits, then V0 and I (BRSUB: and MODE)+ terms += c.t_eq(mx, v - 3, 0, 31, lits=[ind["BADD"] + ind["BSUB"]])+ terms += c.t_eq(mx, v - 4, 0, 31, lits=[ind["BRSUB"]])+ terms += c.t_eq(la_i + lb_i, v - 1, 0, 62, lits=[ind["BZERO"]])+ # LA digits, the carry, VX (the last row: and V0, I)+ terms += c.t_eq(la_i, v - 2, 0, 31, lits=[ind["BMUL"], one - mul_last])+ terms += c.t_eq(la_i, v - 4, 0, 31, lits=[ind["BMUL"], mul_last])+ # the quotient digit, LB digits and the window's top digit, VX, VF+ terms += c.t_eq(lb_i, v - 4, 0, 31, lits=[ind["BDIV"] + ind["BADB"]])+ nw_oh.append(c.node(terms, f"nw=={v}"))+ nw_oh = [one - sum(nw_oh, Expr())] + nw_oh++ skip = c.node(c.t_and([ind["3XNN"], eq_x_nn]) + c.t_and([ind["4XNN"], one - eq_x_nn])+ + c.t_and([ind["5XY0"], eq_x_y]) + c.t_and([ind["9XY0"], one - eq_x_y])+ + c.t_and([ind["EX9E"], key_vx]) + c.t_and([ind["EXA1"], one - key_vx]),+ "skip")+ wait = c.node(c.t_and([ind["FX0A"], one - any_key]), "wait")+ jump = ind["1NNN"] + ind["2NNN"]+ # macros that repeat (PC stays) and macros that jump to T = 0x200 + 2 * XY (BSUB: unless+ # it borrows, decided at its PC write)+ stay = c.node(c.t_and([ind["BTRIM"], trim_z])+ + c.t_and([ind["BMUL"], one - mul_first, one - mul_last]), "macro.stay")+ mul_skip = c.and_([ind["BMUL"], mul_first, mul_last], "mul.skip") # one row: past the loop+ to_t = ind["BADD"] + ind["BSUB"] + ind["BRSUB"]+ tgt = [0] + list(yb) + list(xb) + [1, 0, 0]+ default = one - skip - jump - ind["00EE"] - ind["BNNN"] - wait - stay - to_t - mul_skip+ next_pc = c.select_bits([([skip + mul_skip], pc4), ([jump], nnn), ([ind["00EE"]], stack_top),+ ([ind["BNNN"]], bnnn), ([wait + stay], list(pcb)), ([to_t], tgt),+ ([default], pc2)],+ 12, "next_pc")++ # ------------------------------------------------------------ context copy+ ctx_vals = {+ "x": x_oh, "n": n_oh, "I": list(ib), "Ihi": ihi, "nw": nw_oh, "next_pc": next_pc,+ "fixed": [one - is_vec], "has": has, "A0": fa[0], "A1": fa[1], "A2": fa[2],+ "W0": fv[0], "W1": fv[1], "W2": fv[2], "fx55": [ind["FX55"]], "fx65": [ind["FX65"]],+ "dxyn": [ind["DXYN"]], "dx_byte": vxb[3:6], "dx_off": vxb[0:3], "dy": vyb[0:5],+ "dt_dec": dt_dec, "st_dec": st_dec, "ls_extra": [ls_extra], "i_next": i_next,+ "clip": [q["clip"]], "fd": [ind["00FD"]], "v0": v0b,+ "badd": [ind["BADD"]], "bzero": [ind["BZERO"]], "bmul": [ind["BMUL"]],+ "bsub": [ind["BSUB"]], "brsub": [ind["BRSUB"]], "pc2": pc2,+ "la": lab, "lb": lbb, "vx": vxb[:5], "first": [mul_first], "last": [mul_last],+ "vx1": vx_inc, "stay": [stay], "bdiv": [ind["BDIV"]], "badb": [ind["BADB"]],+ "t": vxb[:6], "pc": list(pcb),+ }+ # the macros' digit chain is the deepest path: its entries get a head of their own, so no+ # later entry holds them up+ fast = ("la", "lb", "vx", "badd", "bsub", "brsub", "bmul", "bzero", "first", "bdiv", "badb",+ "t")+ groups = [[(name, i, e) for name in fast for i, e in enumerate(ctx_vals[name])],+ [(name, i, e) for name, vals in ctx_vals.items() if name not in fast+ for i, e in enumerate(vals)]]+ k = {name: [None] * len(vals) for name, vals in ctx_vals.items()}+ per_head = 126+ chunks = [g[hd:hd + per_head] for g in groups for hd in range(0, len(g), per_head)]+ for h, chunk in enumerate(chunks):+ data = {2 + i: e for i, (_, _, e) in enumerate(chunk)}+ outs = c.slot_head(f"ctx{h}", {0: rs}, {0: one}, data, rs)+ for (name, i, _), o in zip(chunk, outs):+ k[name][i] = o++ # ------------------------------------------------------------ micro-step+ at_val, at_addr = c.is_addr, one - c.is_addr+ not_tick = one - in_tick+ is_pc = c.node(cat(c.t_and([k["nw"][v], jeq(v)]) for v in range(NW + 1)), "j==nw")+ is_end = c.node(cat(c.t_and([k["nw"][v], jeq(v + 1)]) for v in range(NW + 1)),+ "j==nw+1")+ le_x = c.node(cat(c.t_and([k["x"][v], one - cnt[v]]) for v in range(16)), "j<=x")+ lt_2n = c.node(cat(c.t_and([k["n"][v], one - cnt[2 * v - 1]]) for v in range(1, 16)),+ "j<2n")+ eq_2n = c.node(cat(c.t_and([k["n"][v], jeq(2 * v)]) for v in range(16)), "j==2n")+ is_i_adv = c.node(cat(c.t_and([k["x"][v], jeq(v + 1)]) for v in range(16)), "j==x+1")++ row_i = sum((cnt[2 * r - 1] for r in range(1, JMAX // 2 + 1)), Expr()) # j // 2+ side = sum((jeq(v) for v in range(1, JMAX + 1, 2)), Expr()) # j % 2+ kI_lo = from_bits(k["I"][:8])+ i_j = bits12(c.add12(kI_lo, k["Ihi"], j, JMAX + 1, "I+j"))+ # vread address: I + j (FX65), I + j // 2 (DXYN) or I + j - 3 (the digits of a print)+ d = c.node([(1, k["fx65"][0], j), (1, k["dxyn"][0], row_i),+ (1, k["fd"][0], j - 3 * one + 2 * mode)], "vec_off")+ qa = c.add12(kI_lo, k["Ihi"], d, JMAX + 1, "vread_addr")+ vread = [Expr() for _ in range(8)]+ ram_read = k["fx65"][0] + k["dxyn"][0] + k["fd"][0]+ for half in range(2):+ for h in range(16):+ cells = {(half << 11) | (s >> 4 << 8) | ((s & 15) << 4) | h: s for s in range(128)}+ query = {s: qa[0][h] + qa[1][s & 15] + qa[2][(s >> 4) + 8 * half] + ram_read - 3+ for s in range(128)}+ bits = c.bank(f"ramB{half}.{h}", cells, query, 8)+ vread = [a_ + b_ for a_, b_ in zip(vread, bits)]+ vj = c.bank("V[j]", vcells, {r: jeq(r) + k["fx55"][0] - 1 for r in range(16)}, 8)+ vread = [a_ + b_ for a_, b_ in zip(vread, vj)]++ # DXYN: write j targets display byte (dy + j//2, dx_byte + j%2)+ dy = from_bits(k["dy"])+ dxb = from_bits(k["dx_byte"])+ row = dy + row_i+ col_ = dxb + side+ row_w = [c.node(c.t_eq(row, r, 0, 31 + 20) + c.t_eq(row, r + 32, 0, 51), f"row_w=={r}")+ for r in range(32)]+ col_w = [c.node(c.t_eq(col_, v, 0, 8) + c.t_eq(col_, v + 8, 0, 8), f"col_w=={v}")+ for v in range(8)]+ col8 = list(k["dx_byte"]) + [side] # col == 8 exactly when dx_byte == 7 and side == 1+ on_screen = c.node(c.t_lt(row, 32, 0, 51) + c.t_lt(row, 32, 0, 51, -1, lits=col8),+ "on_screen")+ drawn = c.node(c.t_const(1) + c.t_and([k["clip"][0]], -1)+ + c.t_and([k["clip"][0], on_screen]), "drawn")+ idx_bits = onehot_bits(col_w, 3) + onehot_bits(row_w, 5)+ target = c.select_bits([([drawn], idx_bits + [1, 0, 0, 0, 1]),+ ([one - drawn], spec.NULL)], 13, "target")+ off_bits = k["dx_off"]+ part = []+ for p in range(8):+ terms = []+ for kk in range(16):+ o, right = kk // 2, kk % 2+ s_ = p - 8 + o if right else p + o+ if 0 <= s_ < 8:+ lits = nib_lits(off_bits, o)[:3] + [side if right else one - side, vread[s_]]+ terms += c.t_and(lits)+ part.append(c.node(terms, f"part{p}"))+ overlap = [c.and_([old[p], part[p]], f"overlap{p}") for p in range(8)]+ new_byte = [old[p] + part[p] - 2 * overlap[p] for p in range(8)]+ draw_row = [c.is_addr, not_tick, k["dxyn"][0], lt_2n]+ collided = c.node(c.t_ge(sum(overlap, Expr()), 1, 0, 8, lits=draw_row), "collided")+ (any_collision,) = c.slot_head("collision", {0: rs + collided}, {0: one}, {2: collided},+ rs + collided)++ # ------------------------------------------------------------ output+ A = [] # (literals, kind, payload) with kind 0 = VAL, 1 = ADDR, 2 = STEP++ def case(lits, kind, payload):+ A.append((lits, kind, payload))++ # ---------------------------------------------------------------- CALL record+ # At </tool_call> (with a known routine) the circuit writes the arguments' digits to RAM,+ # their lengths, then PC = the routine, then STEP; the machine then runs the routine.+ op_valid = sum(op_oh, Expr())+ # a tool call starts its record at </tool_call>; an inline one predicts STEP at "=" and starts+ # at that STEP (so j can stay a plain token-feature counter)+ call_val = c.node(c.t_and([tcc, op_valid]) + c.t_and([rs, inline_mode, prev_text]), "call.v")+ (calling,) = c.slot_head("calling", {0: tcc + rs + trig}, {0: one}, {2: call_val + trig},+ tcc + rs + trig)+ neq = one - T[B.EQ]+ la, lb = from_bits(la_bits), from_bits(lb_bits)+ za, zz = j - la, j - la - lb+ in_a = c.node(c.t_lt(za, 0, -B.MAX_ARG, JMAX + 1), "call.in_a")+ in_b = c.node(c.t_ge(za, 0, -B.MAX_ARG, JMAX + 1)+ + c.t_ge(zz, 0, -2 * B.MAX_ARG, JMAX + 1, -1), "call.in_b")+ ez = [c.node(c.t_eq(zz, v, -2 * B.MAX_ARG, JMAX + 1), f"call.e{v}") for v in range(5)]+ q_a = [c.and_([jeq(w), in_a], f"call.qa{w}") for w in range(B.MAX_ARG)]+ q_b = [c.node(c.t_eq(za, w, -B.MAX_ARG, JMAX + 1, lits=[in_b]), f"call.qb{w}")+ for w in range(B.MAX_ARG)]+ keys = {32 * (a_ - 1) + w: is_dig + peq(a_) + deq(w) - 2+ for a_ in (1, 2) for w in range(B.MAX_ARG)}+ query = {**{w: q_a[w] for w in range(B.MAX_ARG)},+ **{32 + w: q_b[w] for w in range(B.MAX_ARG)}}+ digit_bits = c.slot_head("call.digits", keys, query, {2 + i: dvb[i] for i in range(4)}, one)+ pad = [0] * (32 - B.MAX_ARG)+ a_addr = onehot_bits(q_a + pad, 5)+ b_addr = onehot_bits(q_b + pad, 5)+ op_div = op_oh[ops.index("div")] # div's A goes to W + 1 (bcd.call_writes)+ a_addr = [(bios.A_ADDR >> i & 1) * (one - op_div) + ((bios.W_ADDR + 1) >> i & 1) * op_div+ if i >= 5 else a_addr[i] for i in range(13)]+ b_addr = [bios.B_ADDR >> i & 1 if i >= 5 else b_addr[i] for i in range(13)]+ entry = [sum((op_oh[k_] for k_, op in enumerate(ops) if bios.OPS[op] >> i & 1), Expr())+ for i in range(12)]+ case([trig], 2, 0)+ case([at_addr, calling, neq, in_a], 1, a_addr)+ case([at_val, calling, in_a], 0, digit_bits)+ case([at_addr, calling, neq, in_b], 1, b_addr)+ case([at_val, calling, in_b], 0, digit_bits)+ case([at_addr, calling, neq, ez[0]], 1, bios.LEN)+ case([at_val, calling, ez[0]], 0, la_bits)+ case([at_addr, calling, neq, ez[1]], 1, bios.LEN + 1)+ case([at_val, calling, ez[1]], 0, lb_bits)+ case([at_addr, calling, neq, ez[2]], 1, bios.MODE)+ mode_bits = [inline_mode, c.and_([inline_mode, a_o1 + a_o2], "mode.$"),+ c.and_([inline_mode, a_o2], "mode.$$")]+ case([at_val, calling, ez[2]], 0, mode_bits)+ case([at_addr, calling, neq, ez[3]], 1, spec.PC)+ case([at_val, calling, ez[3]], 0, entry)+ case([at_addr, calling, neq, ez[4]], 2, 0)++ # ---------------------------------------------------------------- bignum macros+ MD = bios.MAX_DIGITS+ kb_add, kb_zero, kb_mac, kb_sub = k["badd"][0], k["bzero"][0], k["bmul"][0], k["bsub"][0]+ kb_rsub = k["brsub"][0]+ kb_as = kb_add + kb_sub + kb_rsub # BADD, BSUB and BRSUB share their layout+ kla, klb = from_bits(k["la"]), from_bits(k["lb"])+ krow = klb - 1 - from_bits(k["vx"]) # BMUL's row+ wi = j - c.is_val # index of the write this token belongs to+ kmx = c.node([(1, one, kla), (1, one, klb - kla)], "k.max")+ ia, ib = kla - 1 - wi, klb - 1 - wi # digit indices, counted from the right end++ def eqs(z, lo, hi, n, name, lits=()): # one-hot of an integer z over 0..n-1+ return [c.node(c.t_eq(z, v, lo, hi, lits=list(lits)), f"{name}=={v}") for v in range(n)]++ lo_, hi_ = -72, 72 # bounds of index arithmetic, garbage states included (5-bit fields)+ qa = eqs(ia, lo_, hi_, MD, "mac.ia", [kb_as + kb_mac])+ qb = [c.node(c.t_eq(ib, v, lo_, hi_, lits=[kb_as]) + c.t_eq(krow, v, -32, 31, lits=[kb_mac])+ + c.t_eq(klb - wi, v, lo_, hi_, lits=[k["bdiv"][0] + k["badb"][0]]),+ f"mac.ib=={v}") for v in range(MD)]+ kk2 = kla + krow - wi # BMUL target digit i + row + 1+ qr = eqs(kk2, lo_, hi_, 2 * MD, "mac.k", [kb_mac, one - k["first"][0]]) # first row: 0+ # long division (BDIV / BADB): t = VX, the window W[t-LB .. t], quotient digit at P[pq]+ kb_div, kb_adb = k["bdiv"][0], k["badb"][0]+ kb_dv = kb_div + kb_adb+ kt = from_bits(k["t"])+ NWC = 2 + MD + bios.DEC # W: a 0, A, the decimals' zeros+ gt_la = c.node(c.t_ge(kt - kla, 1, -31, 63), "div.t>la")+ pq = kt + gt_la # P index of the quotient digit+ # writes: 0 = the quotient digit P[pq], 1 .. LB+1 = the window (from the right; LB+1 is its+ # top digit), LB+2 = VX, LB+3 = VF+ dv_p = c.and_([kb_dv, jeq(0)], "div.w0")+ in_dv = c.node(c.t_lt(j - klb, 2, lo_, hi_, lits=[kb_dv, cnt[0]]), "div.digit")+ rel = j - klb+ dv_at = [c.node(c.t_eq(rel, r_, lo_, hi_, lits=[kb_dv]), f"div.w{r_}") for r_ in (2, 3)]+ qw = eqs(kt - wi + 1, -64, 65, NWC, "div.kw", [kb_dv])++ def digit_bank(name, base, n, query):+ cells = {base + i: i for i in range(n)}+ return c.bank(name, cells, {i: query[i] for i in range(n)}, 4)++ a_bits = digit_bank("bank.A", bios.A_ADDR, MD, qa)+ b_bits = digit_bank("bank.B", bios.B_ADDR, MD, qb)+ r_bits = digit_bank("bank.R", bios.R_ADDR, 2 * MD, qr)+ w_bits = digit_bank("bank.W", bios.W_ADDR, NWC, qw)+ a_i, b_i, r_i, w_i = from_bits(a_bits), from_bits(b_bits), from_bits(r_bits), from_bits(w_bits)+ p_i = from_bits(digit_bank("bank.P", bios.R_ADDR, 2 * MD,+ eqs(pq, 0, 64, 2 * MD, "div.pqr", [kb_adb]))) # BADB: P - 1++ # BDIV's quotient digit, at its first ADDR token (nothing is written yet): q = #{k in 1..9 :+ # R3 + delta >= k * B2} (lib/macros.div_estimate). R3 + delta - k*B2 reaches 999, beyond+ # bf16's exact integers, so it is split as 10 * H + L (H = 10 W0 + W1 - k b0,+ # L = W2 + delta - k b1, both small): [10 H + L >= 0] = sum over c of [ceil(-L/10) == c] [H >= c]+ wq = [from_bits(digit_bank(f"bank.Wq{c_}", bios.W_ADDR, NWC,+ eqs(kt - klb + c_, -64, 64, NWC, f"div.q{c_}", [kb_div])))+ for c_ in range(3)]+ b0 = from_bits(digit_bank("bank.B0", bios.B_ADDR, 1, {0: kb_div}))+ b1 = from_bits(digit_bank("bank.B1", bios.B_ADDR + 1, 1, {0: kb_div}))+ lb1 = c.node(c.t_eq(klb, 1, 0, 31), "div.lb1")+ lb3 = c.node(c.t_ge(klb, 3, 0, 31), "div.lb3")+ th = [one]+ for k_ in range(1, 10):+ h_ = 10 * wq[0] + wq[1] - k_ * b0 # -81 .. 99+ l_ = wq[2] + lb3 - k_ * b1 # -81 .. 10+ ge_h = {cc: c.node(c.t_ge(h_, cc, -81, 99), f"div.H{k_}>={cc}") for cc in range(-1, 10)}+ l_is = {cc: c.node(c.t_in(l_, -10 * cc, -10 * cc + 10, -81, 10), f"div.L{k_}~{cc}")+ for cc in range(-1, 10)}+ th.append(c.node(cat(c.t_and([l_is[cc], ge_h[cc], one - lb1]) for cc in range(-1, 10))+ + c.t_and([ge_h[0], lb1]), f"div.q>={k_}"))+ th.append(Expr())+ q_new = onehot_bits([th[v] - th[v + 1] for v in range(10)], 4)+ # the record's first write carries q in its VAL token, so the window's writes read it from+ # the token stream (the estimate itself is too deep to feed their carry chain)+ at_q = c.and_([c.is_val, kb_div, jeq(1)], "div.at_q")+ q_bits = c.slot_head("div.q", {0: at_q}, {0: one}, {2 + i: c.vb[i] for i in range(4)}, at_q)++ # Carries. A carry can only reach the next write through the token stream, so each VAL+ # token reconstructs the carry its write produced from the digit it wrote, d:+ # BADD: carry = [a + b >= d + 9]; BMUL: carry = s // 10 + [d < s % 10], s = R[k] + a*b.+ # R[k] is overwritten at that VAL token, so its ADDR token keeps s // 10 and s % 10 in a+ # register (as one-hots) for the VAL token to read.+ prod = c.node(cat(c.t_and([a_bits[p_], b_bits[q_]], 1 << (p_ + q_))+ for p_ in range(4) for q_ in range(4)), "a*b")+ s_mac = r_i + prod # <= 90 (<= 240 in garbage states)+ # BADB is BMAC's carry chain with s = W[k] + b. BDIV subtracts: W[k] - q*b - borrow, kept as+ # g = q*b - W[k] + 9 (0 .. 90) so that borrow' = g // 10 + [d >= 10 - g % 10].+ qprod = c.node(cat(c.t_and([q_bits[p_], b_bits[q_]], 1 << (p_ + q_))+ for p_ in range(4) for q_ in range(4)), "q*b")+ s_adb = w_i + b_i # <= 18 (<= 30)+ g_div = qprod - w_i + 9 # 0 .. 90 (-6 .. 234)+ fl_oh = [c.node(c.t_in(s_mac, 10 * v, 10 * v + 10, 0, 240, lits=[kb_mac])+ + c.t_in(s_adb, 10 * v, 10 * v + 10, 0, 30, lits=[kb_adb])+ + c.t_in(g_div, 10 * v, 10 * v + 10, -15, 240, lits=[kb_div]),+ f"s//10=={v}") for v in range(10)]+ sm_oh = [c.node(cat(c.t_eq(s_mac, v + 10 * q_, 0, 240, lits=[kb_mac]) for q_ in range(10))+ + cat(c.t_eq(s_adb, v + 10 * q_, 0, 30, lits=[kb_adb]) for q_ in range(4))+ + cat(c.t_eq(g_div, v + 10 * q_, -15, 240, lits=[kb_div])+ for q_ in range(10)), f"s%10=={v}")+ for v in range(10)]+ at_mac = c.and_([c.is_addr, kb_mac + kb_dv], "mac.at")+ keep = c.slot_head("mac.s", {0: at_mac}, {0: one},+ {2 + i: e for i, e in enumerate(fl_oh + sm_oh)}, at_mac)+ fl_r, sm_r = keep[:10], keep[10:]+ tok_d = from_bits(c.vb[:4]) # the digit this VAL token wrote+ d_lt = [c.node(c.t_lt(tok_d, v, 0, 15), f"d<{v}") for v in range(10)]+ wrapped = c.node(cat(c.t_and([sm_r[v], d_lt[v]]) for v in range(1, 10)), "mac.wrap")+ cout_mac = sum((v * fl_r[v] for v in range(10)), Expr()) + wrapped # <= 9+ cout_add = c.node(c.t_ge(a_i + b_i - tok_d, 9, -15, 30), "add.cout")+ cout_sub = c.node(c.t_ge(tok_d - a_i + b_i, 1, -15, 30), "sub.borrow") # [d > a - b]+ cout_rsub = c.node(c.t_ge(tok_d - b_i + a_i, 1, -15, 30), "rsub.borrow") # [d > b - a]+ wrap_dv = c.node(cat(c.t_and([sm_r[v], one - d_lt[10 - v]]) for v in range(1, 10)),+ "div.wrap") # [d >= 10 - g % 10]+ cout_div = sum((v * fl_r[v] for v in range(10)), Expr()) + wrap_dv # <= 9+ cm_oh = eqs(cout_mac, 0, 10, 10, "mac.cout")+ cd_oh = eqs(cout_div, 0, 10, 10, "div.cout")+ cout_bits = [c.node(c.t_and([c.is_val, kb_mac + kb_adb, b_])+ + c.t_and([c.is_val, kb_div, bd_])+ + (c.t_and([c.is_val, kb_add, cout_add])+ + c.t_and([c.is_val, kb_sub, cout_sub])+ + c.t_and([c.is_val, kb_rsub, cout_rsub]) if i == 0 else []),+ f"cout.b{i}")+ for i, (b_, bd_) in enumerate(zip(onehot_bits(cm_oh, 4), onehot_bits(cd_oh, 4)))]+ macro = kb_add + kb_mac + kb_sub + kb_rsub + kb_dv+ # only digit writes pass the carry on, so it survives the V0 / I / VX writes after them+ # (BSUB's PC write reads the final borrow)+ # (at a VAL token wi = j - 1; written so, is_val is not in both the value and the literals,+ # whose weights would add up to a non-bf16 257)+ # (BADD / BSUB / BRSUB: not their top digit either, so the carry into it stays: it is R[0])+ dig_w = c.node(c.t_lt(j - 1 - kmx, 0, lo_, hi_, lits=[c.is_val, kb_as])+ + c.t_lt(j - 1 - kla, 0, lo_, hi_, lits=[c.is_val, kb_mac])+ # division: the window's writes (wi = 1 .. LB+1), the last one carrying BDIV's+ # "negative" / BADB's "carried out"+ + c.t_lt(j - klb, 3, lo_, hi_, lits=[c.is_val, kb_dv, cnt[1]]),+ "macro.digit_w")+ dv_win = dig_w # (only the division's terms are live in its records)+ carry = c.slot_head("carry", {0: rs + dig_w + macro - 1}, {0: one},+ {2 + i: cout_bits[i] for i in range(4)}, rs + dig_w + macro - 1)+ c_in = from_bits(carry)++ # BADD digit: (a + b + carry) mod 10; BMAC digit: (R + a*b + carry) mod 10+ t_add = a_i + b_i + c_in # <= 19 (<= 45)+ t_mac = s_mac + c_in # <= 99 (<= 255)+ t_sub = a_i - b_i - c_in # -10 .. 9 (-30 .. 15)+ t_rsub = b_i - a_i - c_in+ t_adb = s_adb + c_in # <= 27 (<= 45)+ t_div = 9 - g_div - c_in # W[k] - q*b - borrow: -99 .. 9 (-246 .. 24)+ dsum = [c.node(c.t_eq(t_add, v, 0, 45, lits=[kb_add]) + c.t_eq(t_add, v + 10, 0, 45,+ lits=[kb_add])+ + cat(c.t_eq(t_mac, v + 10 * q_, 0, 255, lits=[kb_mac]) for q_ in range(10))+ + c.t_eq(t_sub, v, -30, 15, lits=[kb_sub])+ + c.t_eq(t_sub, v - 10, -30, 15, lits=[kb_sub])+ + c.t_eq(t_rsub, v, -30, 15, lits=[kb_rsub])+ + c.t_eq(t_rsub, v - 10, -30, 15, lits=[kb_rsub])+ + cat(c.t_eq(t_adb, v + 10 * q_, 0, 45, lits=[kb_adb]) for q_ in range(3))+ + cat(c.t_eq(t_div, v - 10 * q_, -246, 24, lits=[kb_div]) for q_ in range(10)),+ f"mac.d=={v}") for v in range(10)]+ digit_val = onehot_bits(dsum, 4)++ def addr_bits(oh, base): # bits of base + index for a one-hot index+ return [sum((e for i, e in enumerate(oh) if (base + i) >> b_ & 1), Expr())+ for b_ in range(13)]++ # what a token predicts is about the next write, index j (reads above use wi)+ kk_add = eqs(kmx - j, lo_, hi_, MD + 1, "add.k", [kb_as]) # n - 1 - w+ kk_mac = eqs(kla + krow - j, lo_, hi_, 2 * MD, "mac.kw", [kb_mac])+ zero_k = eqs(j, 0, JMAX + 1, 2 * MD, "zero.k", [kb_zero])+ row_oh = eqs(krow, -32, 31, MD, "mac.row", [kb_mac])+ # V0 = n - [R[0] == 0] = max + carry into the top digit, I = R_ADDR + 1 - carry+ n_add = eqs(kmx + c_in, 0, 47, MD + 2, "add.n", [kb_as])+ i_add = [one - carry[0]] + [bios.R_ADDR >> b_ & 1 for b_ in range(1, 12)]+ n_zero = eqs(kla + klb, 0, 62, 2 * MD + 1, "zero.n", [kb_zero])+ in_add = c.node(c.t_lt(j - kmx, 1, lo_, hi_, lits=[kb_as]), "add.digit")+ end_add = c.node(c.t_eq(j - kmx, 1, lo_, hi_, lits=[kb_as]), "add.end") # V0 = n+ end_add2 = c.node(c.t_eq(j - kmx, 2, lo_, hi_, lits=[kb_as]), "add.end2") # I = R+ end_rsub = c.node(c.t_eq(j - kmx, 3, lo_, hi_, lits=[kb_rsub]), "rsub.end") # MODE |= 8+ in_zero = c.node(c.t_lt(j - kla - klb, 0, lo_, hi_, lits=[kb_zero]), "zero.digit")+ end_zero = c.node(c.t_eq(j - kla - klb, 0, lo_, hi_, lits=[kb_zero]), "zero.end")+ in_mac = c.node(c.t_lt(j - kla, 0, lo_, hi_, lits=[kb_mac]), "mac.digit")+ end_mac = c.node(c.t_eq(j - kla, 0, lo_, hi_, lits=[kb_mac]), "mac.end") # R[row]+ mul_vx = c.node(c.t_eq(j - kla, 1, lo_, hi_, lits=[kb_mac]), "mul.vx") # VX += 1+ mul_v0 = c.node(c.t_eq(j - kla, 2, lo_, hi_, lits=[kb_mac, k["last"][0]]), "mul.v0")+ mul_i = c.node(c.t_eq(j - kla, 3, lo_, hi_, lits=[kb_mac, k["last"][0]]), "mul.i")+ # the last row's R[0] = carry is its own VAL token: keep [it is 0] for V0 and I+ mul_endv = c.node(c.t_eq(j - 1 - kla, 0, lo_, hi_, lits=[c.is_val, kb_mac]), "mul.endv")+ (mul_z,) = c.slot_head("mul.top0", {0: rs + mul_endv}, {0: one},+ {2: c.and_([one - b_ for b_ in c.vb[:4]], "val==0")}, rs + mul_endv)+ n_mul = eqs(kla + klb - mul_z, -1, 62, 2 * MD + 1, "mul.n", [kb_mac])+ i_mul = [mul_z] + [bios.R_ADDR >> b_ & 1 for b_ in range(1, 12)]+ v7 = spec.V + 7++ # machine records (not inside a print or a CALL record)+ nfd = one - k["fd"][0] - calling+ for m in range(3):+ case([at_val, nfd, not_tick, k["fixed"][0], jeq(m), k["has"][m]], 0, k[f"W{m}"])+ case([at_addr, nfd, not_tick, k["fixed"][0], jeq(m), k["has"][m]], 1, k[f"A{m}"])+ jbits = onehot_bits([jeq(v) for v in range(16)], 4)+ case([at_val, nfd, not_tick, k["fx55"][0], le_x], 0, vread)+ case([at_addr, nfd, not_tick, k["fx55"][0], le_x], 1, i_j)+ case([at_val, nfd, not_tick, k["fx65"][0], le_x], 0, vread)+ case([at_addr, nfd, not_tick, k["fx65"][0], le_x], 1, jbits + [0] * 8 + [1])+ case([at_val, nfd, not_tick, k["ls_extra"][0], is_i_adv], 0, k["i_next"])+ case([at_addr, nfd, not_tick, k["ls_extra"][0], is_i_adv], 1, spec.I)+ case([at_val, nfd, not_tick, k["dxyn"][0], lt_2n, drawn], 0, new_byte)+ case([at_addr, nfd, not_tick, k["dxyn"][0], lt_2n], 1, target)+ case([at_val, nfd, not_tick, k["dxyn"][0], eq_2n], 0, [any_collision])+ case([at_addr, nfd, not_tick, k["dxyn"][0], eq_2n], 1, VF)+ case([at_val, nfd, not_tick, is_pc, one - kb_sub - kb_dv], 0, k["next_pc"])+ case([at_val, nfd, not_tick, is_pc, kb_sub, one - carry[0]], 0, k["next_pc"]) # T+ case([at_val, nfd, not_tick, is_pc, kb_sub, carry[0]], 0, k["pc2"]) # borrowed: on+ case([at_addr, nfd, not_tick, is_pc, one - k["stay"][0]], 1, spec.PC)+ case([at_addr, nfd, not_tick, is_pc, k["stay"][0]], 2, 0) # PC stays: no PC write+ case([at_addr, nfd, not_tick, is_end], 2, 0)+ case([at_val, nfd, in_tick, jeq(0)], 0, k["dt_dec"])+ case([at_addr, nfd, in_tick, jeq(0)], 1, spec.DT)+ case([at_val, nfd, in_tick, jeq(1)], 0, k["st_dec"])+ case([at_addr, nfd, in_tick, jeq(1)], 1, spec.ST)+ case([at_addr, nfd, in_tick, jeq(2)], 2, 0)+ case([at_addr, nfd, in_add], 1, addr_bits(kk_add, bios.R_ADDR))+ case([at_val, nfd, in_add], 0, digit_val)+ case([at_addr, nfd, end_add], 1, V)+ case([at_val, nfd, end_add, one - kb_rsub], 0, onehot_bits(n_add, 6))+ case([at_val, nfd, end_add, kb_rsub, mode], 0, onehot_bits(n_add, 6)) # tool call: 0+ case([at_addr, nfd, end_add2], 1, spec.I)+ case([at_val, nfd, end_add2], 0, i_add)+ case([at_addr, nfd, end_rsub], 1, bios.MODE)+ case([at_val, nfd, end_rsub], 0, [mode, mode_d, mode_dd, 1])+ case([at_addr, nfd, in_zero], 1, addr_bits(zero_k, bios.R_ADDR))+ case([at_addr, nfd, end_zero], 1, v7)+ case([at_val, nfd, end_zero], 0, onehot_bits(n_zero, 6))+ case([at_addr, nfd, in_mac], 1, addr_bits(kk_mac, bios.R_ADDR))+ case([at_val, nfd, in_mac], 0, digit_val)+ case([at_addr, nfd, end_mac], 1, addr_bits(row_oh, bios.R_ADDR))+ case([at_val, nfd, end_mac], 0, list(carry))+ case([at_addr, nfd, mul_vx], 1, onehot_bits(k["x"], 4) + [0] * 8 + [1]) # V + x+ case([at_val, nfd, mul_vx], 0, k["vx1"])+ case([at_addr, nfd, mul_v0], 1, V)+ case([at_val, nfd, mul_v0], 0, onehot_bits(n_mul, 6))+ case([at_addr, nfd, mul_i], 1, spec.I)+ case([at_val, nfd, mul_i], 0, i_mul)++ # long division records: the window (LB digits from the right, then its top digit), the+ # quotient digit, VX, VF, then PC (back to the same instruction while it goes on)+ kw_oh = eqs(kt - j + 1, -64, 65, NWC, "div.kwj", [kb_dv])+ case([at_addr, nfd, in_dv], 1, addr_bits(kw_oh, bios.W_ADDR))+ case([at_val, nfd, in_dv], 0, digit_val)+ pq_oh = eqs(pq, 0, 64, 2 * MD, "div.pq", [kb_dv])+ case([at_addr, nfd, dv_p], 1, addr_bits(pq_oh, bios.R_ADDR))+ pm1 = [c.node(c.t_eq(p_i, v + 1, 0, 15), f"P-1=={v}") for v in range(15)]+ p_dec = [sum((e for v, e in enumerate(pm1) if v >> b_ & 1), Expr()) + one - sum(pm1, Expr())+ for b_ in range(8)] # P - 1 (0 - 1 = 255)+ case([at_val, nfd, dv_p, kb_div], 0, q_new)+ case([at_val, nfd, dv_p, kb_adb], 0, p_dec)+ case([at_addr, nfd, dv_at[0]], 1, onehot_bits(k["x"], 4) + [0] * 8 + [1])+ # flags, after the window: its last carry is BDIV's "negative" / BADB's "carried out", and a+ # counter of the window's nonzero digits says whether the remainder is 0+ nz_tok = c.and_([c.is_val, dv_win, one - d_lt[1]], "div.nz_tok")+ (xz,) = c.slot_head("div.nz", {0: rs, 1: one - rs}, {0: one - rs, 1: one - rs}, {2: rs},+ one - rs, 110 * nz_tok + 238 * rs, tiny=2.0**-90)+ dnz = c.node([(1, one - rs, 7 * one - 8 * xz), (-1, one - rs, 6 * one - 8 * xz)], "div.nz")+ ge_la = c.node(c.t_ge(kt - kla, 0, -31, 63), "div.t>=la")+ at_end = c.node(c.t_eq(kt - kla, bios.DEC, -31, 63), "div.t=end")+ neg = carry[0] # 0/1 everywhere (c_in is 0/1 only here)+ f1 = c.node(c.t_and([kb_div, neg]) + c.t_and([kb_adb, one - neg]), "div.f1")+ f2 = c.node(c.t_and([kb_div, one - neg, one - dnz, ge_la])+ + c.t_and([kb_adb, neg, one - dnz, ge_la]), "div.f2")+ f3 = c.node(c.t_and([kb_div, one - neg, dnz, at_end])+ + c.t_and([kb_adb, neg, dnz, at_end]), "div.f3")+ f0 = one - f1 - f2 - f3+ t_oh = eqs(kt, 0, 64, 64, "div.t", [kb_dv])+ t1_oh = eqs(kt + 1, 0, 65, 64, "div.t+1", [kb_dv])+ case([at_val, nfd, dv_at[0], f0], 0, onehot_bits(t1_oh, 8))+ case([at_val, nfd, dv_at[0], one - f0], 0, onehot_bits(t_oh, 8))+ case([at_addr, nfd, dv_at[1]], 1, spec.V + 15)+ case([at_val, nfd, dv_at[1]], 0, [f1 + f3, f2 + f3])+ case([at_val, nfd, not_tick, is_pc, kb_div, f0], 0, k["pc"])+ case([at_val, nfd, not_tick, is_pc, kb_div, one - f0], 0, k["next_pc"])+ case([at_val, nfd, not_tick, is_pc, kb_adb, f1], 0, k["pc"])+ case([at_val, nfd, not_tick, is_pc, kb_adb, one - f1], 0, k["next_pc"])+++ # a print: STEP, then NL <tool_response> NL digits NL </tool_response> NL as text tokens.+ # Position j (STEP = 0, then each printed token) predicts printed token j; n = min(V0, 58).+ v0 = from_bits(k["v0"])+ z = j - v0+ small = c.node(c.t_lt(v0, B.MAX_PRINT + 1, 0, 255), "print.small")+ big = one - small+ lim = B.MAX_PRINT++ def zj(dz, name): # [j == n + dz]+ return c.node(c.t_eq(z, dz, -255, JMAX + 1, lits=[small])+ + c.t_and([big, jeq(lim + dz)]), name)++ nl_end, trc, nl_last = zj(3, "print.nl1"), zj(4, "print.trc"), zj(5, "print.nl2")+ # tool call: [3 <= j < 3 + n] = [j >= 3] - [j >= 3 + n]; inline: [1 <= j < 1 + n]+ tool = one - mode+ digit = c.node(c.t_and([cnt[2], tool]) + c.t_ge(z, 3, -255, JMAX + 1, -1, lits=[small, tool])+ + c.t_and([big, tool, cnt[lim + 2]], -1)+ + c.t_and([cnt[0], mode]) + c.t_ge(z, 1, -255, JMAX + 1, -1, lits=[small, mode])+ + c.t_and([big, mode, cnt[lim]], -1), "print.digit")+ nz = c.node(c.t_ge(v0, 1, 0, 255), "print.nonempty")+ upto = c.node(c.t_lt(z, 6, -255, JMAX + 1, lits=[small, tool])+ + c.t_and([big, tool, one - cnt[lim + 5]])+ + c.t_lt(z, 2, -255, JMAX + 1, lits=[small, mode, nz])+ + c.t_lt(z, 1, -255, JMAX + 1, lits=[small, mode, one - nz])+ + c.t_and([big, mode, one - cnt[lim + 1]]), "print.upto")+ # inline: a space after the digits, so Qwen does not read on into the number+ trail = c.node(c.t_eq(z, 1, -255, JMAX + 1, lits=[small, mode, nz])+ + c.t_and([big, mode, jeq(lim + 1)]), "print.trail")+ in_print = [k["fd"][0], c.is_step + c.is_text, one - calling]+ code = {cls: TEXT_CODE + cls - B.TEXT_BASE+ for cls in (B.NL, B.TR_OPEN, B.TR_CLOSE, B.SPACE, B.MINUS, B.PERIOD, B.DOLLAR,+ B.DDOLLAR)}+ for lits, cls in [([tool, jeq(0)], B.NL), ([tool, jeq(1)], B.TR_OPEN), ([tool, jeq(2)], B.NL),+ ([tool, nl_end], B.NL), ([tool, trc], B.TR_CLOSE), ([tool, nl_last], B.NL),+ ([mode, jeq(0), one - mode_neg], B.SPACE),+ ([mode, jeq(0), mode_neg], B.MINUS), ([trail, one - mode_d], B.PERIOD),+ ([trail, mode_d, one - mode_dd], B.DOLLAR), ([trail, mode_dd], B.DDOLLAR)]:+ case(in_print + lits, 2, code[cls])+ # TEXT_CODE + low nibble; 10 and 11 (a quotient's "." and "...") print as their classes+ p10, p11 = (c.and_(nib_lits(vread[:4], v), f"print.is{v}") for v in (10, 11))+ case(in_print + [digit, one - p10 - p11], 2, list(vread[:4]) + [1])+ case(in_print + [digit, p10], 2, TEXT_CODE + B.PERIOD - B.TEXT_BASE)+ case(in_print + [digit, p11], 2, TEXT_CODE + B.ELLIPSIS - B.TEXT_BASE)+ # who predicts the next token: the circuit on chip tokens and inside a print, else Qwen+ owner = c.node(c.t_and([c.mu]) + c.t_and([c.is_text, k["fd"][0], cnt[0], upto,+ one - calling])+ + c.t_and([c.is_text, calling]), "owner")++ nxt = [c.node([t for lits, kind, _ in A if kind == 1 for t in c.t_and(lits)], "next.is_addr"),+ c.node([t for lits, kind, _ in A if kind == 2 for t in c.t_and(lits)], "next.is_ctrl")]+ nxt += c.select_bits([(lits, p) for lits, _, p in A], 13, "next")+ c.next_bits = nxt+ c.owner = owner+ c.cases = A # (literals, kind, payload), for debugging+ # ---------------------------------------------------------- exit (the last chip sublayer)+ # The emitter clears every other chip dim here, so after the chip layers a token holds only+ # its Qwen part, mu, the output code (KAPPA * +-1 where the circuit owns the position, else+ # 0) and tau (0 on chip tokens, the text value on text tokens).+ for i, e in enumerate(c.code): # the input code goes first, so the exit adds onto 0+ c.ffn([(-1, one, e), (1, one, -1 * e)], f"code.clear{i}", into=e)+ top = max(n.level for n in c.nodes)+ for i, e in enumerate(c.code):+ c.ffn(c.t_and([nxt[i], owner], 2 * KAPPA) + c.t_and([owner], -KAPPA),+ f"exit.code{i}", into=e, min_level=top)+ # tau: 0 wherever the circuit predicts (chip tokens: -1 + 1; owned text: TAU_TEXT - TAU_TEXT),+ # TAU_TEXT where Qwen does, which keeps chip tokens out of Qwen's text+ c.ffn([(1, c.mu, one)] + c.t_and([owner, c.is_text], -TAU_TEXT), "exit.tau", into=c.tau,+ min_level=top)+ c.keep = [next(iter(e.terms)) for e in c.code] + [next(iter(c.tau.terms)),+ next(iter(c.mu.terms))]+ c.out = {"regs": R, "x": x, "cnt": cnt, "in_tick": in_tick, "old": old, "b1": b1, "b2": b2,+ "ind": ind, "vx": vxb, "vy": vyb, "ctx": k, "vread": vread, "next": nxt}+ return cdiff --git a/experimental/riscv/chipformer/q35/demo.py b/experimental/riscv/chipformer/q35/demo.pynew file mode 100644index 0000000..c264dc7--- /dev/null+++ b/experimental/riscv/chipformer/q35/demo.py@@ -0,0 +1,140 @@+"""Qwen uses the computer in its weights: exact arithmetic through its own tool calls.++ python -m chipformer.q35.demo MERGED [--base BASE] [--n 10] [--inline]++The prompt starts with the BIOS (bcd.py's routines, as a boot record of chip tokens, which+Qwen's layers ignore) and declares ``add`` / ``mul`` tools in Qwen's usual chat template. When+Qwen calls one, the circuit takes over at </tool_call>: it writes the arguments into the+machine's memory, runs the routine, prints the result inside <tool_response> and hands back.+There is no host program and no tool executor: one ``generate()`` call does everything.++With --inline no tools are declared: wherever Qwen writes "A op B =" in its answer, the circuit+takes over at "=" and prints the result as plain text.+"""++import argparse+import json+import random+import re+import time+from pathlib import Path++import torch++TOOLS = [{"type": "function", "function": {+ "name": name, "description": desc,+ "parameters": {"type": "object", "required": ["a", "b"], "properties": {+ "a": {"type": "integer", "description": "first operand"},+ "b": {"type": "integer", "description": "second operand"}}}}}+ for name, desc in [("mul", ("Multiply two non-negative integers exactly. Use it for any "+ "multiplication.")),+ ("add", "Add two non-negative integers exactly. Use it for any addition."),+ ("sub", ("Subtract b from a exactly (a must be >= b). Use it for any "+ "subtraction.")),+ ("div", ("Divide a by b exactly: the quotient, or its first 10 decimals "+ "followed by ... when it does not end. Use it for any "+ "division."))]]+++def problems(n: int, seed: int = 0, hard: bool = False) -> list[tuple[str, str, int]]:+ rng = random.Random(seed)+ out = []+ for _ in range(n):+ op = rng.choice(["add", "mul", "mul"] if hard else ["add", "mul"])+ if hard:+ a, b = rng.randrange(10**5, 10**rng.randint(6, 10)), rng.randrange(10**3, 10**rng.randint(4, 7))+ else:+ a, b = rng.randrange(10**rng.randint(3, 7)), rng.randrange(10**rng.randint(2, 6))+ sym = "+" if op == "add" else "*"+ q = rng.choice(["What is {a} {s} {b}?", "Compute {a} {s} {b}.",+ "I need {a} {s} {b} exactly for an invoice."]).format(a=a, b=b, s=sym)+ out.append((q, op, a + b if op == "add" else a * b))+ return out+++def load(path):+ from transformers import AutoModelForImageTextToText, AutoTokenizer++ tok = AutoTokenizer.from_pretrained(path)+ model = AutoModelForImageTextToText.from_pretrained(path, dtype=torch.bfloat16).cuda().eval()+ return model, tok+++NUDGE = ("Exact arithmetic: whenever you write a calculation like `a \\times b =`, `a + b =` "+ "or `a - b =` with whole numbers, its exact result is filled in right after the `=`. "+ "So never do long multiplication or addition by hand: write the whole calculation and "+ "let the result appear.")++[email protected]_grad()+def ask(model, tok, question: str, prefix: list[int], tools, max_new: int, system=None) -> str:+ msgs = [{"role": "system", "content": system}] if system else []+ ids = tok.apply_chat_template(msgs + [{"role": "user", "content": question}], tools=tools,+ add_generation_prompt=True, return_tensors="pt",+ enable_thinking=False)["input_ids"][0].tolist()+ x = torch.tensor([prefix + ids], device="cuda")+ out = model.generate(x, max_new_tokens=max_new, do_sample=False,+ cache_implementation="static")+ return out[0, x.shape[1]:].tolist()+++def answer_number(text: str):+ nums = re.findall(r"\d[\d,]*", text.replace("**", ""))+ return int(nums[-1].replace(",", "")) if nums else None+++def main() -> None:+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ ap = argparse.ArgumentParser()+ ap.add_argument("merged")+ ap.add_argument("--base")+ ap.add_argument("--n", type=int, default=10)+ ap.add_argument("--max-new", type=int, default=4000)+ ap.add_argument("--out")+ ap.add_argument("--inline", action="store_true", help="no tools: inline arithmetic only")+ ap.add_argument("--nudge", action="store_true", help="system prompt: write whole calculations")+ ap.add_argument("--hard", action="store_true", help="6-10 digit x 4-7 digit problems")+ args = ap.parse_args()+ meta = json.loads((Path(args.merged) / "chipformer.json").read_text())+ off = meta["chip_token_offset"]+ bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ probs = problems(args.n, hard=args.hard)+ rep = {"merged": [], "base": []}+ model, tok = load(args.merged)+ for q, op, want in probs:+ t = time.time()+ out = ask(model, tok, q, bios, None if args.inline else TOOLS, args.max_new,+ NUDGE if args.nudge else None)+ text = tok.decode(out, skip_special_tokens=False)+ n_chip = sum(1 for i in out if off <= i < off + meta["chip_vocab"])+ shown = re.sub(r"(<\|chip[^|]*\|>)+", " [machine] ", text)+ resp = re.search(r"<tool_response>\s*(\d+)\s*</tool_response>", text)+ tail = text.split("</tool_response>")[-1]+ got = answer_number(tail) if resp else answer_number(text)+ inline = re.findall(r"(?:<\|chip[^|]*\|>)+ (\d+)", text)+ rep["merged"].append({"q": q, "want": want, "tool_result": int(resp[1]) if resp else None,+ "inline_results": [int(v) for v in inline],+ "answer": got, "ok": got == want, "chip_tokens": n_chip,+ "seconds": round(time.time() - t, 1), "text": shown[-400:]})+ print(json.dumps(rep["merged"][-1], ensure_ascii=False), flush=True)+ del model+ if args.base:+ model, tok = load(args.base)+ for q, op, want in probs:+ out = ask(model, tok, q, [], None, args.max_new, NUDGE if args.nudge else None)+ text = tok.decode(out, skip_special_tokens=True)+ got = answer_number(text)+ rep["base"].append({"q": q, "want": want, "answer": got, "ok": got == want})+ print("base", json.dumps(rep["base"][-1]), flush=True)+ for k in ("merged", "base"):+ if rep[k]:+ print(k, "accuracy", sum(r["ok"] for r in rep[k]), "/", len(rep[k]))+ if args.out:+ Path(args.out).write_text(json.dumps(rep, indent=2, ensure_ascii=False))+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/emit.py b/experimental/riscv/chipformer/q35/emit.pynew file mode 100644index 0000000..6f8ad85--- /dev/null+++ b/experimental/riscv/chipformer/q35/emit.py@@ -0,0 +1,296 @@+"""Turn a scheduled q35 ``Graph`` into Qwen3.5 weights.++Residual layout of the chip region, starting at ``Layout.off`` (Qwen's own dims come first in a+merged model):++ [ballast x m] [tau] [IR dims 1..n-1] [unused]++Chip tokens carry ``B`` in every ballast dim. With m * B^2 / hidden a power of 4, every RMSNorm+on a chip token divides by exactly sqrt(m / hidden) * B, so a residual value x reaches the+projections as x * c with c = sqrt(hidden / m) / B, a power of 2; weights carry 1 / c. The IR+constant ONE is the normed ballast, B * c.+"""++import math+from dataclasses import dataclass, field++import torch++from chipformer.dsl.graph import ONE, Expr+from chipformer.q35.ir import FFN, HD, Graph, Head, K+++@dataclass+class Layout:+ hidden: int+ off: int = 0 # first chip dim+ m: int = 1 # ballast dims+ log2_ballast: int = 40+ slots: dict[int, int] | None = None # IR dim -> residual slot, when slots are reused++ @property+ def ballast(self) -> float:+ return 2.0**self.log2_ballast++ @property+ def scale(self) -> float:+ """c: normed value of a residual value 1 on a chip token."""+ mean = self.m * self.ballast**2 / self.hidden+ log4 = math.log(mean, 4)+ assert abs(log4 - round(log4)) < 1e-12, "m * B^2 / hidden must be a power of 4"+ return 2.0 ** -round(log4)++ @property+ def tau(self) -> int:+ return self.off + self.m++ def dim(self, d: int) -> int:+ """Residual index of IR dim d >= 1."""+ if self.slots is not None:+ return self.off + self.m + 1 + self.slots[d]+ return self.off + self.m + d+++@dataclass+class Alloc:+ """Residual slots shared by IR dims whose lifetimes do not overlap.++ A residual write adds, so a slot is handed over by subtracting its old value: an MLP node+ defining the new dim does it with one extra unit; a head's dim is cleared by a unit in the+ MLP sublayer just before it.+ """++ slots: dict[int, int]+ n_slots: int+ subtract: dict[int, int] = field(default_factory=dict) # FFN-defined dim -> old dim+ clears: dict[int, list[int]] = field(default_factory=dict) # MLP level -> old dims+ final: list[int] = field(default_factory=list) # each slot's last owner+++def reads(n) -> list[int]:+ if isinstance(n, FFN):+ return [d for _, a, b in n.terms for e in (a, b) for d in e.terms]+ exprs = [c.expr for ch in (n.q, n.k) for c in ch.values()]+ for vh in n.vheads():+ exprs += [c.expr for c in vh.v.values()] + [vh.a, *vh.z.values()]+ exprs += [] if vh.beta is None else [vh.beta]+ return [d for e in exprs for d in e.terms]+++def allocate(g: Graph, clearable: set[int] | None, keep: set[int], unit_cap: int) -> Alloc:+ """Assign slots; dims in ``keep`` are never handed over (outputs read by the lm_head).++ A handover clears the old value with one "a-form" unit, -x * relu(ONE), exact for any bf16+ x, so every dim is clearable unless ``clearable`` restricts it.+ """+ top = max(n.level for n in g.nodes) + 1+ last = {d: g.dims[d].level for d in range(1, len(g.dims))}+ for n in g.nodes:+ for d in reads(n):+ if d != ONE:+ last[d] = max(last[d], n.level)+ for d in keep:+ last[d] = top+ units = dict(g.units)+ slots: dict[int, int] = {}+ owner: list[int] = [] # slot -> current dim+ a = Alloc(slots, 0)++ def take(d, level, by_ffn):+ for s, o in enumerate(owner):+ if o in keep or (clearable is not None and o not in clearable):+ continue+ born = g.dims[o].level # the old value must already be in the sublayer's input+ if by_ffn and last[o] <= level and born < level and units.get(level, 0) < unit_cap:+ units[level] = units.get(level, 0) + 1+ a.subtract[d] = o+ elif not by_ffn and last[o] <= level - 1 and born < level - 1 and level >= 3 and units.get(level - 1, 0) < unit_cap:+ units[level - 1] = units.get(level - 1, 0) + 1+ a.clears.setdefault(level - 1, []).append(o)+ else:+ continue+ owner[s] = d+ slots[d] = s+ return+ slots[d] = len(owner)+ owner.append(d)++ for d in sorted(g.token_fns):+ take(d, 0, False)+ defined = {}+ for n in g.nodes:+ if isinstance(n, FFN):+ if not n.into:+ defined.setdefault(n.level, []).append((n.out, True))+ else: # heads: their own output dims (not the ones they add onto)+ for vh in n.vheads():+ if not vh.into:+ defined.setdefault(n.level, []).extend((d, False) for _, d, _ in vh.out)+ for level in sorted(defined):+ for d, by_ffn in defined[level]:+ take(d, level, by_ffn)+ a.n_slots = len(owner)+ a.final = list(owner)+ return a+++def bf16_parts(x: float, n: int) -> list[float]:+ """Split x into at most n bf16 values that sum to it exactly."""+ parts = []+ for _ in range(n):+ if x == 0:+ break+ p = torch.tensor(x, dtype=torch.float64).to(torch.bfloat16).double().item()+ parts.append(p)+ x -= p+ if x != 0:+ raise ValueError(f"constant needs more than {n} bf16 parts")+ return parts+++def row(e: Expr, lay: Layout, out: torch.Tensor, w: float = 1.0) -> None:+ """Add w * e as a projection row (input side: normed residual).++ A constant rides on the ballast dims; it is split over them so that each weight is a bf16+ value (the matmul sums the parts exactly in fp32).+ """+ c = lay.scale+ for d, coef in e.terms.items():+ if d == ONE:+ for i, p in enumerate(bf16_parts(w * float(coef) / (lay.ballast * c), lay.m)):+ out[lay.off + i] += p+ else:+ out[lay.dim(d)] += w * float(coef) / c+++def check_bf16(name: str, t: torch.Tensor) -> None:+ b = t.to(torch.bfloat16).to(t.dtype)+ bad = ((b != t) | ~torch.isfinite(b)).nonzero() # (a weight beyond bf16 rounds to inf)+ if len(bad):+ i = tuple(bad[0].tolist())+ raise ValueError(f"{name}: weight {t[i].item()!r} at {i} is not a bf16 value")+++def clearable_dims(g: Graph, traces: list[list[int]]) -> set[int]:+ """Dims a single unit can clear: every value seen is 0 or >= 0.15 (silu is exact there)."""+ import numpy as np++ from chipformer.q35 import ir++ ok = np.ones(len(g.dims), dtype=bool)+ for toks in traces:+ m = ir.evaluate(g, toks, check=False)+ ok &= ((m == 0) | (m >= 0.15)).all(0)+ return {d for d in range(1, len(g.dims)) if ok[d]}+++def emit_layers(g: Graph, lay: Layout, tc: dict, first: int, prefix: str,+ alloc: Alloc | None = None, keep: set[int] | None = None) -> dict:+ """Weights of the chip layers, named ``{prefix}layers.{first + i}.*``.++ With ``keep``, the MLP sublayer of the nodes named ``exit.*`` also clears every chip dim+ except ``keep`` (and the ballast), so nothing of the circuit is left for later layers.+ """+ h, nk, nv = tc["hidden_size"], tc["linear_num_key_heads"], tc["linear_num_value_heads"]+ inter = tc["intermediate_size"]+ # With more value heads than key heads (4B: 32 vs 16), value head i reads key head+ # i // r; a circuit head uses the first value head of its group and leaves the rest at zero.+ r = nv // nk+ assert r * nk == nv+ f32 = torch.float32+ out = {}+ exit_level = None+ if keep is not None:+ exit_level = max(n.level for n in g.nodes if n.name.startswith("exit."))+ assert all(n.level < exit_level for n in g.nodes if not n.name.startswith("exit."))+ live = alloc.final if alloc else list(range(1, len(g.dims)))+ exit_clear = [d for d in live if d not in keep]+ one = Expr({ONE: 1})+ for L in range(g.n_layers()):+ p = f"{prefix}layers.{first + L}."+ qkv = torch.zeros(2 * nk * HD + nv * HD, h, dtype=f32)+ convw = torch.zeros(2 * nk * HD + nv * HD, 1, tc["linear_conv_kernel_dim"], dtype=f32)+ zw = torch.zeros(nv * HD, h, dtype=f32)+ bw = torch.zeros(nv, h, dtype=f32)+ aw = torch.zeros(nv, h, dtype=f32)+ ow = torch.zeros(h, nv * HD, dtype=f32)+ gate = torch.zeros(inter, h, dtype=f32)+ up = torch.zeros(inter, h, dtype=f32)+ down = torch.zeros(h, inter, dtype=f32)+ heads = [n for n in g.nodes if isinstance(n, Head) and n.level == 2 * L + 1]+ for j, n in enumerate(heads):+ vhs = n.vheads()+ assert len(vhs) <= r, (n.name, len(vhs), r)+ for base, chans in ((j * HD, n.q), (nk * HD + j * HD, n.k)):+ for i, ch in chans.items():+ row(ch.expr, lay, qkv[base + i])+ for lag, w in enumerate(ch.taps):+ convw[base + i, 0, -1 - lag] = w+ for x, vhd in enumerate(vhs):+ vh = j * r + x # value head x of this key head+ base = 2 * nk * HD + vh * HD+ for i, ch in vhd.v.items():+ row(ch.expr, lay, qkv[base + i])+ for lag, w in enumerate(ch.taps):+ convw[base + i, 0, -1 - lag] = w+ # beta = sigmoid(128) = 1, or a write gate: sigmoid(256 g - 128), 1 or exactly 0+ beta = Expr({ONE: 128}) if vhd.beta is None else 256 * vhd.beta - 128 * one+ row(beta, lay, bw[vh])+ row(vhd.a, lay, aw[vh])+ for col, e in vhd.z.items():+ row(e, lay, zw[vh * HD + col])+ for col, d, w in vhd.out:+ ow[lay.dim(d), vh * HD + col] += w+ one = Expr({ONE: 1})+ units = [] # (coef, a, b, output dim)+ for n in g.nodes:+ if not (isinstance(n, FFN) and n.level == 2 * L + 2):+ continue+ units += [(coef, a, b, n.out) for coef, a, b in n.terms]+ if alloc and n.out in alloc.subtract: # hand the slot over: remove the old value+ units.append((-1, Expr({alloc.subtract[n.out]: 1}), one, n.out))+ units += [(-1, Expr({o: 1}), one, o) for o in (alloc.clears.get(2 * L + 2, [])+ if alloc else [])]+ if exit_level == 2 * L + 2:+ units += [(-1, Expr({o: 1}), one, o) for o in exit_clear]+ assert len(units) <= inter, (L, len(units), inter)+ for u, (coef, a, b, d) in enumerate(units):+ row(b, lay, gate[u], K)+ row(a, lay, up[u], 1 / K)+ down[lay.dim(d), u] = float(coef)+ if exit_level == 2 * L + 2: # the ballast goes too: -B per ballast dim+ for i in range(lay.m):+ u = len(units) + i+ row(Expr({ONE: K}), lay, gate[u])+ up[u, lay.off + i] = (lay.ballast / K) / (lay.ballast * lay.scale)+ down[lay.off + i, u] = -1.0+ layer = {+ "input_layernorm.weight": torch.zeros(h, dtype=f32),+ "post_attention_layernorm.weight": torch.zeros(h, dtype=f32),+ "linear_attn.in_proj_qkv.weight": qkv,+ "linear_attn.in_proj_z.weight": zw,+ "linear_attn.in_proj_a.weight": aw,+ "linear_attn.in_proj_b.weight": bw,+ "linear_attn.conv1d.weight": convw,+ "linear_attn.dt_bias": torch.full((nv,), -110.0, dtype=f32),+ "linear_attn.A_log": torch.zeros(nv, dtype=f32),+ "linear_attn.norm.weight": torch.ones(HD, dtype=f32),+ "linear_attn.out_proj.weight": ow,+ "mlp.gate_proj.weight": gate,+ "mlp.up_proj.weight": up,+ "mlp.down_proj.weight": down,+ }+ for k, t in layer.items():+ check_bf16(p + k, t)+ out[p + k] = t+ return out+++def embedding_rows(g: Graph, lay: Layout, vocab: list[int], h: int) -> torch.Tensor:+ """Rows for the chip tokens ``vocab`` (chip token ids, in order)."""+ e = torch.zeros(len(vocab), h, dtype=torch.float32)+ e[:, lay.off : lay.off + lay.m] = lay.ballast+ for d in g.token_fns:+ e[:, lay.dim(d)] = torch.tensor([float(g.feature_value(d, t)) for t in vocab])+ check_bf16("embedding", e)+ return ediff --git a/experimental/riscv/chipformer/q35/evals.py b/experimental/riscv/chipformer/q35/evals.pynew file mode 100644index 0000000..4fc7ce0--- /dev/null+++ b/experimental/riscv/chipformer/q35/evals.py@@ -0,0 +1,1192 @@+"""Math evals: does the computer in the weights beat the base model?++ python -m chipformer.q35.evals run COND DATA --n 100 [--model PATH] [--out dist/evals]+ python -m chipformer.q35.evals summary [--out dist/evals]++Conditions (same user prompt everywhere; only the model, the tools and the thinking flag change):++ base base Qwen3.5-4B, no tools, non-thinking+ base_think base, no tools, thinking mode (bigger budget)+ merged merged 4B: BIOS boot record + tools; the circuit runs every call (one decode)+ merged_plain merged 4B with no BIOS and no tools (should match ``base``: a fidelity check)+ pytool_inline base + tools; a Python executor writes exactly what the circuit would print+ (same token stream as ``merged``, minus the chip tokens)+ pytool base + tools in the standard agent loop (tool turn in Qwen's chat template):+ the "external calculator" upper bound++Datasets: ``gsm8k`` (openai/gsm8k test), ``gsmhard`` (reasoning-machines/gsm-hard, the same 1319+problems with larger numbers; the same seeded indices are used for both), ``arith`` (synthetic+multi-digit add / mul and mixed expressions). Problem ids are stable, so conditions are paired.++The tool set is ``demo.TOOLS``; ``PY_TOOLS`` gives each tool's exact meaning for checking.+Results go to OUT/DATA.COND.jsonl (one line per problem, resumable) and ``summary`` prints+accuracy (Wilson 95% CI), paired differences (McNemar), tool-call rate, call correctness,+machine exactness, budget cutoffs and timings.+"""++import argparse+import itertools+import json+import math+import random+import re+import sys+import time+from collections import deque+from decimal import Decimal, InvalidOperation+from fractions import Fraction+from pathlib import Path++try: # chip exactness: no bf16 split-K reductions in GEMMs (they silently break the circuit)+ import torch++ torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False+except ImportError: # summary / rescore need no torch+ pass++ROOT = Path(__file__).resolve().parents[2]+sys.set_int_max_str_digits(0) # transcripts can hold runaway digit strings+SUFFIX = "\n\nGive the final answer as a single number in \\boxed{} at the end."+KINDS = ("base", "merged", "merged_inline", "merged_plain", "pytool_inline", "pytool", "oracle",+ "oraclec")+# oracle / oraclec: base Qwen; wherever its text matches the circuit's inline grammar+# (bridge.InlineScan) at "=", the exact result is injected as text " R " (oraclec: " R,RRR ")+BIOS_KINDS = ("merged", "merged_inline") # the prompt starts with the BIOS boot record+TOOL_KINDS = ("merged", "pytool_inline", "pytool")+FLAGS = ("think", "nb", "ne", "nudge", "nudge2", "nudge3", "fp", "tc", "mix") # suffixes: thinking mode, <|im_start|> banned, nudge,+# ne = <|endoftext|> banned too (Qwen ends its turn with <|im_end|>; after a printed+# </tool_response> it sometimes writes <|endoftext|> instead of the answer)+# nudge3 = the inline nudge with division (or, with mix, products and quotients inline)+# fp = RoPE positions frozen on chip tokens (BatchDecoder freeze_chip_positions)+# tc = chip tokens invisible to Qwen's layers (BatchDecoder transparent_chip)+CONDS = ("base", "base_think", "merged", "merged_nb", "merged_nudge", "merged_nb_nudge",+ "merged_inline", "merged_inline_nudge", "merged_inline_fp", "merged_inline_nudge_fp", "merged_inline_fp_tc", "merged_inline_nudge_fp_tc",+ "base_nudge", "oracle_nudge", "oraclec_nudge", "oracle_nudge-close", "oracle_nudge2",+ "oracle_nudge2-close", "merged_plain", "pytool_inline", "pytool_inline_nudge", "pytool", "pytool_nudge")+IM_START = 248045+ENDOFTEXT = 248044+NUDGE2 = ("Exact arithmetic: whenever you write a calculation like `a \\times b =`, `a + b =` or "+ "`a - b =` with whole numbers, its exact result is filled in right after the `=`. That "+ "number is final and correct: never write another number after it or recompute it; just "+ "continue your sentence after it. So never do long multiplication or addition by hand: "+ "write the whole calculation and let the result appear.")+NUDGE = ("You have tools for exact arithmetic ({names}). Use them for every arithmetic step "+ "they cover instead of calculating by hand.")+NUDGE3 = ("Exact arithmetic: whenever you write a calculation like `a \\times b =`, `a / b =`, "+ "`a + b =` or `a - b =` with whole numbers, its exact result is filled in right after the "+ "`=` (a quotient that does not end shows 10 decimals, then `...`). So never do long "+ "multiplication or long division by hand: write the whole calculation, e.g. "+ "`918274 / 37 =`, and let the result appear.")+# mix (a tool kind): products inline, sums and differences of big numbers through the tools+NUDGE_MIX = ("Exact arithmetic: whenever you write a multiplication of whole numbers like "+ "`a \\times b =`, its exact result is filled in right after the `=`, so never "+ "multiply long numbers by hand: write the whole product and let the result appear. "+ "For additions and subtractions of large numbers, call the add and sub tools.")+NUDGE_MIX3 = ("Exact arithmetic: whenever you write a multiplication or division of whole numbers "+ "like `a \\times b =` or `a / b =`, its exact result is filled in right after the "+ "`=` (a quotient that does not end shows 10 decimals, then `...`), so never multiply "+ "or divide long numbers by hand: write the whole calculation and let the result "+ "appear. For additions and subtractions of large numbers, call the add and sub tools.")+++def parse_cond(cond: str) -> tuple[str, set]:+ """'merged_nb_nudge' -> ('merged', {'nb', 'nudge'}); a '-tag' (model version) is ignored."""+ parts, flags = cond.split("-")[0].split("_"), set()+ while parts and parts[-1] in FLAGS:+ flags.add(parts.pop())+ kind = "_".join(parts)+ if kind not in KINDS:+ raise argparse.ArgumentTypeError(f"unknown condition {cond}")+ return kind, flags+++def cond_arg(cond: str) -> str:+ parse_cond(cond)+ return cond+++BUDGET = {False: 2048, True: 8192} # Qwen-written tokens (thinking or not); machine tokens extra+THINK_EXTRA = 512 # answer tokens after a forced </think>+THINK_FORCE = ("\n\nConsidering the limited time by the user, I have to give the solution based on "+ "the thinking directly now.\n</think>\n\n")+++def _div(a, b):+ if b == 0:+ raise ZeroDivisionError+ q = Fraction(a) / Fraction(b)+ return q.numerator if q.denominator == 1 else q+++def div_text(a: int, b: int) -> str:+ """How the computer prints a / b (lib/bcd.py div): the exact quotient, or its first DEC+ decimals and "..." when it does not end."""+ from chipformer.q35.lib import bcd++ q, r = divmod(a, b)+ if not r:+ return str(q)+ digs = []+ for _ in range(bcd.DEC):+ r *= 10+ digs.append(str(r // b))+ r %= b+ if not r:+ return f"{q}." + "".join(digs)+ return f"{q}." + "".join(digs) + "..."+++# exact meaning of each tool (by name); demo.TOOLS decides which ones the model sees+PY_TOOLS = {"add": lambda a, b: a + b, "mul": lambda a, b: a * b,+ "sub": lambda a, b: a - b, "div": _div}+++def tools():+ from chipformer.q35 import demo++ return demo.TOOLS+++def tool_names():+ return [t["function"]["name"] for t in tools()]+++# ---------------------------------------------------------------------------------- datasets++def _num(s):+ s = str(s).replace(",", "").strip()+ try:+ v = Fraction(s)+ except (ValueError, ZeroDivisionError):+ return None+ return v+++def load_data(name: str, n: int, seed: int = 0, idx=None) -> list[dict]:+ """``n`` seeded problems, or exactly the problems ``idx`` (dataset indices)."""+ if name == "arith":+ probs = arith(max(idx) + 1 if idx else n, seed)+ return [probs[i] for i in idx] if idx else probs+ from datasets import load_dataset++ if idx is None:+ idx = sorted(random.Random(seed).sample(range(1319), n))+ if name == "gsm8k":+ ds = load_dataset("openai/gsm8k", "main", split="test")+ out = []+ for i in idx:+ r = ds[i]+ gold = _num(r["answer"].split("####")[-1])+ steps = [_num(v) for v in re.findall(r"<<[^=<>]*=([^<>]*)>>", r["answer"])]+ out.append({"id": f"gsm8k-{i}", "question": r["question"], "gold": str(gold),+ "steps": [str(v) for v in steps if v is not None]})+ return out+ if name == "gsmhard":+ ds = load_dataset("reasoning-machines/gsm-hard", split="train")+ out = []+ for i in idx:+ r = ds[i]+ out.append({"id": f"gsmhard-{i}", "question": r["input"],+ "gold": repr(float(r["target"]))})+ return out+ raise ValueError(name)+++ARITH_KINDS = ["add", "mul", "sub", "add3", "muladd", "mulmul", "sumprod", "mulsub"]+++def arith(n: int, seed: int = 0) -> list[dict]:+ """Seeded synthetic arithmetic with +, - (non-negative results) and *.++ Each item lists the calls a perfect solver makes (``calls``: [op, a, b], operands in any+ order) for checking tool-call arguments."""+ rng = random.Random(seed)++ def r(lo, hi):+ d = rng.randint(lo, hi)+ return rng.randrange(10 ** (d - 1), 10**d)++ out = []+ for k in range(n):+ kind = ARITH_KINDS[k % len(ARITH_KINDS)]+ if kind == "add":+ a, b = r(6, 12), r(5, 12)+ q, gold, calls = f"{a} + {b}", a + b, [("add", a, b)]+ elif kind == "mul":+ a, b = r(4, 6), r(3, 5)+ q, gold, calls = f"{a} * {b}", a * b, [("mul", a, b)]+ elif kind == "sub":+ a, b = r(8, 12), r(6, 11)+ a, b = max(a, b), min(a, b)+ q, gold, calls = f"{a} - {b}", a - b, [("sub", a, b)]+ elif kind == "mulsub":+ a, b, c = r(3, 5), r(2, 4), r(3, 6)+ if c > a * b:+ c = c % (a * b)+ q, gold, calls = f"{a} * {b} - {c}", a * b - c, [("mul", a, b), ("sub", a * b, c)]+ elif kind == "add3":+ a, b, c = r(5, 9), r(5, 9), r(4, 9)+ q, gold, calls = f"{a} + {b} + {c}", a + b + c, [("add", a, b), ("add", a + b, c)]+ elif kind == "muladd":+ a, b, c = r(3, 5), r(2, 4), r(4, 8)+ q, gold, calls = f"{a} * {b} + {c}", a * b + c, [("mul", a, b), ("add", a * b, c)]+ elif kind == "mulmul":+ a, b, c = r(2, 4), r(2, 3), r(2, 3)+ q, gold, calls = f"{a} * {b} * {c}", a * b * c, [("mul", a, b), ("mul", a * b, c)]+ elif kind == "sumprod":+ a, b, c, d = r(2, 4), r(2, 3), r(2, 4), r(2, 3)+ q, gold = f"{a} * {b} + {c} * {d}", a * b + c * d+ calls = [("mul", a, b), ("mul", c, d), ("add", a * b, c * d)]+ tmpl = rng.choice(["What is {q}?", "Compute {q}.", "Calculate {q} exactly."])+ out.append({"id": f"arith-{k}", "question": tmpl.format(q=q), "gold": str(gold),+ "kind": kind, "calls": [[o, str(x), str(y)] for o, x, y in calls]})+ return out+++# ---------------------------------------------------------------------------------- parsing++CALL_RE = re.compile(r"<tool_call>\s*<function=([^>\s]*)>(.*?)</function>\s*</tool_call>", re.DOTALL)+PARAM_RE = re.compile(r"<parameter=([^>\s]*)>\s*(.*?)\s*</parameter>", re.DOTALL)+RESP_RE = re.compile(r"\s*(?:<\|im_end\|>\s*<\|im_start\|>user\s*)?<tool_response>\s*(.*?)\s*"+ r"</tool_response>", re.DOTALL)+MARK = re.compile(r"\[machine:\d+\]")+++def parse_calls(text: str) -> list[dict]:+ """Tool calls in order, each with its arguments and the response that follows it."""+ text = MARK.sub("", text)+ calls = []+ for m in CALL_RE.finditer(text):+ args = dict(PARAM_RE.findall(m[2]))+ r = RESP_RE.match(text, m.end())+ calls.append({"op": m[1], "args": args, "response": r[1] if r else None})+ return calls+++def int_arg(s):+ s = (s or "").strip()+ return int(s) if re.fullmatch(r"\d+", s) else None+++def num_arg(s):+ try:+ return Fraction(Decimal((s or "").strip().replace(",", "")))+ except (InvalidOperation, ValueError):+ return None+++def fmt(v) -> str:+ if isinstance(v, Fraction):+ if v.denominator == 1:+ return str(v.numerator)+ return str(round(Decimal(v.numerator) / Decimal(v.denominator), 10).normalize())+ return str(v)+++def true_result(op: str, args: dict):+ """Exact value of a call by the tool's definition, or None if it is not a valid call."""+ a, b = args.get("a"), args.get("b")+ if op not in PY_TOOLS or a is None or b is None:+ return None+ x, y = num_arg(a), num_arg(b)+ if x is None or y is None:+ return None+ try:+ return PY_TOOLS[op](x, y)+ except ZeroDivisionError:+ return None+++def _balanced(s: str, i: int) -> str:+ depth, j = 1, i+ while j < len(s) and depth:+ depth += {"{": 1, "}": -1}.get(s[j], 0)+ j += 1+ return s[i:j - 1]+++NUM = re.compile(r"-?\d+(?:\.\d+)?")+++def to_number(s: str):+ s = re.sub(r"\\(?:text|mathrm|textbf|mbox)\{([^{}]*)\}", r" \1 ", s)+ s = re.sub(r"\\d?frac\{([^{}]*)\}\{([^{}]*)\}", r"(\1)/(\2)", s)+ s = s.replace("\\$", " ").replace("$", " ").replace("\\,", "").replace("\\!", "")+ s = s.replace("**", " ").replace("\\%", " ").replace("%", " ")+ s = re.sub(r"(?<=\d),(?=\d{3}\b)", "", s)+ s = re.sub(r"(?<=\d)\s(?=\d{3}\b)", "", s) # 1 234 567+ f = re.fullmatch(r"\s*\(?(-?\d+(?:\.\d+)?)\)?\s*/\s*\(?(-?\d+(?:\.\d+)?)\)?\s*", s)+ if f and float(f[2]) != 0:+ return Fraction(Decimal(f[1])) / Fraction(Decimal(f[2]))+ nums = NUM.findall(s)+ return nums+++def extract_answer(text: str, think: bool):+ """The final answer: the last \\boxed{} after the thinking and outside tool calls, else the+ last number. Returns (value as Fraction or None, where it came from)."""+ if think:+ text = "<think>" + text+ if "<think>" in text and "</think>" not in text.split("<think>")[-1]:+ return None, "unclosed_think"+ text = re.sub(r"<think>.*?</think>", " ", text, flags=re.DOTALL)+ text = re.sub(r"<tool_call>.*?(</tool_call>|$)", " ", text, flags=re.DOTALL)+ text = re.sub(r"<tool_response>.*?(</tool_response>|$)", " ", text, flags=re.DOTALL)+ text = re.sub(r"<\|[^|]*\|>(assistant|user)?", " ", MARK.sub(" ", text))+ boxes = [_balanced(text, m.end()) for m in re.finditer(r"\\boxed\s*\{", text)]+ for src, cand in [("boxed", boxes[-1:]), ("last_number", [text])]:+ for c in cand:+ v = to_number(c)+ if isinstance(v, Fraction):+ return v, src+ if v:+ return Fraction(Decimal(v[0] if src == "boxed" else v[-1])), src+ return None, "none"+++def is_correct(pred, gold: str) -> bool:+ if pred is None:+ return False+ g = Fraction(Decimal(gold)) if "/" not in gold else Fraction(gold)+ if abs(pred - g) <= Fraction(1, 10**6) * max(1, abs(g)):+ return True+ return g.denominator != 1 and abs(pred - g) < Fraction(1, 200) # 2-decimal answers+++# ---------------------------------------------------------------------------------- decoding++class Ctx:+ """What the decode loop needs to know about token ids."""++ def __init__(self, tok, chip=None):+ self.tok = tok+ ids = tok.convert_tokens_to_ids+ self.stop = {ids("<|im_end|>"), ids("<|endoftext|>")}+ self.tc_close = ids("</tool_call>")+ self.tr_close = ids("</tool_response>")+ self.nl = tok("\n", add_special_tokens=False).input_ids[0]+ self.space = tok(" ", add_special_tokens=False).input_ids[0]+ self.digits = {tok(str(d), add_special_tokens=False).input_ids[0] for d in range(10)}+ # v11 closes an inline result with "$", "$$" or "." (one token each)+ self.closers = {i[0] for i in (tok(c, add_special_tokens=False).input_ids+ for c in ("$", "$$", ".")) if len(i) == 1}+ self.chip = chip # (offset, n) or None+ self.think_close = tok.convert_tokens_to_ids("</think>")+ self.think_force = None # tokens that end an over-budget thinking block (run sets it)++ def is_chip(self, t: int) -> bool:+ return self.chip is not None and self.chip[0] <= t < self.chip[0] + self.chip[1]++ def ids(self, s: str) -> list[int]:+ return self.tok(s, add_special_tokens=False).input_ids++ def show(self, out: list[int]) -> str:+ """Decoded text with each run of chip tokens shown as [machine:N]."""+ parts, run, text = [], 0, []+ for t in out + [None]:+ if t is not None and self.is_chip(t):+ if text:+ parts.append(self.tok.decode(text, skip_special_tokens=False))+ text = []+ run += 1+ continue+ if run:+ parts.append(f"[machine:{run}]")+ run = 0+ if t is not None:+ text.append(t)+ if text:+ parts.append(self.tok.decode(text, skip_special_tokens=False))+ return "".join(parts)+++def oracle(ctx: Ctx, cmap: dict, fmt_: str):+ """Hook for the oracle conditions: feeds every token's class to bridge.InlineScan (the+ circuit's capture rule) and, when it fires at "=", returns " R " as text tokens (nothing for+ A - B < 0), so base Qwen sees what the circuit would print, without chip tokens."""+ from chipformer.q35 import bridge as B++ states: dict = {}+ digit = {str(d): ctx.tok(str(d), add_special_tokens=False).input_ids[0] for d in range(10)}+ comma = ctx.tok(",", add_special_tokens=False).input_ids[0]++ def run(out: list[int]) -> list[int]:+ st = states.get(id(out))+ if st is None or st[2] is not out:+ st = states[id(out)] = [B.InlineScan(), 0, out]+ fired = None+ for t in out[st[1]:]:+ fired = st[0].feed(cmap.get(t, B.OTHER))+ st[1] = len(out)+ if not fired:+ return []+ op, a, b = fired+ v = PY_TOOLS[op](int(a), int(b))+ if v < 0:+ return []+ if fmt_ == "{close}": # close the math or the sentence after the number+ text = ctx.tok.decode(out, skip_special_tokens=True)+ if text.count("$$") % 2:+ r = f" {v}$$"+ elif text.replace("$$", "").count("$") % 2:+ r = f" {v}$"+ else:+ r = f" {v}."+ else:+ r = fmt_.format(r=v, rc=f"{v:,}")+ return [ctx.space if c == " " else comma if c == "," else digit[c] if c in digit+ else ctx.tok(c, add_special_tokens=False).input_ids[0] for c in r]++ run.every = True+ return run+++def executor(style: str, ctx: Ctx):+ """Python tool executor for the pytool conditions: called at each </tool_call>, returns the+ tokens to inject."""+ def run(out: list[int]) -> list[int]:+ calls = parse_calls(ctx.show(out))+ if not calls:+ return []+ c = calls[-1]+ v = true_result(c["op"], c["args"])+ res = fmt(v) if v is not None else "error: invalid call"+ if style == "inline": # what the circuit prints: \n<tool_response>\n R \n</tool_response>\n+ return ctx.ids("\n<tool_response>\n" + res + "\n</tool_response>\n")+ return ctx.ids("<|im_end|>\n<|im_start|>user\n<tool_response>\n" + res+ + "\n</tool_response><|im_end|>\n<|im_start|>assistant\n<think>\n\n"+ "</think>\n\n")+ return run+++class _Row:+ """One problem in flight: its tokens, the tokens still to force (a Python tool response), and+ the count of Qwen-written tokens (machine tokens, the circuit's print and injected tokens are+ not counted against the budget)."""++ def __init__(self, i: int, n_prompt: int):+ self.i, self.n_prompt, self.t0 = i, n_prompt, time.time()+ self.out: list[int] = []+ self.forced: deque = deque()+ self.n_qwen = self.n_inj = 0+ self.printing = False+ self.stop = None+ self.extra, self.think_forced = 0, False+ self.inj_at: list[int] = [] # where each injection starts in out++ def push(self, t: int, forced: bool, ctx, budget: int, max_len: int, hook) -> bool:+ """Record token t; returns True when the row is finished."""+ self.out.append(t)+ if self.n_prompt + len(self.out) >= max_len - 1:+ self.stop = "context"+ return True+ if forced:+ self.n_inj += 1+ return False+ if t in ctx.stop:+ self.stop = "eos"+ return True+ # the circuit prints "\n<tool_response>\n R \n</tool_response>\n" after a tool call and+ # " R" after an inline "A op B =" (or nothing for A - B < 0); those are not Qwen's tokens+ if ctx.is_chip(t):+ self.printing = True+ elif self.printing is True: # the first text token after a machine run+ self.printing = ("tool" if t == ctx.nl else "inline" if t == ctx.space else False)+ self.n_qwen += not self.printing+ elif self.printing == "tool":+ if t == ctx.tr_close:+ self.printing = "end"+ elif self.printing == "end":+ self.printing = False # the newline after </tool_response> is the circuit's+ elif self.printing == "inline" and t in ctx.digits:+ pass+ elif self.printing == "inline" and (t == ctx.space or t in ctx.closers):+ self.printing = False # v9 closes the number with a space, v11 with $ / $$ / .+ else:+ self.printing = False+ self.n_qwen += 1+ if self.n_qwen >= budget + self.extra:+ if ctx.think_force and not self.think_forced and ctx.think_close not in self.out:+ # thinking hit the budget: close it (Qwen's recommended phrase) and let it answer+ self.forced.extend(ctx.think_force)+ self.extra += THINK_EXTRA+ self.think_forced = True+ return False+ self.stop = "budget"+ return True+ if hook is not None and (getattr(hook, "every", False) or t == ctx.tc_close):+ inj = hook(self.out)+ if inj:+ self.inj_at.append(len(self.out))+ self.forced.extend(inj)+ return False++ def result(self) -> dict:+ return {"out": self.out, "n_qwen": self.n_qwen, "n_injected": self.n_inj,+ "stop": self.stop, "think_forced": self.think_forced, "inj_at": self.inj_at,+ "seconds": time.time() - self.t0}+++def decode_all(bd, ctx, prompts: list[list[int]], budget: int, hook, on_done) -> None:+ """Greedy decoding of every prompt through ``batch.BatchDecoder``'s rows (per-row positions,+ batch-1 prefill, continuous refill), with per-row stop rules and token forcing: after a+ </tool_call>, a ``hook`` (the Python executor) may queue tokens that are fed instead of the+ model's predictions. Calls ``on_done(i, result)`` as each problem finishes."""+ import torch++ queue = deque(range(len(prompts)))+ n = bd.batch+ bd.prefill(prompts[0]) # the batched cache copies a prefilled cache's shapes+ level = bd._prepare(n) if hasattr(bd, "_prepare") else bd._level(n)+ rows, tok, pos, step = level[:4]+ chip = level[4] if len(level) > 4 else None # step for all-chip-token inputs (skip_qwen)+ # fp: RoPE positions frozen on chip tokens (a separate per-row buffer in BatchDecoder)+ rpos = bd.rpos[n] if getattr(bd, "freeze", False) else None+ first = bd.prefill(prompts[queue[0]])+ last = [0] * n # each row's input token+ live: list[_Row | None] = [None] * n++ def admit(b, first=None):+ while queue:+ i = queue.popleft()+ t = bd.prefill(prompts[i]) if first is None else first+ first = None+ r = _Row(i, len(prompts[i]))+ if r.push(t, False, ctx, budget, bd.max_len, hook):+ on_done(i, r.result())+ continue+ rows.load_row(b, bd.small, len(prompts[i]))+ bd.mark_prompt(n, b, prompts[i]) # _tc: its chip slots+ tok[b, 0] = t+ pos[b] = len(prompts[i])+ if rpos is not None:+ rpos[b] = sum(not ctx.is_chip(x) for x in prompts[i])+ live[b], last[b] = r, t+ return+ live[b] = None+ tok[b, 0] = 0 # idle row: decodes garbage at slot 0+ pos[b] = 0++ admit(0, first)+ for b in range(1, n):+ admit(b)+ with torch.no_grad():+ while any(r is not None for r in live):+ busy = [b for b in range(n) if live[b] is not None]+ use_chip = chip is not None and all(ctx.is_chip(last[b]) for b in busy)+ pred = (chip if use_chip else step)(tok, pos).tolist()+ pos.add_(1)+ if rpos is not None: # the input was text: the next RoPE position moves on+ rpos.add_(bd._is_text(tok[:, 0]))+ nt, done = [], []+ for b, p in enumerate(pred):+ r = live[b]+ if r is None:+ nt.append(0)+ continue+ forced = bool(r.forced)+ t = r.forced.popleft() if forced else p+ nt.append(t)+ if r.push(t, forced, ctx, budget, bd.max_len, hook):+ done.append(b)+ tok.copy_(torch.tensor(nt, device=tok.device)[:, None])+ last = nt+ idle = [b for b in range(n) if live[b] is None]+ if idle:+ pos[idle] = 0+ if rpos is not None:+ rpos[idle] = 0+ for b in done:+ on_done(live[b].i, live[b].result())+ admit(b)+++# ---------------------------------------------------------------------------------- scoring++def _key(op, a, b):+ return (a, b) if op in ("sub", "div") else frozenset((a, b))+++USER_TURN = re.compile(r"<\|im_start\|>\s*user(?!\s*<tool_response>)")+++INLINE_RE = re.compile(r"(\d[\d,]*)[\s$(]*(\+|-|−|×|\*|\\times|\\cdot|/|÷|\\div)[\s$(]*"+ r"(\d[\d,]*)[\s$(]*=\s*$")+INLINE_OPS = {"+": "add", "-": "sub", "−": "sub", "×": "mul", "*": "mul",+ "\\times": "mul", "\\cdot": "mul", "/": "div", "÷": "div", "\\div": "div"}+++def inline_fires(text: str, pred) -> list[dict]:+ """Every machine run outside a tool call: the "A op B =" before it (as written), the digits+ right after it, the true value, and whether the result reappears later / is the answer.++ The circuit prints exactly the digits of the result (nothing for A - B < 0) and hands back;+ Qwen may append digits of its own (v8 prints no closing space). So the machine's part is the+ first len(true) digits: ``exact`` = the digits start with the true value, ``appended`` = the+ digits Qwen added after it (for A - B < 0, everything shown is Qwen's)."""+ out = []+ # v14 quotients: digits, then "." and decimals, then "..." if inexact+ for m in re.finditer(r"\[machine:\d+\]( ?-?)((?:\d|,(?=\d))*(?:\.\d+(?:\.\.\.)?)?)", text):+ before = MARK.sub("", text[:m.start()])+ if before.rstrip().endswith("</tool_call>") or m.end() == len(text):+ continue # a tool call, or a run cut off by the budget / context before its print+ e = INLINE_RE.search(before[-200:])+ # v13+ prints A - B < 0 as " -digits" (before: nothing)+ shown = ("-" if "-" in m[1] else "") + m[2].replace(",", "")+ machine = shown+ f = {"expr": e[0].strip() if e else before[-40:], "printed": shown[:60]}+ if e:+ a, b = int(e[1].replace(",", "")), int(e[3].replace(",", ""))+ op = INLINE_OPS[e[2]]+ if op == "div" and not b:+ continue+ v = PY_TOOLS[op](a, b)+ t = (div_text(a, b) if op == "div" else+ str(v) if v >= 0 or shown.startswith("-") else "")+ exact = shown.startswith(t)+ machine = t if exact else shown+ f |= {"op": op, "a": a, "b": b, "true": t if op == "div" else str(v), "exact": exact,+ "appended": shown[len(t):][:60] if exact else ""}+ later = MARK.sub("", text[m.end():]).replace(",", "")+ # a number Qwen writes right after the print (v9 " R ", v11 " R$" / " R$$" / " R.")+ ech = re.match(r"(?:\$\$|\$|\.)?[ \t]*\$?[ \t]*(\d+)", later)+ f["echo"] = (None if not ech or not machine or f.get("appended") else+ "same" if ech[1] == machine else "diff")+ f["reused"] = bool(machine) and machine in later+ f["is_answer"] = (bool(machine) and len(machine) < 60 and pred is not None+ and Fraction(Decimal(machine.removesuffix("..."))) == pred)+ out.append(f)+ return out+++def judge(p: dict, text: str, think: bool) -> dict:+ """Score a transcript: the answer, its correctness, and every tool call. A user turn that+ Qwen writes itself (after a tool response, instead of an assistant turn) ends the answer:+ a chat runtime would hand the turn back there."""+ full = text+ m = USER_TURN.search(text)+ lenient = None+ if m: # lenient: a boxed answer Qwen wrote inside the user turn it made up+ rest = text[m.start():]+ boxes = [_balanced(rest, b.end()) for b in re.finditer(r"\\boxed\s*\{", rest)]+ v = to_number(boxes[-1]) if boxes else None+ lenient = v if isinstance(v, Fraction) else (Fraction(Decimal(v[0])) if v else None)+ text = text[:m.start()]+ pred, src = extract_answer(text, think)+ fake_resp = False+ if pred is None and lenient is None:+ # lenient: Qwen's last number, counting <tool_response> blocks it wrote itself (with+ # <|im_start|> banned it sometimes answers inside one instead of continuing)+ own = re.sub(r"<tool_call>.*?(</tool_call>|$)", " ", MARK.sub(" ", text), flags=re.DOTALL)+ own = re.sub(r"<\|[^|]*\|>", " ", own)+ nums = NUM.findall(re.sub(r"(?<=\d),(?=\d{3}\b)", "", own))+ if nums:+ lenient, fake_resp = Fraction(Decimal(nums[-1])), True+ gold_steps = {Fraction(s) for s in p.get("steps", [])} | {Fraction(Decimal(p["gold"]))}+ want_calls = [(o, _key(o, Fraction(x), Fraction(y))) for o, x, y in p.get("calls", [])]+ rec_calls = []+ for c in parse_calls(text):+ v = true_result(c["op"], c["args"])+ ia, ib = int_arg(c["args"].get("a")), int_arg(c["args"].get("b"))+ valid = (c["op"] in tool_names() and ia is not None and ib is not None+ and (c["op"] != "sub" or ia >= ib))+ resp = (c["response"] or "").strip()+ rv = num_arg(resp) if resp else None+ e = {"op": c["op"], "a": c["args"].get("a"), "b": c["args"].get("b"), "valid": valid,+ "response": c["response"], "true": fmt(v) if v is not None else None,+ "exact": (resp == div_text(ia, ib) if c["op"] == "div" and valid and ib+ else rv is not None and v is not None and rv == v)}+ if "steps" in p and v is not None:+ e["in_gold_steps"] = v in gold_steps+ if want_calls and v is not None:+ key = _key(c["op"], num_arg(c["args"]["a"]), num_arg(c["args"]["b"]))+ e["planned"] = (c["op"], key) in want_calls+ rec_calls.append(e)+ fires = inline_fires(full, pred)+ return {"pred": fmt(pred) if pred is not None else None, "source": src, "inline": fires,+ "correct": is_correct(pred, p["gold"]), "user_turn": bool(m), "fake_resp": fake_resp,+ "correct_lenient": is_correct(pred, p["gold"]) or is_correct(lenient, p["gold"]),+ "calls": rec_calls}+++def score(p: dict, cond: str, ctx: Ctx, r: dict, seconds: float, think: bool) -> dict:+ out = r["out"]+ if parse_cond(cond)[0] in ("oracle", "oraclec") and r.get("inj_at"):+ # mark each injection like a machine run, so inline_fires reads it the same way+ cuts = [0, *r["inj_at"], len(out)]+ text = "[machine:0]".join(ctx.show(out[a:b]) for a, b in itertools.pairwise(cuts))+ else:+ text = ctx.show(out)+ return {"id": p["id"], "cond": cond, "question": p["question"], "gold": p["gold"],+ **judge(p, text, think), "stop": r["stop"],+ "n_qwen": r["n_qwen"], "n_chip": sum(1 for t in r["out"] if ctx.is_chip(t)),+ "n_out": len(r["out"]), "n_injected": r.get("n_injected", 0),+ "think_forced": r.get("think_forced", False),+ "seconds": round(seconds, 2), "text": text,+ "ids": r["out"] if ctx.chip else None} # generated ids (merged: for circuit replays)+++def rescore(args) -> None:+ """Re-judge every stored transcript with the current scoring code (no model needed)."""+ for f in sorted(Path(args.out).glob("*.jsonl")):+ data, cond = f.name.split(".")[:2]+ recs = [json.loads(line) for line in f.read_text(encoding="utf-8").splitlines() if line]+ idx = sorted({int(r["id"].rsplit("-", 1)[1]) for r in recs})+ probs = {p["id"]: p for p in load_data(data, 0, idx=idx)}+ changed = 0+ for r in recs:+ new = judge(probs[r["id"]], r["text"], "think" in parse_cond(cond)[1])+ changed += new["correct"] != r["correct"]+ r.update(new)+ f.write_text("".join(json.dumps(r, ensure_ascii=False) + "\n" for r in recs),+ encoding="utf-8")+ print(f"{f.name}: {len(recs)} records, {changed} changed verdicts")+++# ---------------------------------------------------------------------------------- running++def default_model(cond: str) -> str:+ if cond.startswith("merged"):+ return str(ROOT / "dist" / "q4-v5")+ snaps = sorted(Path.home().glob(".cache/huggingface/hub/models--Qwen--Qwen3.5-4B/snapshots/*"))+ return str(snaps[-1])+++def build_prompt(tok, question: str, cond: str, bios: list[int]) -> list[int]:+ kind, flags = parse_cond(cond)+ msgs = [{"role": "user", "content": question + SUFFIX}]+ if flags & {"nudge", "nudge2", "nudge3", "mix"}: # tools: use them; others: inline+ if "nudge3" in flags: # with division+ nudge = NUDGE_MIX3 if "mix" in flags else NUDGE3+ elif "mix" in flags:+ nudge = NUDGE_MIX+ elif "nudge2" in flags: # the inline nudge, plus: the filled-in number is final+ nudge = NUDGE2+ elif kind in TOOL_KINDS:+ nudge = NUDGE.format(names=", ".join(tool_names()))+ else:+ from chipformer.q35 import demo++ nudge = demo.NUDGE+ msgs.insert(0, {"role": "system", "content": nudge})+ ids = tok.apply_chat_template(msgs, tools=tools() if kind in TOOL_KINDS else None,+ add_generation_prompt=True, return_tensors="pt",+ enable_thinking="think" in flags)["input_ids"][0].tolist()+ return (bios if kind in BIOS_KINDS else []) + ids+++class BannedHead(torch.nn.Module):+ """The output head with -inf added at banned token ids (generation_config bad_words_ids of+ length 1), so the argmax inside the compiled step never picks them."""++ def __init__(self, head, banned: list[int], n: int):+ super().__init__()+ self.inner, self.weight = head, head.weight+ mask = torch.zeros(n, dtype=head.weight.dtype, device=head.weight.device)+ mask[banned] = float("-inf")+ self.register_buffer("mask", mask)++ def forward(self, h):+ return self.inner(h) + self.mask+++SELFTEST = [("mul", 4, 4), ("mul", 12345, 678), ("add", 987654321, 123456789),+ ("sub", 1000000, 1), ("sub", 5000, 4321), ("div", 1000000, 7), ("div", 8352, 48)]+++def selftest(bd, ctx, tok, bios) -> None:+ """Before a merged run: Qwen calls each tool on known operands and every call it makes+ must print the exact result. Catches a BIOS that does not match the circuit (the routine+ entry points are compiled into the weights) or inexact kernels."""+ sym = {"add": "+", "mul": "*", "sub": "-", "div": "/"}+ cases = [c for c in SELFTEST if c[0] in tool_names()]+ prompts = [build_prompt(tok, f"Use the {op} tool to compute {a} {sym[op]} {b}.", "merged", bios)+ for op, a, b in cases]+ got = {}+ decode_all(bd, ctx, prompts, 400, None, lambda i, r: got.__setitem__(i, r))+ bad, n = [], 0+ for i, (op, a, b) in enumerate(cases):+ for c in parse_calls(ctx.show(got[i]["out"])):+ v = true_result(c["op"], c["args"])+ if v is None:+ continue+ n += 1+ want = (div_text(int_arg(c["args"]["a"]), int_arg(c["args"]["b"]))+ if c["op"] == "div" else fmt(v)) # div truncates: 10 decimals, then "..."+ if (c["response"] or "").strip() != want:+ bad.append((c["op"], c["args"], c["response"], want))+ print(f"selftest: {n} calls, {len(bad)} wrong {bad}", flush=True)+ if bad or n == 0:+ raise SystemExit("selftest failed: the BIOS does not match this checkpoint's circuit?")+++def run(args) -> None:+ import torch+ from transformers import AutoModelForImageTextToText, AutoTokenizer++ cond = args.cond + (f"-{args.tag}" if args.tag else "")+ kind, flags = parse_cond(cond)+ think = "think" in flags+ probs = load_data(args.data, args.n, args.seed)+ out_dir = Path(args.out)+ out_dir.mkdir(parents=True, exist_ok=True)+ path = out_dir / f"{args.data}.{cond}.jsonl"+ done = set()+ if path.exists():+ done = {json.loads(line)["id"] for line in path.read_text(encoding="utf-8").splitlines()+ if line.strip()}+ todo = [p for p in probs if p["id"] not in done]+ if args.shard:+ k, m = map(int, args.shard.split("/"))+ todo = todo[k::m]+ print(f"{path}: {len(done)} done, {len(todo)} to go", flush=True)+ if not todo:+ return+ mpath = args.model or default_model(cond)+ tok = AutoTokenizer.from_pretrained(mpath)+ model = AutoModelForImageTextToText.from_pretrained(mpath, dtype=torch.bfloat16)+ model = model.cuda().eval()+ chip, bios, bios_id = None, [], None+ meta_path = Path(mpath) / "chipformer.json"+ if meta_path.exists():+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ meta = json.loads(meta_path.read_text())+ chip = (meta["chip_token_offset"], meta["chip_vocab"])+ if kind in BIOS_KINDS:+ import hashlib++ bios = [t + chip[0] for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ bios_id = f"{hashlib.sha1(bcd.ROM).hexdigest()[:10]} ({len(bcd.ROM)} B, OPS {bcd.OPS})"+ print(f"BIOS {bios_id}, boot record {len(bios)} tokens", flush=True)+ ctx = Ctx(tok, chip)+ budget = args.budget or BUDGET[think]+ if think:+ ctx.think_force = ctx.ids(THINK_FORCE)+ hook = {"pytool_inline": executor("inline", ctx),+ "pytool": executor("template", ctx)}.get(kind)+ if kind in ("oracle", "oraclec"):+ from chipformer.q35.spikes_replay import class_map++ cmeta = json.loads((Path(args.class_model) / "chipformer.json").read_text())+ fmt_ = args.oracle_fmt or (" {rc} " if kind == "oraclec" else " {r} ")+ print(f"oracle format {fmt_!r}", flush=True)+ hook = oracle(ctx, class_map(tok, cmeta), fmt_)+ banned = []+ if "nb" in flags or kind == "merged_inline":+ gc = Path(mpath) / "generation_config.json"+ bad = json.loads(gc.read_text()).get("bad_words_ids", []) if gc.exists() else []+ banned = sorted({w[0] for w in bad if len(w) == 1} | {IM_START}+ | ({ENDOFTEXT} if "ne" in flags else set()))+ model.lm_head = BannedHead(model.lm_head, banned, model.lm_head.weight.shape[0])+ print(f"banned token ids {banned}", flush=True)++ def emit(rec):+ with path.open("a", encoding="utf-8") as f:+ f.write(json.dumps(rec, ensure_ascii=False) + "\n")+ print(json.dumps({k: rec[k] for k in ("id", "gold", "pred", "correct", "stop", "n_qwen",+ "n_chip", "seconds")}+ | {"calls": [(c["op"], c["a"], c["b"], c["response"], c["exact"])+ for c in rec["calls"]]}), flush=True)++ prompts = [build_prompt(tok, p["question"], cond, bios) for p in todo]+ import torch._inductor.config++ from chipformer.q35.batch import BatchDecoder++ torch._inductor.config.shape_padding = False # its padded weight copies don't fit 24 GB++ max_len = args.max_len or (max(map(len, prompts)) + budget + (3072 if chip else 1024 if think else 512))+ max_len = -(-max_len // 1024) * 1024+ if args.fast and chip: # exact: chip layers read only their nonzero blocks, and steps on+ from chipformer.q35 import fast # chip-token inputs skip Qwen's (no-op) layers++ print("compact", fast.compact(model, qwen=False), flush=True)+ bd = BatchDecoder(model, batch=min(args.batch, len(todo)), max_len=max_len,+ eos=sorted(ctx.stop), sizes=[min(args.batch, len(todo))],+ compile=not args.no_compile,+ **({"skip_qwen": True} if args.fast and chip else {}),+ **({"freeze_chip_positions": True} if "fp" in flags else {}),+ **({"transparent_chip": True} if "tc" in flags else {}))+ if kind == "merged" and not args.no_selftest: # (tool mode)+ selftest(bd, ctx, tok, bios)+ t0 = time.time()+ print(f"batch {bd.batch}, max_len {max_len}, budget {budget}", flush=True)++ def done(i, r):+ emit(score(todo[i], cond, ctx, r, r["seconds"], think)+ | {"batch": bd.batch, "model": mpath, "bios": bios_id, "banned": banned})++ decode_all(bd, ctx, prompts, budget, hook, done)+ wall = time.time() - t0+ with (out_dir / "runs.log").open("a", encoding="utf-8") as f:+ f.write(json.dumps({"data": args.data, "cond": cond, "n": len(todo), "batch": bd.batch,+ "wall_s": round(wall, 1), "s_per_problem": round(wall / len(todo), 2),+ "model": mpath, "bios": bios_id,+ "gpu": torch.cuda.get_device_name()}) + "\n")+ print(f"done {len(todo)} in {wall:.0f}s ({wall / len(todo):.1f} s/problem)", flush=True)+++# ---------------------------------------------------------------------------------- summary++def wilson(k: int, n: int, z: float = 1.96):+ if n == 0:+ return (float("nan"),) * 2+ p = k / n+ d = 1 + z * z / n+ c = (p + z * z / (2 * n)) / d+ h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d+ return c - h, c + h+++def mcnemar(b: int, c: int) -> float:+ """Exact two-sided McNemar p-value on discordant counts."""+ n = b + c+ if n == 0:+ return 1.0+ k = min(b, c)+ p = sum(math.comb(n, i) for i in range(k + 1)) / 2**n+ return min(1.0, 2 * p)+++def load_results(out: Path) -> dict:+ res = {}+ for f in sorted(out.glob("*.jsonl")):+ data, cond = f.name.split(".")[:2]+ recs = [json.loads(line) for line in f.read_text(encoding="utf-8").splitlines() if line]+ res[(data, cond)] = {r["id"]: r for r in recs}+ return res+++def failure(r: dict) -> str:+ if r["correct"]:+ return "ok"+ if r["stop"] != "eos":+ return "cutoff"+ if any(c["valid"] and not c["exact"] for c in r["calls"]):+ return "machine_wrong"+ if any(not c["valid"] for c in r["calls"]):+ return "invalid_call"+ if r.get("user_turn"):+ return "user_turn"+ if r.get("fake_resp") and r.get("correct_lenient"):+ return "answer_in_own_tool_response"+ if r["pred"] is None:+ return "no_answer"+ return "qwen_wrong_tools" if r["calls"] else "qwen_wrong"+++PAIRS = [("merged_inline_nudge-v11", "base_nudge"), ("merged_inline_nudge_fp-v11", "base_nudge"),+ ("merged_inline_nudge_fp-v11", "merged_inline_nudge-v11"), ("merged_inline-v11", "base"),+ ("merged_inline_nudge-v11", "oracle_nudge-close"),+ ("merged_inline_fp-v11", "merged_inline-v11"), ("merged_inline_fp-v11", "base"),+ ("oracle_nudge", "base_nudge"), ("oraclec_nudge", "base_nudge"),+ ("oracle_nudge-close", "base_nudge"), ("oracle_nudge2", "base_nudge"),+ ("oracle_nudge2-close", "base_nudge"),+ ("merged_inline_nudge", "oracle_nudge"), ("merged_inline_fp", "merged_inline"), ("merged_inline_nudge_fp", "merged_inline_nudge"),+ ("merged_inline_nudge_fp", "base_nudge"), ("merged_inline", "base"), ("merged_nb-v9", "base"), ("merged_inline", "merged_nb-v9"),+ ("merged_inline-v8", "base"), ("merged_inline_nudge-v8", "base_nudge"), ("merged_inline_nudge", "base"),+ ("merged_inline_nudge", "base_nudge"), ("merged", "base"), ("merged_nb", "base"), ("merged_nb", "merged"),+ ("merged_nudge", "base"), ("merged_nb_nudge", "base"), ("pytool_nudge", "base"),+ ("merged", "pytool_inline"), ("merged", "pytool"),+ ("pytool", "base"), ("pytool_inline", "base"), ("base_think", "base"),+ ("merged", "base_think"), ("merged_plain", "base")]+# pairs whose token streams should match (same Qwen text, merged vs base weights)+SAME_TEXT = [("merged", "pytool_inline"), ("merged_plain", "base")]+++def _plain(r):+ return MARK.sub("", r["text"])+++def _wall(out: Path) -> dict:+ """Wall seconds per problem from runs.log (all rows of a batch share the GPU)."""+ tot = {}+ log = out / "runs.log"+ if log.exists():+ for line in log.read_text(encoding="utf-8").splitlines():+ x = json.loads(line)+ w, n = tot.get((x["data"], x["cond"]), (0.0, 0))+ tot[(x["data"], x["cond"])] = (w + x["wall_s"], n + x["n"])+ return {k: w / n for k, (w, n) in tot.items() if n}+++NL = chr(10)+++def _clean(gold: str) -> bool:+ g = Fraction(Decimal(gold))+ return g >= 0 and g.denominator == 1+++def _looping(fires: list[dict], n: int = 3) -> bool:+ ex = [f["expr"] for f in fires]+ return any(len(set(ex[i:i + n])) == 1 for i in range(len(ex) - n + 1))+++def summary(args) -> None:+ out = Path(args.out)+ res = load_results(out)+ wall = _wall(out)+ lines = [("Accuracy with Wilson 95% CI. tool-call rate = problems with >= 1 call. valid = "+ "the tool exists and both arguments are non-negative integers. machine exact = "+ "printed result == true result (valid calls). args ok = the call is a planned step "+ "(arith) or its result is one of the reference solution's intermediate values "+ "(gsm8k). s/prob = wall time per problem at the run's batch size.")]+ datas = sorted({d for d, _ in res})+ for data in datas:+ conds = [c for c in CONDS if (data, c) in res] + sorted(+ c for d, c in res if d == data and c not in CONDS)+ lines.append(f"\n## {data}\n")+ lines.append("| cond | N | acc | 95% CI | lenient | tool-call rate | calls/prob | "+ "valid calls | machine exact | args ok | cutoffs | Qwen tok | chip tok | "+ "s/prob |")+ lines.append("|---|---|---|---|---|---|---|---|---|---|---|---|---|---|")+ for c in conds:+ rs = list(res[(data, c)].values())+ n, k = len(rs), sum(r["correct"] for r in rs)+ lo, hi = wilson(k, n)+ calls = [x for r in rs for x in r["calls"]]+ valid = [x for x in calls if x["valid"]]+ ex = [x["exact"] for x in valid if x["response"] is not None]+ argok = [x.get("planned", x.get("in_gold_steps")) for x in calls]+ argok = [a for a in argok if a is not None]++ def pct(xs):+ return f"{100 * sum(xs) / len(xs):.0f}% ({sum(xs)}/{len(xs)})" if xs else "-"++ w = wall.get((data, c))+ lines.append(+ f"| {c} | {n} | {100 * k / n:.1f}% | {100 * lo:.0f}-{100 * hi:.0f} | "+ f"{100 * sum(r.get('correct_lenient', r['correct']) for r in rs) / n:.1f}% | "+ f"{pct([bool(r['calls']) for r in rs])} | {len(calls) / n:.2f} | "+ f"{pct([x['valid'] for x in calls])} | {pct(ex)} | {pct(argok)} | "+ f"{sum(r['stop'] != 'eos' for r in rs)} | "+ f"{sum(r['n_qwen'] for r in rs) / n:.0f} | {sum(r['n_chip'] for r in rs) / n:.0f} | "+ f"{f'{w:.1f}' if w else '-'} |")+ inl = [c for c in conds if any(r.get("inline") for r in res[(data, c)].values())]+ if inl:+ lines.append("\nInline arithmetic (machine runs outside tool calls): exact = printed "+ "== value of the \"A op B =\" as written (empty for A - B < 0); "+ "unparsed = no \"A op B =\" found before the run.\n")+ lines.append("| cond | problems with a fire | fires/prob | machine exact | "+ "Qwen digits glued on (after R / A<B fill) | Qwen re-writes a number "+ "(same / different) | unparsed | result reused later | "+ "result is the answer | answer from a fire | "+ "looping (3+ same fires in a row) |")+ lines.append("|---|---|---|---|---|---|---|---|---|---|---|")+ for c in inl:+ rs = list(res[(data, c)].values())+ fs = [f for r in rs for f in r.get("inline", [])]+ ex = [f["exact"] for f in fs if "exact" in f]+ lines.append(+ f"| {c} | {sum(bool(r.get('inline')) for r in rs)}/{len(rs)} | "+ f"{len(fs) / len(rs):.2f} | {sum(ex)}/{len(ex)} | "+ f"{sum(bool(f.get('appended')) and f.get('true', '0')[0] != '-' for f in fs)} / "+ f"{sum(bool(f.get('appended')) and f.get('true', '0')[0] == '-' for f in fs)} | "+ f"{sum(f.get('echo') == 'same' for f in fs)} / "+ f"{sum(f.get('echo') == 'diff' for f in fs)} | "+ f"{sum('exact' not in f for f in fs)} | {sum(f['reused'] for f in fs)}/{len(fs)} | "+ f"{sum(f['is_answer'] for f in fs)}/{len(fs)} | "+ f"{sum(any(f['is_answer'] for f in r.get('inline', [])) for r in rs)}/{len(rs)} | "+ f"{sum(_looping(r.get('inline', [])) for r in rs)} |")+ lines.append("\nFailures (wrong answers by cause):\n")+ for c in conds:+ cnt = _count(failure(r) for r in res[(data, c)].values())+ lines.append(f"- {c}: " + (", ".join(f"{k} {v}" for k, v in sorted(cnt.items())+ if k != "ok") or "none"))+ if data == "gsmhard":+ lines.append(NL + "GSM-Hard subsets: many targets are negative or fractional (label "+ "artifacts of swapping in large numbers); 'clean' = non-negative integer "+ "target." + NL)+ lines.append("| cond | clean acc | clean 95% CI | other acc |")+ lines.append("|---|---|---|---|")+ for c in conds:+ rs = list(res[(data, c)].values())+ cl = [r["correct"] for r in rs if _clean(r["gold"])]+ ot = [r["correct"] for r in rs if not _clean(r["gold"])]+ lo, hi = wilson(sum(cl), len(cl))+ lines.append(f"| {c} | {sum(cl)}/{len(cl)} = {100 * sum(cl) / max(1, len(cl)):.1f}% "+ f"| {100 * lo:.0f}-{100 * hi:.0f} | {sum(ot)}/{len(ot)} |")+ if data == "arith":+ lines.append("\n| cond | " + " | ".join(ARITH_KINDS) + " |")+ lines.append("|---|" + "---|" * len(ARITH_KINDS))+ for c in conds:+ accs = []+ for j, kind in enumerate(ARITH_KINDS):+ rs = [r for r in res[(data, c)].values()+ if int(r["id"].split("-")[1]) % len(ARITH_KINDS) == j]+ accs.append(f"{sum(r['correct'] for r in rs)}/{len(rs)}")+ lines.append(f"| {c} | " + " | ".join(accs) + " |")+ rows = []+ for a, b in PAIRS:+ if (data, a) not in res or (data, b) not in res:+ continue+ ids = sorted(set(res[(data, a)]) & set(res[(data, b)]))+ if not ids:+ continue+ ra, rb = res[(data, a)], res[(data, b)]+ xa = [ra[i]["correct"] for i in ids]+ xb = [rb[i]["correct"] for i in ids]+ n = len(ids)+ bo = sum(x and not y for x, y in zip(xa, xb))+ co = sum(y and not x for x, y in zip(xa, xb))+ d = (bo - co) / n+ se = math.sqrt(max(0.0, (bo + co) / n - d * d) / n)+ same = "-"+ if (a, b) in SAME_TEXT:+ same = f"{sum(_plain(ra[i]) == _plain(rb[i]) for i in ids)}/{n}"+ rows.append(f"| {a} vs {b} | {n} | {100 * sum(xa) / n:.1f}% | "+ f"{100 * sum(xb) / n:.1f}% | {100 * d:+.1f} | "+ f"{100 * (d - 1.96 * se):+.1f}..{100 * (d + 1.96 * se):+.1f} | {bo} | {co} | "+ f"{mcnemar(bo, co):.3g} | {same} |")+ if rows:+ lines.append("\n| paired (A vs B) | N | A | B | A-B (pts) | 95% CI | A only | B only | "+ "McNemar p | same text |")+ lines.append("|---|---|---|---|---|---|---|---|---|---|")+ lines += rows+ text = "\n".join(lines)+ print(text)+ (out / "summary.md").write_text(text + "\n", encoding="utf-8")+++def _count(xs):+ d = {}+ for x in xs:+ d[x] = d.get(x, 0) + 1+ return d+++def main() -> None:+ ap = argparse.ArgumentParser()+ sub = ap.add_subparsers(dest="cmd", required=True)+ r = sub.add_parser("run")+ r.add_argument("cond", type=cond_arg, help=f"one of {', '.join(CONDS)} (or KIND_FLAGS)")+ r.add_argument("data", choices=("gsm8k", "gsmhard", "arith"))+ r.add_argument("--n", type=int, default=100)+ r.add_argument("--seed", type=int, default=0)+ r.add_argument("--model")+ r.add_argument("--out", default=str(ROOT / "dist" / "evals"))+ r.add_argument("--budget", type=int, help="Qwen-written tokens (default 2048, thinking 8192)")+ r.add_argument("--max-len", type=int, help="cache length (default: prompt + budget + room)")+ r.add_argument("--batch", type=int, default=8)+ r.add_argument("--shard", help="k/m: run every m-th remaining problem from k")+ r.add_argument("--no-compile", action="store_true")+ r.add_argument("--no-selftest", action="store_true")+ r.add_argument("--fast", action="store_true", help="merged: compact chip layers + skip_qwen")+ r.add_argument("--tag", help="model tag appended to the condition name (e.g. v6)")+ r.add_argument("--class-model", default=str(ROOT / "dist" / "q4-v9"),+ help="oracle: merged checkpoint whose text classes define the inline grammar")+ r.add_argument("--oracle-fmt", help="oracle: injected text, {r} = result, {rc} = with commas "+ "(default ' {r} ', oraclec ' {rc} '); name the variant with --tag")+ for name in ("summary", "rescore"):+ s = sub.add_parser(name)+ s.add_argument("--out", default=str(ROOT / "dist" / "evals"))+ args = ap.parse_args()+ {"run": run, "summary": summary, "rescore": rescore}[args.cmd](args)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/fast.py b/experimental/riscv/chipformer/q35/fast.pynew file mode 100644index 0000000..f16d71a--- /dev/null+++ b/experimental/riscv/chipformer/q35/fast.py@@ -0,0 +1,295 @@+"""Fast greedy decoding of a q35 checkpoint with stock modules: a static cache plus one+compiled (CUDA-graph) single-token step that also takes the argmax on the GPU.++Pure decoding: every token the machine writes is the model's own greedy prediction; the host+(chipformer.session.Session) only injects the boot prompt, TICKs and key writes, exactly as with+any other model. Long sessions are bounded by ``max_len``; past it the session re-boots from its+mirror of the cells (the trace's own rollover record), which also resets the cache.++ runner = Runner(model, max_len=65536)+ s = runner.session(rom, quirks)+ s.tick(); s.step(); ...+"""++import json+from pathlib import Path++import torch+import torch.nn.functional as F++from chipformer.session import Session+from chipformer.trace import boot_tokens++# --- weight blocks ---------------------------------------------------------------------------+#+# Most of a merged checkpoint's weight bytes are exact zeros: the chip layers use a few hundred+# of their 3584+ input dims and a fraction of their units, Qwen's widened matrices never touch+# the chip dims (bar one mute column), and the output head reads no chip dim past the output+# code. Decoding at batch 1 is bound by reading weights, so reading only each matrix's nonzero+# block is the largest exact speedup there is. The checkpoint itself stays as it is; this only+# changes which kernels run.+++def exact_matmuls() -> None:+ """Make cuBLAS reduce split-K partial sums in fp32 (torch's default allows bf16).++ The circuit's GEMMs are exact only if their fp32 accumulation is: with bf16 partial sums a+ split-K kernel rounds e.g. 128 - 33280 to -33280 before adding 65536. Which kernel cuBLAS+ picks depends on the shapes (batch size, a compacted K), so every exact path sets this.+ """+ torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False+++exact_matmuls() # importing this module makes the process's matmuls safe for the circuit+++def nonzero_cols(w: torch.Tensor, chunk: int = 16384) -> torch.Tensor:+ """``(w != 0).any(0)`` without materializing the whole mask (output heads are large)."""+ nz = torch.zeros(w.shape[1], dtype=torch.bool, device=w.device)+ for i in range(0, w.shape[0], chunk):+ nz |= (w[i:i + chunk] != 0).any(0)+ return nz+++def _hull(mask: torch.Tensor) -> tuple[int, int]:+ idx = mask.nonzero().flatten()+ return (int(idx[0]), int(idx[-1]) + 1) if len(idx) else (0, 0)+++class SparseLinear(torch.nn.Module):+ """``F.linear(x, W)`` for a weight that is zero outside the block ``W[r0:r1, c0:c1]``;+ reads only the block. Equal to the dense product up to the order of the accumulation over+ the block's columns, so exactly equal wherever that sum is exact (all chip arithmetic)."""++ def __init__(self, w: torch.Tensor, rows=None, cols=None, block=None, dense_above=None):+ super().__init__()+ nz = w != 0 if rows is None or cols is None else None+ self.out_features, self.in_features = w.shape+ self.r0, self.r1 = rows or _hull(nz.any(1))+ self.c0, self.c1 = cols or _hull(nz.any(0))+ # dense_above: inputs of more rows than this (prefills) use the dense weight, so they+ # stay bit-identical to the stock model; the block is then a view (no extra memory)+ self.dense_above = dense_above+ self.dense = w if dense_above is not None else None+ if block is None:+ block = w[self.r0:self.r1, self.c0:self.c1]+ if dense_above is None:+ block = torch.nn.Parameter(block.contiguous(), requires_grad=False)+ self.weight = block++ def forward(self, x):+ if self.dense is not None and x.numel() > self.dense_above * x.shape[-1]:+ return F.linear(x, self.dense)+ if self.r1 == self.r0 or self.c1 == self.c0:+ return x.new_zeros(*x.shape[:-1], self.out_features)+ y = F.linear(x[..., self.c0:self.c1], self.weight)+ if self.r0 or self.r1 != self.out_features:+ y = F.pad(y, (self.r0, self.out_features - self.r1))+ return y++ def extra_repr(self):+ return (f"{self.out_features}x{self.in_features} -> rows {self.r0}:{self.r1}, "+ f"cols {self.c0}:{self.c1}")+++class SparseEmbedding(torch.nn.Module):+ """Embedding whose table is zero outside columns ``c0:c1`` (shared with the output head)."""++ def __init__(self, block: torch.nn.Parameter, c0: int, dim: int):+ super().__init__()+ self.weight, self.c0, self.dim = block, c0, dim++ def forward(self, ids):+ e = F.embedding(ids, self.weight)+ return F.pad(e, (self.c0, self.dim - self.c0 - self.weight.shape[1]))+++def chip_meta(model) -> dict | None:+ """The checkpoint's chipformer.json (merged checkpoints only)."""+ meta = Path(getattr(model.config, "_name_or_path", "") or ".") / "chipformer.json"+ return json.loads(meta.read_text()) if meta.exists() else None+++def _n_chip_layers(model) -> int:+ meta = chip_meta(model)+ if meta is not None:+ return meta["chip_layers"]+ cfg = getattr(model.config, "text_config", model.config)+ return cfg.num_hidden_layers # a chip-only checkpoint (tiny.py)++[email protected]_grad()+def compact(model, qwen: bool | str = True, n_chip: int | None = None) -> dict:+ """Replace the text stack's linear maps by their nonzero blocks, in place.++ ``qwen=False`` touches only the chip layers, whose arithmetic is exact, so every logit stays+ bit-identical. ``qwen="rows"`` also drops the zero output rows of Qwen's matrices (the chip+ dims they never write) in decode steps (at most 16 rows): each output is the same dot+ product, so it stays bit-identical too (dist/spikes/b19_qwenblocks.py). ``qwen=True``+ compacts Qwen's layers fully, plus the embedding and the output head: the dropped terms are+ exact zeros, but the shorter sums round differently in Qwen's own (inexact) text arithmetic,+ like any change of GEMM kernel. Returns weight counts.+ """+ exact_matmuls()+ inner = model.model+ lm = getattr(inner, "language_model", inner)+ n_chip = _n_chip_layers(model) if n_chip is None else n_chip+ before = after = 0+ heads = []+ for i, layer in enumerate(lm.layers):+ if i >= n_chip and not qwen:+ continue+ if (i < n_chip and hasattr(layer, "linear_attn")+ and isinstance(layer.linear_attn.out_proj, torch.nn.Linear)):+ heads.append(prune_heads(layer.linear_attn))+ for parent in list(layer.modules()):+ for name, child in list(parent.named_children()):+ if isinstance(child, torch.nn.Linear) and child.bias is None:+ if i >= n_chip and qwen == "rows":+ r0, r1 = _hull((child.weight != 0).any(1))+ if (r0, r1) == (0, child.out_features):+ continue+ s = SparseLinear(child.weight, rows=(r0, r1),+ cols=(0, child.in_features), dense_above=16)+ else:+ s = SparseLinear(child.weight)+ before += child.weight.numel()+ after += s.weight.numel()+ setattr(parent, name, s)+ if qwen is True and isinstance(model.lm_head, torch.nn.Linear):+ w = model.lm_head.weight+ emb = lm.embed_tokens.weight+ c0, c1 = _hull(nonzero_cols(w))+ if w.data_ptr() == emb.data_ptr():+ ce0, ce1 = _hull((emb != 0).any(0))+ c0, c1 = min(c0, ce0), max(c1, ce1)+ block = torch.nn.Parameter(w[:, c0:c1].contiguous(), requires_grad=False)+ before += w.numel()+ after += block.numel()+ if w.data_ptr() == emb.data_ptr():+ lm.embed_tokens = SparseEmbedding(block, c0, w.shape[1])+ model.lm_head = SparseLinear(w, rows=(0, w.shape[0]), cols=(c0, c1), block=block)+ torch.cuda.empty_cache()+ return {"dense_weights": before, "compact_weights": after, "chip_deltanet_heads": heads}++[email protected]_grad()+def prune_heads(la) -> tuple[int, int]:+ """Drop the value heads of a Gated DeltaNet whose output columns are all zero (with the key+ heads only they read). Heads are independent, so the kept heads compute exactly what they+ did; the recurrent state shrinks with them. Returns (kept, original) value heads."""+ nv, nk, dk, dv = la.num_v_heads, la.num_k_heads, la.head_k_dim, la.head_v_dim+ r = nv // nk+ out = la.out_proj.weight+ used = (out.view(out.shape[0], nv, dv) != 0).any(0).any(-1).view(nk, r).any(-1)+ k = used.nonzero().flatten()+ if len(k) == nk:+ return nv, nv+ if len(k) == 0:+ k = torch.zeros(1, dtype=torch.long, device=out.device) # keep one (inert) head+ v = (k[:, None] * r + torch.arange(r, device=k.device)).flatten()+ kd = (k[:, None] * dk + torch.arange(dk, device=k.device)).flatten()+ vd = (v[:, None] * dv + torch.arange(dv, device=k.device)).flatten()+ qkv = torch.cat([kd, la.key_dim + kd, 2 * la.key_dim + vd])++ def keep(mod, idx, dim=0):+ w = mod.weight.index_select(dim, idx).contiguous()+ mod.weight = torch.nn.Parameter(w, requires_grad=False)+ if isinstance(mod, torch.nn.Linear):+ mod.out_features, mod.in_features = w.shape++ keep(la.in_proj_qkv, qkv)+ keep(la.conv1d, qkv)+ la.conv1d.in_channels = la.conv1d.out_channels = la.conv1d.groups = len(qkv)+ keep(la.in_proj_z, vd)+ keep(la.in_proj_a, v)+ keep(la.in_proj_b, v)+ keep(la.out_proj, vd, 1)+ la.A_log = torch.nn.Parameter(la.A_log[v].contiguous(), requires_grad=False)+ la.dt_bias = torch.nn.Parameter(la.dt_bias[v].contiguous(), requires_grad=False)+ la.num_k_heads, la.num_v_heads = len(k), len(v)+ la.key_dim, la.value_dim = len(k) * dk, len(v) * dv+ la.conv_dim = 2 * la.key_dim + la.value_dim+ return len(v), nv+++class Runner:+ def __init__(self, model, max_len: int = 65536, token_offset: int = 0, n_chip: int = 8708,+ compile: bool = True):+ from transformers import StaticCache++ self.model = model+ self.off = token_offset+ self.n_chip = n_chip+ self.max_len = max_len+ self.dev = next(model.parameters()).device+ cfg = getattr(model.config, "text_config", model.config)+ self.cache = StaticCache(config=cfg, max_cache_len=max_len)+ # Run the text stack directly (a multimodal wrapper would recompute vision-aware rope+ # indices every step) and score only the chip rows of the tied output head: in a machine+ # record the predicted token is always a chip token (logit ~240 vs ~0 for text), so the+ # greedy choice is the same as over the full vocabulary.+ inner = model.model+ self.lm = getattr(inner, "language_model", inner)+ lm_head = model.lm_head+ self.head = lm_head.weight[token_offset:token_offset + n_chip]+ self.cols = (slice(lm_head.c0, lm_head.c1) if isinstance(lm_head, SparseLinear)+ else slice(None))+ self.pos = 0+ self.tok = torch.zeros(1, 1, dtype=torch.long, device=self.dev)+ self.cpos = torch.zeros(1, dtype=torch.long, device=self.dev)+ self._step = torch.compile(self._forward, mode="reduce-overhead", fullgraph=False) \+ if compile else self._forward++ def _forward(self, tok, cpos):+ h = self.lm(input_ids=tok, past_key_values=self.cache, cache_position=cpos,+ use_cache=True).last_hidden_state[0, -1]+ return (self.head @ h[self.cols]).argmax(-1)++ @torch.no_grad()+ def prefill(self, toks: list[int]) -> int:+ x = torch.tensor([[t + self.off for t in toks]], device=self.dev)+ cpos = torch.arange(self.pos, self.pos + len(toks), device=self.dev)+ h = self.lm(input_ids=x, past_key_values=self.cache, cache_position=cpos,+ use_cache=True).last_hidden_state[0, -1]+ self.pos += len(toks)+ return int((self.head @ h[self.cols]).argmax())++ @torch.no_grad()+ def feed(self, token: int) -> int:+ self.tok.fill_(token + self.off)+ self.cpos.fill_(self.pos)+ nxt = self._step(self.tok, self.cpos)+ self.pos += 1+ return int(nxt)++ def reset(self) -> None:+ self.cache.reset()+ self.pos = 0++ def session(self, rom: bytes, quirks=None) -> Session:+ """A Session whose boot prompt goes through one prefill; rolls over before max_len."""+ runner = self+ boot: list[int] = []++ class Fast(Session):+ def _generate_record(self):+ if runner.pos > runner.max_len - 200: # room for the longest record+ runner.reset()+ dump = boot_tokens(self.cells)+ self.tokens += dump+ self.pred = runner.prefill(dump)+ return super()._generate_record()++ collecting = [True]++ def feed(tok: int) -> int:+ if collecting[0]:+ boot.append(tok)+ return -1+ return runner.feed(tok)++ s = Fast(feed, rom, quirks)+ collecting[0] = False+ s.pred = runner.prefill(boot)+ return sdiff --git a/experimental/riscv/chipformer/q35/host.py b/experimental/riscv/chipformer/q35/host.pynew file mode 100644index 0000000..9e68ea4--- /dev/null+++ b/experimental/riscv/chipformer/q35/host.py@@ -0,0 +1,61 @@+"""Drive a q35 checkpoint with chipformer.session.Session (prefill + cached decode).++``Feeder.feed(token)`` processes one position through the HF model with its cache and returns the+greedy next token. Prompts can be pushed in one prefill with ``prefill``; the host's TICK and key+injections then go through the normal decode path.+"""++import torch++from chipformer.session import Session+++class Feeder:+ def __init__(self, model, token_ids=None):+ """``token_ids`` maps a chip token to the model's vocabulary (identity for tiny models)."""+ self.model = model+ self.ids = token_ids+ self.chip = None if token_ids is None else torch.tensor(token_ids)+ self.cache = None+ self.pending: list[int] = []+ self.n = 0++ def _id(self, t: int) -> int:+ return t if self.ids is None else self.ids[t]++ def _run(self, toks: list[int]) -> int:+ x = torch.tensor([[self._id(t) for t in toks]], device=self.model.device)+ with torch.no_grad():+ out = self.model(input_ids=x, past_key_values=self.cache, use_cache=True)+ self.cache = out.past_key_values+ self.n += len(toks)+ logits = out.logits[0, -1].float()+ if self.chip is not None:+ return int(logits[self.chip].argmax())+ return int(logits.argmax())++ def feed(self, token: int) -> int:+ return self._run([token])++ def reset(self) -> None:+ self.cache = None+ self.n = 0+++def session(model, rom: bytes, quirks=None, token_ids=None) -> Session:+ """A Session whose boot prompt goes through one prefill, then decodes token by token."""+ f = Feeder(model, token_ids)+ boot: list[int] = []++ def feed(tok: int) -> int:+ if boot is not None and collecting[0]:+ boot.append(tok)+ return -1+ return f.feed(tok)++ collecting = [True]+ s = Session(feed, rom, quirks)+ collecting[0] = False+ s.pred = f._run(boot)+ s.feeder = f+ return sdiff --git a/experimental/riscv/chipformer/q35/ir.py b/experimental/riscv/chipformer/q35/ir.pynew file mode 100644index 0000000..63e8d18--- /dev/null+++ b/experimental/riscv/chipformer/q35/ir.py@@ -0,0 +1,510 @@+"""Circuit graph for the Qwen3.5 target: SwiGLU units and Gated DeltaNet heads.++Like ``chipformer.dsl.graph``, every value is a scalar dim of the residual stream and linear+forms (``Expr``) are free. The node types are the two sublayers of a Qwen3.5 linear-attention+layer, described closely enough that the emitter only has to scale weights:++``FFN`` a sum of units ``coef * a * relu(b)``, emitted as ``silu(K b) * (a / K)``. silu equals+ relu on every value the circuit produces: 0, >= 18 (sigmoid rounds to 1) or <= -104+ (sigmoid underflows to 0).+``Head`` one Gated DeltaNet key head with its value head(s), channel by channel: q/k/v+ channels are linear forms with conv taps over the last 4 tokens followed by silu;+ keys must be one-hot (an exact overwrite of one state row) or zero, queries+ multi-hot or zero; ``a`` picks the per-token decay (keep / halve / clear); ``z``+ gates each output column. Rows carry a ballast in their ``ref`` columns that+ dominates the per-head output RMSNorm, so reads come back exactly rescaled.++Values live in IR units: a residual value x reaches every sublayer as exactly x * 2^-r because+chip tokens carry a large ballast (see spike.py), and the emitter folds 2^r into the weights.+The constant ONE is that ballast. On text tokens every chip dim, ONE included, is 0, so every+node outputs exactly 0 there.++Tokens are ints: chip tokens, text classes, and (for the object store) a text class with a Qwen+token id above it (``qid << 16 | cls``). Token features see ``t & 0xFFFF`` unless declared+``full``, so only features that read Qwen's own token id (the store's id code) can tell two text+tokens of one class apart.++Exactness is checked numerically: ``evaluate`` runs a token sequence through the graph and+asserts that every intermediate the hardware materializes is a bf16 value, that silu inputs are+in its exact range, and that keys, queries and decays have the allowed shapes.+"""++from dataclasses import dataclass, field++import numpy as np++from chipformer.dsl.graph import ONE, Expr, Num, as_expr++K = 128 # silu gate scale: silu(K n) = K relu(n) for integers n+A_REF = 2**24 # ballast value of a state row (two columns)+A_BIT = 2**7 # value of a set bit in a state row: (A_BIT / A_REF)^2 * 126 is absorbed+REF_COLS = (0, 1)+DECAY = {0.0: 1.0, 110.0: 0.5, 238.0: 0.0} # a -> exp(g) with dt_bias = -110, A_log = 0+HD = 128 # key/value head dim+TOKEN_MASK = 0xFFFF # the part of a token that ordinary features see (see above)+++@dataclass+class Dim:+ name: str+ kind: str # "one" | "token" | "ffn" | "head"+ level: int+++@dataclass+class FFN:+ name: str+ terms: list[tuple[Num, Expr, Expr]] # coef * a * relu(b)+ out: int+ level: int+ into: bool = False # out is an existing dim that this node adds to+++@dataclass+class Chan:+ expr: Expr+ taps: tuple[float, float, float, float] = (1.0, 0.0, 0.0, 0.0) # x_t, x_{t-1}, ...+++@dataclass+class VHead:+ """A further value head of a key head (Qwen3.5-4B has two per key head): the same queries+ and keys, its own values, decay, write gate and output gates, so it holds a second table+ with the same rows."""++ v: dict[int, Chan]+ a: Expr+ z: dict[int, Expr]+ out: list[tuple[int, int, float]]+ beta: Expr | None = None # write gate (0/1); None: always 1+ into: bool = False # outputs add onto existing dims+++@dataclass+class Head:+ name: str+ q: dict[int, Chan] # key-dim channel -> query+ k: dict[int, Chan]+ v: dict[int, Chan] # value column -> value (columns REF_COLS are the ballast)+ a: Expr # decay selector, see DECAY+ z: dict[int, Expr] # value column -> output gate silu(z)+ out: list[tuple[int, int, float]] # (value column, residual dim, weight)+ level: int+ n_cols: int = HD+ tiny: float = 0.0 # outputs below this may round (the counter's 2^-j for j >> JMAX)+ beta: Expr | None = None # write gate of the first value head (0/1); None: always 1+ into: bool = False # outputs add onto existing dims (several heads summed into one dim)+ vx: list[VHead] = field(default_factory=list) # further value heads of this key head+ # initial state (a snapshot the host injects): per value head, row -> state row. "bit" rows+ # hold 0 / A_BIT data; "byte" rows (read-only tables) hold integers 0..255 and must be read+ # with a 2-hot query (see kernel_check); "decay" rows hold A_BIT * 2^-n (a halving counter)+ init: list | None = None+ levels: str = "bit"++ def vheads(self) -> list[VHead]:+ return [VHead(self.v, self.a, self.z, self.out, self.beta, self.into), *self.vx]+++@dataclass+class Target:+ """Per-layer capacity of the host model."""++ key_heads: int = 16+ value_heads: int = 16+ intermediate: int = 3584++ @property+ def cols(self) -> int:+ return HD # value columns a head uses (one value head; the output norm is per head)+++@dataclass+class Graph:+ target: Target = field(default_factory=Target)++ def __post_init__(self) -> None:+ self.dims: list[Dim] = [Dim("one", "one", 0)]+ self.token_fns: dict[int, object] = {}+ self.full_fns: set[int] = set() # features that see the whole token (TOKEN_MASK)+ self.nodes: list[FFN | Head] = []+ self.one = Expr({ONE: 1})+ self.units: dict[int, int] = {} # sublayer -> FFN units used+ self.heads: dict[int, int] = {} # sublayer -> key heads used++ # --- dims -----------------------------------------------------------------++ def token_feature(self, name: str, fn, full: bool = False) -> Expr:+ d = self._dim(name, "token", 0)+ self.token_fns[d] = fn+ if full:+ self.full_fns.add(d)+ return Expr({d: 1})++ def feature_value(self, d: int, t: int):+ fn = self.token_fns[d]+ return fn(t) if d in self.full_fns else fn(t & TOKEN_MASK)++ def _dim(self, name: str, kind: str, level: int) -> int:+ self.dims.append(Dim(name, kind, level))+ return len(self.dims) - 1++ def level(self, *exprs: Expr) -> int:+ return max((self.dims[d].level for e in exprs for d in e.terms), default=0)++ # --- placement ------------------------------------------------------------++ def _place(self, lvl: int, odd: bool, need: int, used: dict[int, int], cap: int) -> int:+ sub = lvl + 1+ while (sub % 2 == 1) != odd or used.get(sub, 0) + need > cap:+ sub += 1+ used[sub] = used.get(sub, 0) + need+ return sub++ # --- nodes ----------------------------------------------------------------++ def ffn(self, terms, name: str = "ffn", into: Expr | None = None, min_level: int = 0) -> Expr:+ terms = [(c, as_expr(a), as_expr(b)) for c, a, b in terms if c != 0]+ lvl = max(self.level(*(e for _, a, b in terms for e in (a, b))), min_level)+ sub = self._place(lvl, False, len(terms), self.units, self.target.intermediate)+ if into is not None:+ (d,) = into.terms+ self.dims[d].level = max(self.dims[d].level, sub)+ else:+ d = self._dim(name, "ffn", sub)+ self.nodes.append(FFN(name, terms, d, sub, into is not None))+ return Expr({d: 1})++ def head(self, name: str, q: dict, k: dict, v: dict, a: "Expr | Num", z: dict,+ outs: list[tuple[int, float]], min_level: int = 0, tiny: float = 0.0,+ beta=None, into: list | None = None, vx: list | None = None) -> list[Expr]:+ """Add a head; ``outs`` lists (value column, weight) pairs, one new dim each (or, with+ ``into``, onto these existing dims). ``vx``: further value heads, dicts with keys v, a,+ z, outs and optionally beta / into, sharing this head's queries and keys; their output+ dims follow the first value head's in the returned list."""+ def chans(ch):+ return {i: c if isinstance(c, Chan) else Chan(as_expr(c)) for i, c in ch.items()}++ q, k, v = chans(q), chans(k), chans(v)+ a = as_expr(a)+ z = {i: as_expr(e) for i, e in z.items()}+ beta = None if beta is None else as_expr(beta)+ vx = [dict(x) for x in (vx or [])]+ assert len(vx) < self.target.value_heads // self.target.key_heads or not vx, \+ "more value heads than the key head has"+ for x in vx:+ x["v"], x["a"] = chans(x["v"]), as_expr(x["a"])+ x["z"] = {i: as_expr(e) for i, e in x["z"].items()}+ x["beta"] = None if x.get("beta") is None else as_expr(x["beta"])+ exprs = [c.expr for ch in (q, k, v) for c in ch.values()] + [a, *z.values()]+ exprs += [] if beta is None else [beta]+ for x in vx:+ exprs += [c.expr for c in x["v"].values()] + [x["a"], *x["z"].values()]+ exprs += [] if x["beta"] is None else [x["beta"]]+ lvl = max(self.level(*exprs), min_level)+ sub = self._place(lvl, True, 1, self.heads, self.target.key_heads)++ def out_dims(tag, outs_, into_):+ if into_ is None:+ return [self._dim(f"{name}{tag}[{col}]", "head", sub) for col, _ in outs_]+ assert len(into_) == len(outs_)+ dims_ = []+ for e in into_:+ (d_,) = e.terms+ self.dims[d_].level = max(self.dims[d_].level, sub)+ dims_.append(d_)+ return dims_++ dims = out_dims("", outs, into)+ extra, ret = [], [Expr({d: 1}) for d in dims]+ for i, x in enumerate(vx):+ if isinstance(x.get("into"), str) and x["into"] == "main": # onto the first's dims+ x["into"] = [Expr({d: 1}) for d in dims]+ xd = out_dims(f".v{i + 1}", x["outs"], x.get("into"))+ extra.append(VHead(x["v"], x["a"], x["z"],+ [(col, d, w) for (col, w), d in zip(x["outs"], xd)], x["beta"],+ x.get("into") is not None))+ ret += [Expr({d: 1}) for d in xd]+ self.nodes.append(Head(name, q, k, v, a, z,+ [(col, d, w) for (col, w), d in zip(outs, dims)], sub,+ self.target.cols, tiny, beta, into is not None, extra))+ return ret++ def n_layers(self) -> int:+ top = max((n.level for n in self.nodes), default=0)+ return (top + 1) // 2++ def stats(self) -> dict[str, int]:+ return {+ "layers": self.n_layers(),+ "dims": len(self.dims),+ "heads": sum(isinstance(n, Head) for n in self.nodes),+ "ffn_units": sum(len(n.terms) for n in self.nodes if isinstance(n, FFN)),+ "max_units": max(self.units.values(), default=0),+ "max_heads": max(self.heads.values(), default=0),+ }+++# --- exactness checks -------------------------------------------------------------+#+# The evaluator computes every node twice: ideally (exact float64) and as the bf16 hardware does+# (every materialized tensor rounded to bf16, sums accumulated in fp32). The two must agree on+# every node output; fp32 hardware has no rounding and computes the ideal values.+++class ExactnessError(AssertionError):+ pass+++def bf16(x: np.ndarray) -> np.ndarray:+ """Round to the nearest bf16 value (ties to even), returned as float64."""+ f = np.asarray(x, dtype=np.float32).copy()+ u = f.view(np.uint32)+ u += 0x7FFF + ((u >> 16) & 1)+ u &= 0xFFFF0000+ return f.astype(np.float64)+++def is_bf16(x: np.ndarray) -> np.ndarray:+ return (bf16(x) == x) & np.isfinite(x)+++def require(ok, what: str, x) -> None:+ ok, x = np.asarray(ok), np.asarray(x)+ if not np.all(ok):+ t = np.unravel_index(int(np.argmin(ok)), ok.shape)+ raise ExactnessError(f"{what}: bad value {x[t]!r} at {t}")+++def silu_ok(x: np.ndarray) -> np.ndarray:+ """silu(x) == relu(x) exactly (fp32 and bf16)."""+ return (x == 0) | (x >= 18) | (x <= -104)+++# --- evaluation ---------------------------------------------------------------------+++def ev(e: Expr, m: np.ndarray) -> np.ndarray:+ if not e.terms:+ return np.zeros(m.shape[0])+ idx = np.fromiter(e.terms.keys(), dtype=np.int64, count=len(e.terms))+ coef = np.array([float(c) for c in e.terms.values()])+ return m[:, idx] @ coef+++def conv(c: Chan, m: np.ndarray, check: bool, carry: dict | None = None,+ key=None) -> np.ndarray:+ """silu(conv(projection)) as computed in bf16; must equal the exact value. With ``carry``+ the projection's last values of the previous chunk come first (see evaluate)."""+ x = ev(c.expr, m)+ if check:+ require(is_bf16(x), "projection", x)+ n = len(x)+ if carry is not None and any(c.taps[1:]):+ prev = carry.get(key, np.zeros(0))+ xx = np.concatenate([prev, x])+ carry[key] = xx[-3:].copy()+ else:+ xx = x+ p = len(xx) - n+ out = np.zeros(len(xx))+ for lag, w in enumerate(c.taps):+ if w:+ out[lag:] += w * xx[: len(xx) - lag]+ out = out[p:]+ if check:+ require(is_bf16(out) & silu_ok(out), "conv output (silu input)", out)+ return np.maximum(out, 0.0)+++def evaluate(g: Graph, tokens: list[int], check: bool = True,+ carry: dict | None = None, kernel: bool = False) -> np.ndarray:+ """Residual stream after every node: (n_tokens, n_dims), exact in float64.++ Text tokens (ids outside the chip vocabulary) can be passed as -1: all chip dims are 0.+ ``carry`` (a dict, empty at the start of a stream) evaluates a long stream chunk by chunk:+ head states and conv windows pass from one call to the next. ``kernel`` also replays every+ head read the way the HF torch kernels compute it (kernel_check).+ """+ n = len(tokens)+ m = np.zeros((n, len(g.dims)))+ toks = np.asarray(tokens, dtype=np.int64)+ chip = toks >= 0+ m[:, ONE] = chip+ uniq, inv = np.unique(toks, return_inverse=True)+ for d in g.token_fns:+ col = np.array([float(g.feature_value(d, int(u))) if u >= 0 else 0.0 for u in uniq])[inv]+ if check:+ require(is_bf16(col), f"token feature {g.dims[d].name}", col)+ m[:, d] = col+ for i, node in enumerate(g.nodes):+ try:+ if isinstance(node, FFN):+ eval_ffn(node, m, chip, check)+ else:+ eval_head(node, m, chip, check, carry, i, kernel)+ except ExactnessError as e:+ raise ExactnessError(f"{node.name}: {e}") from None+ if check:+ # every RMSNorm divides by the ballast alone: other residual values must vanish next to+ # it (x^2 far below m * 2^80)+ big = np.abs(m[:, 1:]).max(0) if m.shape[0] else np.zeros(0)+ if len(big) and big.max() > RESIDUAL_MAX:+ d = int(big.argmax()) + 1+ raise ExactnessError(f"{g.dims[d].name}: residual value {big[d - 1]!r} would move "+ f"the norms (> {RESIDUAL_MAX})")+ return m+++RESIDUAL_MAX = 2.0**28 # x^2 <= 2^56 is lost in fp32 next to the ballast's m * 2^80+++def eval_ffn(node: FFN, m: np.ndarray, chip: np.ndarray, check: bool) -> None:+ ideal = np.zeros(m.shape[0])+ hw = np.zeros(m.shape[0])+ for c, a, b in node.terms:+ bv, av = ev(b, m), ev(a, m)+ ideal += float(c) * av * np.maximum(bv, 0.0)+ if check:+ gate = bf16(K * bv)+ require(silu_ok(gate), "gate outside silu's exact range", gate)+ up = bf16(av / K)+ hw += float(c) * bf16(np.maximum(gate, 0.0) * up)+ if check:+ hw = bf16(hw)+ require(hw == ideal, "bf16 output differs from exact", np.stack([hw, ideal], 1)[:, 0])+ if node.into:+ ideal = m[:, node.out] + ideal+ if check:+ require(is_bf16(ideal), "residual add", ideal)+ m[:, node.out] = ideal+BYTE_LEVELS = np.arange(256.0)+# a decay counter's row (2^-n of a set bit after n halvings), as a snapshot taken at n > 0 holds it+DECAY_LEVELS = np.array([0.0] + [A_BIT * 2.0**-k for k in range(127)])+RMS_EPS = 1e-6 # Qwen3.5 rms_norm_eps (the gated output norm)+++def kernel_check(o_rows: list, q: np.ndarray, n_cols: int, ideal: np.ndarray) -> None:+ """A read as the HF torch Gated DeltaNet computes it, against the IR's exact value.++ q (all key dims) is L2-normalized in fp32 and scaled by 1/sqrt(128) (the recurrent kernel+ divides, the chunk kernel multiplies); o = sum of state rows times q in fp32, cast to bf16+ (the kernel's output dtype); the gated RMSNorm then normalizes o in fp32 and casts to bf16.+ The result must equal the IR's exact normalized row on every column. (One-hot queries give a+ scale of 2^-3.5, so only 0/power-of-2 data survives the bf16 cast; 2-hot queries give+ 2^-4.)+ """+ import torch++ qv = torch.tensor(q[q > 0], dtype=torch.float32)+ inv = torch.rsqrt((qv * qv).sum() + 1e-6)+ ql = qv * inv+ for name, qs in (("recurrent", ql / (HD ** 0.5)), ("chunk", ql * (HD ** -0.5))):+ o = torch.zeros(n_cols, dtype=torch.float32)+ for (row, qi) in o_rows:+ o = o + torch.tensor(row, dtype=torch.float32) * qs[qi]+ h = o.to(torch.bfloat16).float()+ var = h.pow(2).mean()+ normed = (h * torch.rsqrt(var + RMS_EPS)).to(torch.bfloat16).double().numpy()+ bad = normed != ideal+ if bad.any():+ col = int(np.argmax(bad))+ raise ExactnessError(f"{name} kernel read: column {col} is {normed[col]!r}, "+ f"exact {ideal[col]!r}")+++def eval_head(node: Head, m: np.ndarray, chip: np.ndarray, check: bool,+ carry: dict | None = None, ni: int = 0, kernel: bool = False) -> None:+ n = m.shape[0]+ q = (np.stack([conv(c, m, check, carry, (ni, "q", s)) for s, c in node.q.items()], 1)+ if node.q else np.zeros((n, 0)))+ k = (np.stack([conv(c, m, check, carry, (ni, "k", s)) for s, c in node.k.items()], 1)+ if node.k else np.zeros((n, 0)))+ qi, ki = list(node.q), list(node.k)+ if check:+ nz = (k > 0).sum(1)+ require(nz <= 1, "key must be one-hot", nz)+ require(~((nz > 0) & ~chip), "key on a text token", nz)+ for t in np.nonzero((q > 0).sum(1) > 1)[0]:+ vals = q[t][q[t] > 0]+ require(vals == vals[0], "query must be multi-hot", vals)+ has_key = k.any(1) if k.shape[1] else np.zeros(n, dtype=bool)+ key_row = [ki[int(np.argmax(k[t]))] if has_key[t] else None for t in range(n)]+ has_q = q.any(1) if q.shape[1] else np.zeros(n, dtype=bool)+ for x_, vh in enumerate(node.vheads()):+ v = np.zeros((n, node.n_cols))+ for col, c in vh.v.items():+ v[:, col] = conv(c, m, check, carry, (ni, "v", x_, col))+ a = ev(vh.a, m)+ z = {col: ev(e, m) for col, e in vh.z.items()}+ beta = np.ones(n) if vh.beta is None else ev(vh.beta, m)+ writes = has_key & (beta > 0)+ if check:+ require(is_bf16(a) & np.isin(a, list(DECAY)), "decay selector a", a)+ # beta = sigmoid(256 g - 128): exactly 1 at g = 1 and exactly 0 for integers g <= 0+ require((beta == 1.0) | ((beta <= 0) & (beta == np.round(beta))), "write gate beta",+ beta)+ for col, zz in z.items():+ require(is_bf16(zz) & silu_ok(zz), f"gate z[{col}]", zz)+ written = v[writes]+ if len(written):+ data = np.delete(written, REF_COLS, 1)+ require(np.isin(data, [0.0, A_BIT]), "state row data column", data)+ ref = written[:, list(REF_COLS)]+ require((ref == 0) | (ref == A_REF), "state row ballast column", ref)+ if carry is not None and (ni, "S", x_) in carry:+ state = carry[ni, "S", x_]+ else:+ state = {r: np.asarray(row, dtype=np.float64).copy()+ for r, row in ((node.init or [{}] * (x_ + 1))[x_] if node.init+ and x_ < len(node.init) else {}).items()}+ if check and state:+ rows = np.stack(list(state.values()))+ data = np.delete(rows, REF_COLS, 1)+ ok = np.isin(data, BYTE_LEVELS if node.levels == "byte" else+ DECAY_LEVELS if node.levels == "decay" else [0.0, A_BIT])+ require(ok, f"initial state data ({node.levels})", data)+ ref = rows[:, list(REF_COLS)]+ require((ref == 0) | (ref == A_REF), "initial state ballast column", ref)+ out = np.zeros((n, node.n_cols))+ decay = np.vectorize(DECAY.get)(a)+ # only tokens that decay, write or read change anything+ events = np.nonzero((decay != 1.0) | writes | has_q)[0]+ for t in events:+ if decay[t] == 0.0:+ state.clear()+ elif decay[t] == 0.5:+ state = {s: row * 0.5 for s, row in state.items()}+ if writes[t]:+ state[key_row[t]] = v[t].copy()+ if has_q[t]:+ o = sum((state[s] for s, w in zip(qi, q[t]) if w > 0 and s in state),+ np.zeros(node.n_cols))+ ref = (o[list(REF_COLS)] ** 2).sum()+ if ref == 0:+ if check:+ require(o == 0, "read without ballast", o)+ continue+ if check:+ data = (np.delete(o, REF_COLS) ** 2).sum()+ require(np.array([data <= ref * 2.0**-26]), "data not absorbed by ballast",+ np.array([data / ref]))+ out[t] = o / np.sqrt(ref / node.n_cols)+ if kernel and check:+ rows = [(state[s], j) for j, (s, w) in enumerate(+ (s, w) for s, w in zip(qi, q[t]) if w > 0) if s in state]+ kernel_check(rows, q[t], node.n_cols, out[t])+ if carry is not None:+ carry[ni, "S", x_] = state+ for col, d, w in vh.out:+ zz = z.get(col, np.zeros(n))+ val = out[:, col] * np.maximum(zz, 0.0)+ if check:+ small = np.abs(out[:, col]) < node.tiny+ require(is_bf16(out[:, col]) | small, f"normed column {col}", out[:, col])+ require(is_bf16(val * w) | small, f"head output column {col}", val * w)+ if vh.into:+ m[:, d] += val * w+ if check:+ require(is_bf16(m[:, d]), "head output sum", m[:, d])+ else:+ m[:, d] = val * wdiff --git a/experimental/riscv/chipformer/q35/lib/__init__.py b/experimental/riscv/chipformer/q35/lib/__init__.pynew file mode 100644index 0000000..96c2340--- /dev/null+++ b/experimental/riscv/chipformer/q35/lib/__init__.py@@ -0,0 +1 @@+"""The library the merged model's computer runs: an assembler and BCD arithmetic routines."""diff --git a/experimental/riscv/chipformer/q35/lib/asm.py b/experimental/riscv/chipformer/q35/lib/asm.pynew file mode 100644index 0000000..602d3cb--- /dev/null+++ b/experimental/riscv/chipformer/q35/lib/asm.py@@ -0,0 +1,151 @@+"""A small two-pass CHIP-8 assembler (Cowgod mnemonics, plus PRINT for the bridge's 00FD).++ label: define a label (its address)+ db 1, 2, 0x03 raw bytes+ org 0x600 set the address (gaps are zero-filled)+ ; comment++Instructions: CLS RET PRINT JP CALL SE SNE LD ADD OR AND XOR SUB SHR SUBN SHL RND DRW SKP SKNP.+Operands: V0..VF, I, [I], DT, ST, K, F, B, numbers (decimal / 0x / 0b) and labels.+"""++import re++START = 0x200+++def _num(tok: str, labels: dict[str, int]) -> int:+ tok = tok.strip()+ if tok in labels:+ return labels[tok]+ return int(tok, 0)+++def _reg(tok: str) -> int | None:+ m = re.fullmatch(r"[vV]([0-9a-fA-F])", tok.strip())+ return int(m[1], 16) if m else None+++def assemble(src: str, start: int = START) -> tuple[bytes, dict[str, int]]:+ """Returns (bytes from ``start``, labels)."""+ lines = []+ for raw in src.splitlines():+ line = raw.split(";")[0].strip()+ while line:+ m = re.match(r"([A-Za-z_][\w.]*):\s*(.*)", line)+ if not m:+ break+ lines.append(("label", m[1]))+ line = m[2].strip()+ if line:+ op, _, rest = line.partition(" ")+ args = [a.strip() for a in rest.split(",")] if rest.strip() else []+ lines.append((op.upper(), args))+ labels: dict[str, int] = {}+ for pass_ in range(2):+ pc = start+ out = bytearray()+ for op, args in lines:+ if op == "label":+ labels[args] = pc+ continue+ if op == "ORG":+ target = _num(args[0], labels)+ assert target >= pc, f"org {target:#x} goes backwards from {pc:#x}"+ out += bytes(target - pc)+ pc = target+ continue+ if op == "DB":+ data = [_num(a, labels) & 0xFF for a in args] if pass_ else [0] * len(args)+ out += bytes(data)+ pc += len(data)+ continue+ word = _encode(op, args, labels) if pass_ else 0+ out += bytes([word >> 8, word & 0xFF])+ pc += 2+ return bytes(out), labels+++def _encode(op: str, args: list[str], labels: dict[str, int]) -> int:+ r = [_reg(a) for a in args]+ a = [x.upper() for x in args]++ def n(i, bits=12):+ v = _num(args[i], labels)+ assert 0 <= v < 1 << bits, (op, args, v)+ return v++ if op == "CLS":+ return 0x00E0+ if op == "RET":+ return 0x00EE+ if op == "PRINT":+ return 0x00FD+ if op in ("BADD", "BSUB", "BRSUB"): # bignum macros (macros.py), with a jump target+ t = n(0)+ assert 0x200 <= t < 0x400 and t % 2 == 0, (op, args)+ return 0x5000 | (t - 0x200) // 2 << 4 | {"BADD": 1, "BSUB": 4, "BRSUB": 7}[op]+ if op == "BZERO":+ return 0x5002+ if op == "BMUL":+ return 0x5003 | r[0] << 8+ if op == "BDIV":+ return 0x5005 | r[0] << 8+ if op == "BADB":+ return 0x5006 | r[0] << 8+ if op == "BMUL0":+ return 0x5013 | r[0] << 8+ if op == "BTRIM":+ return 0x5008+ if op == "JP":+ if len(args) == 2:+ return 0xB000 | n(1)+ return 0x1000 | n(0)+ if op == "CALL":+ return 0x2000 | n(0)+ if op in ("SE", "SNE"):+ x = r[0]+ if r[1] is not None:+ return (0x5000 if op == "SE" else 0x9000) | x << 8 | r[1] << 4+ return (0x3000 if op == "SE" else 0x4000) | x << 8 | n(1, 8)+ if op == "LD":+ if a[0] == "I":+ return 0xA000 | n(1)+ if a[0] == "[I]":+ return 0xF055 | r[1] << 8+ if a[1] == "[I]":+ return 0xF065 | r[0] << 8+ if a[0] == "DT":+ return 0xF015 | r[1] << 8+ if a[0] == "ST":+ return 0xF018 | r[1] << 8+ if a[0] == "F":+ return 0xF029 | r[1] << 8+ if a[0] == "B":+ return 0xF033 | r[1] << 8+ if a[1] == "DT":+ return 0xF007 | r[0] << 8+ if a[1] == "K":+ return 0xF00A | r[0] << 8+ if r[1] is not None:+ return 0x8000 | r[0] << 8 | r[1] << 4+ return 0x6000 | r[0] << 8 | n(1, 8)+ if op == "ADD":+ if a[0] == "I":+ return 0xF01E | r[1] << 8+ if r[1] is not None:+ return 0x8004 | r[0] << 8 | r[1] << 4+ return 0x7000 | r[0] << 8 | n(1, 8)+ alu = {"OR": 1, "AND": 2, "XOR": 3, "SUB": 5, "SHR": 6, "SUBN": 7, "SHL": 0xE}+ if op in alu:+ y = r[1] if len(r) > 1 and r[1] is not None else 0+ return 0x8000 | r[0] << 8 | y << 4 | alu[op]+ if op == "RND":+ return 0xC000 | r[0] << 8 | n(1, 8)+ if op == "DRW":+ return 0xD000 | r[0] << 8 | r[1] << 4 | n(2, 4)+ if op == "SKP":+ return 0xE09E | r[0] << 8+ if op == "SKNP":+ return 0xE0A1 | r[0] << 8+ raise ValueError(f"unknown instruction {op} {args}")diff --git a/experimental/riscv/chipformer/q35/lib/bcd.py b/experimental/riscv/chipformer/q35/lib/bcd.pynew file mode 100644index 0000000..6657b47--- /dev/null+++ b/experimental/riscv/chipformer/q35/lib/bcd.py@@ -0,0 +1,144 @@+"""Exact decimal arithmetic for the computer inside the model ("the BIOS").++Calling convention (what the circuit's CALL record writes, see bridge.py):+ A = digits of the first argument, most significant first, at A_ADDR (LA of them at LEN)+ B = digits of the second argument at B_ADDR (LB at LEN + 1)+ PC = the routine's entry+Each routine leaves the result's digits at R_ADDR and ends with PRINT (00FD), which prints V0+digits from I and halts. Leading zeros are stripped. Nothing ticks during a call, so the routines+never use the timers or wait for keys.+"""++from chipformer import spec+from chipformer.q35.lib.asm import assemble++A_ADDR, B_ADDR, LEN, R_ADDR = 0x600, 0x620, 0x640, 0x660+# 0: a tool call (<tool_response>); 1: an inline call (" digits"), | 2 inside "$", | 4 inside+# "$$", | 8 the result is negative (set by sub's BRSUB)+MODE = LEN + 2+# div: W = 0, A's digits (the CALL record writes them at W_ADDR + 1), DEC zeros; the quotient goes+# to R_ADDR as P[t] (t <= LA), "." (10) at P[LA + 1], P[t + 1] (t > LA), "..." (11) if inexact+W_ADDR = 0x6BF+DEC = 10 # decimals of an inexact quotient+T10, MUL, DIV, MOD = 0x700, 0x710, 0x780, 0x800+MAX_DIGITS = 24+QUIRKS = ("loadstore",) # FX55/FX65 leave I unchanged+++SRC = f"""+idle: jp idle ; PC rests here between calls (not a PRINT)++; ---------------------------------------------------------------- add: R = A + B+add: badd trim ; macro: R = A + B, v0 = length, i = R, then trim+ db 0, 0 ; (entries are compiled into the weights)++; ---------------------------------------------------------------- sub: R = A - B+sub: bsub trim ; macro: R = A - B, v0 = length, i = R; trim unless A < B+ jp sub_neg++; ---------------------------------------------------------------- mul: R = A * B+mul: ld v1, 0 ; rows done+ bmul0 v1 ; macro: the first schoolbook row (no BZERO)+ bmul v1 ; macro: the next row; repeats until the last row,+ ; which sets v0 = LA + LB, i = R++; ---------------------------------------------------------------- print mem[i..i+v0)+trim: btrim ; macro: drop a leading 0 (keeping one digit); repeats++; ---------------------------------------------------------------- A - B < 0: B - A+; inline: MODE |= 8 (the print starts with " -"); a tool call prints nothing (v0 = 0)+sub_neg: brsub trim++; ---------------------------------------------------------------- div: A / B+; digits of an exact quotient, else DEC decimals and "..."; nothing if B starts with 0+div: bzero ; P[0 .. LA+LB) = 0+ ld i, {LEN}+ ld v1, [i] ; v0 = LA, v1 = LB+ ld va, v0+ ld vb, v1+ ld i, {B_ADDR}+ ld v0, [i]+ sne v0, 0+ jp div_z+ ld i, {R_ADDR + 1}+ add i, va+ ld v0, 10+ ld [i], v0 ; P[LA + 1] = "."+ ld i, zeros+ ld v9, [i] ; v0..v9 = 0+ ld i, {W_ADDR}+ ld [i], v0 ; W[0] = 0+ ld i, {W_ADDR + 1}+ add i, va+ ld [i], v9 ; W[LA+1 .. LA+DEC] = 0+ ld vc, vb ; t = LB+ ld v0, va+ add v0, {DEC}+ sub v0, vc ; VF = [LA + DEC >= LB]+ se vf, 0+ jp div_lp+ ld vc, va+ add vc, {DEC} ; B is too long: the quotient is 0.000..., t = LA + DEC+ jp div_rem+div_lp: bdiv vc ; repeats itself, one quotient digit per record+ se vf, 1+ jp div_end ; VF = 2 (exact) or 3 (DEC decimals)+ badb vc ; repeats itself while the window is negative+ se vf, 0+ jp div_end+ jp div_lp+div_end: se vf, 3+ jp div_ex+div_rem: ld i, {R_ADDR + 2}+ add i, vc+ ld v0, 11+ ld [i], v0 ; "..." after the last decimal+ ld ve, vc+ add ve, 2 ; the last index printed+ jp div_pr+div_ex: ld ve, vc+ se vc, va+ add ve, 1 ; t > LA: P[t + 1]; t == LA: no "."+div_pr: ld v6, vb ; first digit: P[LB] (P[LA] if LB > LA), then skip zeros+ ld v0, va+ sub v0, v6+ sne vf, 0+ ld v6, va+dp_lp: se v6, va+ jp dp_chk+ jp dp_go+dp_chk: ld i, {R_ADDR}+ add i, v6+ ld v0, [i]+ se v0, 0+ jp dp_go+ add v6, 1+ jp dp_lp+dp_go: ld i, {R_ADDR}+ add i, v6+ ld v0, ve+ sub v0, v6+ add v0, 1+div_z: ld v0, 0+zeros: db 0, 0, 0, 0, 0, 0, 0, 0, 0, 0++"""++ROM, LABELS = assemble(SRC)+OPS = {"add": LABELS["add"], "mul": LABELS["mul"], "sub": LABELS["sub"], "div": LABELS["div"]}+# The entries are compiled into the circuit's CALL record (chip8_q35.py), so they are part of the+# weights: moving a routine means rebuilding every checkpoint. Add new code after the entries.+assert {k: v for k, v in OPS.items() if k != "div"} == {"add": 0x202, "sub": 0x206,+ "mul": 0x20A}, OPS+++def call_writes(op: str, a: str, b: str, mode: int = 0) -> list[tuple[int, int]]:+ """The writes of a CALL record: the arguments' digits, their lengths, the mode, then PC."""+ da, db = [int(ch) for ch in a], [int(ch) for ch in b]+ assert 1 <= len(da) <= MAX_DIGITS and 1 <= len(db) <= MAX_DIGITS, (a, b)+ a_at = W_ADDR + 1 if op == "div" else A_ADDR # div works in place on W+ w = [(a_at + i, d) for i, d in enumerate(da)] + [(B_ADDR + i, d) for i, d in enumerate(db)]+ return w + [(LEN, len(da)), (LEN + 1, len(db)), (MODE, mode), (spec.PC, OPS[op])]diff --git a/experimental/riscv/chipformer/q35/lib/macros.py b/experimental/riscv/chipformer/q35/lib/macros.pynew file mode 100644index 0000000..a66118e--- /dev/null+++ b/experimental/riscv/chipformer/q35/lib/macros.py@@ -0,0 +1,165 @@+"""Bignum macro instructions: the circuit computes a whole row of digits in one record.++They live in CHIP-8's unused 5XYN space (5XY0 is the usual skip) and work on the BIOS layout+(bcd.py): digits most significant first at A_ADDR / B_ADDR, lengths at LEN / LEN+1, result at+R_ADDR. Writes go least significant first so a carry flows from one write to the next. T is a+jump target in the instruction itself, T = 0x200 + 2 * XY.++ 5XY1 BADD R[0..n) = A + B, n = max(LA, LB) + 1, then V0 = n - z, I = R_ADDR + z+ (z = [R[0] == 0]: the print skips a leading zero); PC = T+ 5XY2 BZERO R[0..n) = 0, n = LA + LB, then V7 = n+ 5XY4 BSUB R[0..n) = A - B mod 10^n, n = max(LA, LB) + 1, then V0 and I as BADD;+ PC = T if A >= B, else the next instruction+ 5XY7 BRSUB R[0..n) = B - A (for B >= A), then V0 and I as BADD (V0 = 0 for a tool call:+ MODE bit 1 clear), MODE |= 8 (the print starts with " -"); PC = T+ 5X03 BMUL one schoolbook row of A * B; VX = rows done, row r = LB - 1 - VX:+ for i = LA-1 .. 0: R[i+r+1] = (R[i+r+1] + A[i] * B[r] + carry) mod 10, carry = ..div 10+ then R[r] = carry, VX += 1, and after the last row V0 = LA + LB - z,+ I = R_ADDR + z (z = [R[0] == 0]); PC stays (the next row) until the last row+ 5X13 BMUL0 the first row (VX = 0): the same with R read as 0 (no BZERO needed); PC = the+ next instruction (a BMUL), or the one after it if LB = 1+ 5008 BTRIM if mem[I] == 0, V0 >= 2 and I is in the result: V0 -= 1, I += 1, PC stays;+ else PC = the next instruction (strips leading zeros before a PRINT)++A record whose PC stays has no PC write (the circuit fetches from the unchanged PC).++Each record's values come from the state before the record plus the carry chain inside it, so a+multiplication costs LB records of LA + 4 writes.++Long division works in place on W (bcd.W_ADDR: a 0, the digits of A, then DEC zeros), with the+divisor B and the quotient's digits in P (bcd.R_ADDR; P[t] for t <= LA, P[t + 1] after the "."):++ 5XY5 BDIV one quotient digit, t = VX: the remainder window W[t-LB .. t] -= q * B, P[.] = q,+ with q estimated from the window's top 3 digits and B's top 2 (q <= true + 2)+ 5XY6 BADB the add-back after an estimate that was too big: W[t-LB .. t] += B, P[.] -= 1++Both end with VF = 1 (negative: add back), 2 (done, exact), 3 (done, DEC decimals and a+remainder) or 0 (next digit, VX = t + 1). BDIV repeats itself (PC unchanged) on 0 and BADB on 1,+so a division needs loop instructions only around its add-backs.+"""++from chipformer import spec+from chipformer.q35.lib import bcd++BADD, BZERO, BMUL, BSUB, BDIV, BADB, BRSUB, BTRIM = 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8+KINDS = (BADD, BZERO, BMUL, BSUB, BDIV, BADB, BRSUB, BTRIM)+R_CELLS = 2 * bcd.MAX_DIGITS # the result region BTRIM can read+++def target(op: int) -> int:+ return 0x200 + 2 * (op >> 4 & 0xFF)+++def div_estimate(W: list[int], B: list[int], t: int) -> int:+ """BDIV's quotient digit: never below the true one, at most 2 above it (for B[0] != 0)."""+ lb = len(B)+ if lb == 1:+ r3, b2, delta = 10 * W[t - 1] + W[t], B[0], 0+ else:+ r3 = 100 * W[t - lb] + 10 * W[t - lb + 1] + W[t - lb + 2]+ b2, delta = 10 * B[0] + B[1], int(lb >= 3)+ return min(9, (r3 + delta) // b2) if b2 else 9+++def _div_record(cells: list[int], op: int, kind: int) -> list[tuple[int, int]]:+ la, lb = cells[bcd.LEN], cells[bcd.LEN + 1]+ B = [cells[bcd.B_ADDR + i] for i in range(lb)]+ x = op >> 8 & 0xF+ t = cells[spec.V + x]+ W = [cells[(bcd.W_ADDR + i) & 0xFFF] for i in range(64)]+ p_addr = bcd.R_ADDR + t + int(t > la)+ q = div_estimate(W, B, t) if kind == BDIV else 0+ # the quotient digit goes first: later tokens of the record read it from the token stream+ w = [(p_addr, q if kind == BDIV else (cells[p_addr] - 1) & 0xFF)]+ carry, digits = 0, []+ for k in range(lb): # least significant first+ i = lb - 1 - k+ if kind == BDIV:+ v = W[t - k] - q * B[i] - carry+ d = v % 10+ carry = (d - v) // 10 # the borrow+ else:+ v = W[t - k] + B[i] + carry+ d, carry = v % 10, v // 10+ w.append((bcd.W_ADDR + t - k, d))+ digits.append(d)+ v = W[t - lb] + (carry if kind == BADB else -carry)+ d = v % 10+ w.append((bcd.W_ADDR + t - lb, d))+ digits.append(d)+ # BDIV: negative if the top digit borrowed; BADB: still negative unless it carried out+ neg = v < 0 if kind == BDIV else v < 10+ zero = not neg and not any(digits)+ flag = 1 if neg else 2 if zero and t >= la else 3 if t == la + bcd.DEC else 0+ w.append((spec.V + x, t + (flag == 0)))+ w.append((spec.V + 15, flag))+ pc = cells[spec.PC]+ again = flag == (0 if kind == BDIV else 1)+ return w + [(spec.PC, pc if again else (pc + 2) & 0xFFF)]+++def macro_writes(cells: list[int], op: int):+ """The record for opcode ``op`` if it is a macro, else None."""+ if op >> 12 != 5 or op & 0xF not in KINDS:+ return None+ if op & 0xF in (BDIV, BADB):+ return _div_record(cells, op, op & 0xF)+ la, lb = cells[bcd.LEN], cells[bcd.LEN + 1]+ A = [cells[bcd.A_ADDR + i] for i in range(la)]+ B = [cells[bcd.B_ADDR + i] for i in range(lb)]+ pc = cells[spec.PC]+ nxt = (pc + 2) & 0xFFF+ kind = op & 0xF+ w = []+ if kind in (BADD, BSUB, BRSUB):+ n = max(la, lb) + 1+ carry = 0+ for k in range(n): # w-th write is digit k from the right+ a = A[la - 1 - k] if k < la else 0+ b = B[lb - 1 - k] if k < lb else 0+ if kind == BRSUB:+ a, b = b, a+ if kind == BADD:+ s = a + b + carry+ w.append((bcd.R_ADDR + n - 1 - k, s % 10))+ carry = s // 10+ else: # carry is the borrow+ s = a - b - carry+ w.append((bcd.R_ADDR + n - 1 - k, s % 10))+ carry = int(s < 0)+ mode = cells[bcd.MODE]+ z = int(w[-1][1] == 0) # the top digit, R[0]+ w.append((spec.V, n - z if kind != BRSUB or mode & 1 else 0))+ w.append((spec.I, bcd.R_ADDR + z))+ if kind == BRSUB:+ w.append((bcd.MODE, mode | 8))+ new_pc = nxt if kind == BSUB and carry else target(op)+ elif kind == BZERO:+ n = la + lb+ w = [(bcd.R_ADDR + k, 0) for k in range(n)] + [(spec.V + 7, n)]+ new_pc = nxt+ elif kind == BMUL:+ x, first = op >> 8 & 0xF, op >> 4 & 0xF == 1+ d = cells[spec.V + x]+ row = lb - 1 - d+ b = B[row] if 0 <= row < lb else 0+ carry = 0+ for i in range(la - 1, -1, -1):+ k = i + row + 1+ t = (0 if first else cells[bcd.R_ADDR + k]) + A[i] * b + carry+ w.append((bcd.R_ADDR + k, t % 10))+ carry = t // 10+ w.append((bcd.R_ADDR + row, carry))+ w.append((spec.V + x, (d + 1) & 0xFF))+ last = d + 1 == lb+ if last:+ z = int(carry == 0) # R[0]+ w += [(spec.V, la + lb - z), (spec.I, bcd.R_ADDR + z)]+ new_pc = ((nxt + 2) & 0xFFF if last else nxt) if first else nxt if last else pc+ else: # BTRIM+ i, v0 = cells[spec.I], cells[spec.V]+ z = bcd.R_ADDR <= i < bcd.R_ADDR + R_CELLS and cells[i] & 15 == 0 and v0 >= 2+ if z:+ w = [(spec.V, v0 - 1), (spec.I, i + 1)]+ new_pc = pc if z else nxt+ return w if new_pc == pc else w + [(spec.PC, new_pc)]diff --git a/experimental/riscv/chipformer/q35/merge.py b/experimental/riscv/chipformer/q35/merge.pynew file mode 100644index 0000000..b2f0e17--- /dev/null+++ b/experimental/riscv/chipformer/q35/merge.py@@ -0,0 +1,316 @@+"""Widen a Qwen3.5 checkpoint and append chip layers, keeping Qwen's own computation intact.++Qwen's matrices are zero-padded to the new hidden size, so its dims never read or write chip+dims. The RMSNorms that see the full residual (input, post-attention, final) average over more+dims after widening; their gains are scaled by sqrt(D / D') so Qwen's normalized inputs stay the+same up to one bf16 rounding. The MTP head is dropped: HF ignores it, and its shapes would change.+"""++import copy+import json+import re+from pathlib import Path++import torch+from safetensors import safe_open++TEXT = "model.language_model."+# which axis of each weight is the hidden dim, by name+IN_COLS = re.compile(+ r"(in_proj_(qkv|z|a|b)|q_proj|k_proj|v_proj|gate_proj|up_proj)\.weight$|embed_tokens\.weight$"+)+OUT_ROWS = re.compile(r"(out_proj|o_proj|down_proj)\.weight$|visual\.merger\.linear_fc2\.(weight|bias)$")+WIDE_NORMS = re.compile(r"(input_layernorm|post_attention_layernorm)\.weight$|language_model\.norm\.weight$")++IM_START = 248045 # <|im_start|> in Qwen3.5's tokenizer+++def load(path):+ path = Path(path)+ cfg = json.loads((path / "config.json").read_text())+ sd = {}+ for f in sorted(path.glob("*.safetensors")):+ with safe_open(f, "pt") as fh:+ for k in fh.keys(): # noqa: SIM118 - safe_open is not iterable+ if not k.startswith("mtp."):+ sd[k] = fh.get_tensor(k)+ return cfg, sd+++def widen(cfg, sd, hidden, n_chip_layers=0):+ """Return (config, state_dict) with hidden size `hidden` and n zero chip layers appended."""+ tc = cfg["text_config"]+ d = tc["hidden_size"]+ assert hidden >= d+ scale = (d / hidden) ** 0.5+ out = {}+ for k, w in sd.items():+ if IN_COLS.search(k):+ w = torch.cat([w, w.new_zeros(*w.shape[:-1], hidden - d)], -1)+ elif OUT_ROWS.search(k):+ w = torch.cat([w, w.new_zeros(hidden - d, *w.shape[1:])], 0)+ elif WIDE_NORMS.search(k):+ g = (1.0 + w.float()) * scale - 1.0 # zero-centred gain: (1 + w)+ w = torch.cat([g.to(w.dtype), w.new_zeros(hidden - d)])+ out[k] = w+ cfg = copy.deepcopy(cfg)+ tc = cfg["text_config"]+ tc["hidden_size"] = hidden+ cfg["vision_config"]["out_hidden_size"] = hidden+ tc["mtp_num_hidden_layers"] = 0+ tc.pop("full_attention_interval", None)+ n0 = tc["num_hidden_layers"]+ tc["layer_types"] = tc["layer_types"] + ["linear_attention"] * n_chip_layers+ tc["num_hidden_layers"] = n0 + n_chip_layers+ for i in range(n0, n0 + n_chip_layers):+ out.update(empty_chip_layer(tc, i, out[f"{TEXT}layers.0.input_layernorm.weight"].dtype))+ return cfg, out+++def empty_chip_layer(tc, i, dtype):+ """Zero weights for a linear-attention layer: on text tokens it adds exactly 0."""+ h, nk, nv = tc["hidden_size"], tc["linear_num_key_heads"], tc["linear_num_value_heads"]+ dk, dv, inter = tc["linear_key_head_dim"], tc["linear_value_head_dim"], tc["intermediate_size"]+ p = f"{TEXT}layers.{i}."+ z = lambda *s: torch.zeros(*s, dtype=dtype)+ return {+ p + "input_layernorm.weight": z(h),+ p + "post_attention_layernorm.weight": z(h),+ p + "linear_attn.in_proj_qkv.weight": z(2 * nk * dk + nv * dv, h),+ p + "linear_attn.in_proj_z.weight": z(nv * dv, h),+ p + "linear_attn.in_proj_a.weight": z(nv, h),+ p + "linear_attn.in_proj_b.weight": z(nv, h),+ p + "linear_attn.conv1d.weight": z(2 * nk * dk + nv * dv, 1, tc["linear_conv_kernel_dim"]),+ p + "linear_attn.dt_bias": z(nv),+ p + "linear_attn.A_log": z(nv),+ p + "linear_attn.norm.weight": torch.ones(dv, dtype=dtype),+ p + "linear_attn.out_proj.weight": z(h, nv * dv),+ p + "mlp.gate_proj.weight": z(inter, h),+ p + "mlp.up_proj.weight": z(inter, h),+ p + "mlp.down_proj.weight": z(h, inter),+ }+++def instantiate(cfg, sd, dtype=torch.bfloat16):+ from transformers import Qwen3_5Config, Qwen3_5ForConditionalGeneration++ config = Qwen3_5Config(**{k: v for k, v in cfg.items() if k not in ("architectures", "model_type")})+ with torch.device("meta"):+ model = Qwen3_5ForConditionalGeneration(config)+ missing, unexpected = model.load_state_dict(sd, strict=False, assign=True)+ missing = [k for k in missing if k != "lm_head.weight"] # tied to embed_tokens+ assert not missing and not unexpected, (missing[:5], unexpected[:5])+ model.tie_weights()+ # rotary tables are non-persistent buffers, still on the meta device: rebuild them+ from transformers.models.qwen3_5 import modeling_qwen3_5 as q++ inner = model.model+ inner.language_model.rotary_emb = q.Qwen3_5TextRotaryEmbedding(config=config.text_config)+ inner.visual.rotary_pos_emb = q.Qwen3_5VisionRotaryEmbedding(config.vision_config)+ return model.to(dtype).eval()+++# --- the merged checkpoint -----------------------------------------------------------------++TAU_TEXT = 2.0**-40 # tau in every text token's embedding row+TAU_GAIN = 2.0**60 # final-norm gain on tau: chip logits at text positions ~ -2^20 / rms+++def chip_token_names() -> list[str]:+ from chipformer import spec++ names = []+ for t in range(spec.VOCAB_SIZE):+ if spec.is_val(t):+ names.append(f"<|chip={t - spec.VAL_BASE:#05x}|>")+ elif spec.is_addr(t):+ names.append(f"<|chip@{t - spec.ADDR_BASE:#06x}|>")+ else:+ names.append({spec.STEP: "<|chip_step|>", spec.TICK: "<|chip_tick|>",+ spec.BOOT: "<|chip_boot|>"}[t])+ return names+++def extend_tokenizer(src: Path, dst: Path, first_id: int) -> None:+ """Copy the tokenizer, padding with reserved tokens up to first_id, then the chip tokens."""+ import shutil++ from tokenizers import AddedToken, Tokenizer++ tok = Tokenizer.from_file(str(src / "tokenizer.json"))+ n = tok.get_vocab_size(with_added_tokens=True)+ pad = [AddedToken(f"<|reserved_{i}|>", special=True) for i in range(first_id - n)]+ chip = [AddedToken(s, special=True) for s in chip_token_names()]+ tok.add_special_tokens(pad + chip)+ assert tok.token_to_id(chip_token_names()[0]) == first_id+ dst.mkdir(parents=True, exist_ok=True)+ tok.save(str(dst / "tokenizer.json"))+ for f in ("chat_template.jinja", "preprocessor_config.json",+ "video_preprocessor_config.json", "generation_config.json", "LICENSE"):+ if (src / f).exists():+ shutil.copy(src / f, dst / f)+ tc = json.loads((src / "tokenizer_config.json").read_text(encoding="utf-8"))+ dec = tc.setdefault("added_tokens_decoder", {})+ for i, t in enumerate(pad + chip):+ dec[str(n + i)] = {"content": t.content, "lstrip": False, "normalized": False,+ "rstrip": False, "single_word": False, "special": True}+ (dst / "tokenizer_config.json").write_text(json.dumps(tc, indent=2, ensure_ascii=False),+ encoding="utf-8")+++def pick_hidden(base: int, n_chip: int) -> tuple[int, int]:+ """Smallest (hidden, m) with hidden = m * 4^k >= base + m + n_chip, a multiple of 128, and+ 3 <= m <= 64 ballast dims (m * B^2 / hidden must be a power of 4)."""+ best = None+ for m in range(3, 65):+ k = 0+ while m * 4**k < base + m + n_chip or (m * 4**k) % 128:+ k += 1+ h = m * 4**k+ if best is None or h < best[0] or (h == best[0] and m > best[1]):+ best = (h, m)+ return best+++def build_merged(src, dst, hidden: int | None = None, m: int | None = None,+ mute_qwen: bool = True):+ """Write Qwen3.5 + the CHIP-8 circuit as one checkpoint ("v2" layout); returns (circuit,+ layout).++ The chip layers come first. Every token carries the ballast and the features of its+ class (chip token, or one of bridge.py's text classes); the last chip sublayer clears all+ of it, leaving each token's Qwen part, mu (1 on chip tokens), the output code and tau, so+ Qwen's layers see their usual input on text and a muted, empty token at chip positions.+ """+ from safetensors.torch import save_file+ from transformers import AutoTokenizer++ from chipformer import spec+ from chipformer.q35 import bridge, chip8_q35, tiny+ from chipformer.q35.emit import Layout, allocate, check_bf16, embedding_rows, emit_layers+ from chipformer.q35.ir import Target++ src, dst = Path(src), Path(dst)+ cfg, sd = load(src)+ tc = cfg["text_config"]+ d0, v0, inter = tc["hidden_size"], tc["vocab_size"], tc["intermediate_size"]+ c = chip8_q35.build(Target(tc["linear_num_key_heads"], tc["linear_num_value_heads"], inter))+ keep = set(c.keep)+ code = [next(iter(e.terms)) for e in c.code]+ tau, mu = next(iter(c.tau.terms)), next(iter(c.mu.terms))+ alloc = allocate(c, None, keep, inter)+ if hidden is None:+ hidden, m = pick_hidden(d0, 1 + alloc.n_slots)+ lay = Layout(hidden=hidden, off=d0, m=m, slots=alloc.slots)+ assert d0 + m + 1 + alloc.n_slots <= hidden, (alloc.n_slots, hidden)+ lay.scale # noqa: B018 - asserts m * B^2 / hidden is a power of 4++ n_chip = c.n_layers()+ cfg, sd = widen(cfg, sd, hidden)+ tc = cfg["text_config"]+ n0 = tc["num_hidden_layers"]+ # Qwen's layers move behind the chip layers+ shift = re.compile(r"^(model\.language_model\.layers\.)(\d+)\.")+ sd = {shift.sub(lambda mt: f"{mt[1]}{int(mt[2]) + n_chip}.", k): v for k, v in sd.items()}+ tc["layer_types"] = ["linear_attention"] * n_chip + tc["layer_types"]+ tc["num_hidden_layers"] = n_chip + n0+ vocab = -(-(v0 + spec.VOCAB_SIZE) // 64) * 64+ tc["vocab_size"] = vocab++ if mute_qwen:+ from chipformer.q35.mute import fit_anti_sink, mute++ dev = "cuda" if torch.cuda.is_available() else "cpu"+ sinks = fit_anti_sink(src, dev)+ mute(sd, tc, n0, lay.dim(mu), hidden**0.5, sinks=sinks, first=n_chip)+ layers = emit_layers(c, lay, tc, 0, TEXT, alloc, keep)+ sd.update({k: v.to(torch.bfloat16) for k, v in layers.items()})+ del layers++ # embeddings: Qwen's rows keep their Qwen part and get their text class's chip features+ tok = AutoTokenizer.from_pretrained(src)+ text_ids = {}+ for cls, string in bridge.QWEN_TEXT.items():+ ids = tok.encode(string, add_special_tokens=False)+ assert len(ids) == 1, (string, ids)+ text_ids[cls] = ids[0]+ classes = list(range(bridge.TEXT_BASE, bridge.TEXT_BASE + bridge.N_TEXT))+ rows = dict(zip(classes, embedding_rows(c, lay, classes, hidden).to(torch.bfloat16)))+ full = torch.zeros(vocab, hidden, dtype=torch.bfloat16)+ full[:v0, :d0] = sd[TEXT + "embed_tokens.weight"][:, :d0]+ full[:v0, d0:] = rows[bridge.OTHER][d0:]+ for cls, i in text_ids.items():+ full[i, d0:] = rows[cls][d0:]+ for cls, strings in bridge.EXTRA_TEXT.items():+ for string in strings:+ (i,) = tok.encode(string, add_special_tokens=False)+ full[i, d0:] = rows[cls][d0:]+ chip = embedding_rows(c, lay, list(range(spec.VOCAB_SIZE)), hidden)+ full[v0 : v0 + spec.VOCAB_SIZE] = chip.to(torch.bfloat16)+ sd[TEXT + "embed_tokens.weight"] = full+ # image tokens come from the vision merger: give them the OTHER class too+ bias = "model.visual.merger.linear_fc2.bias"+ if bias in sd:+ sd[bias] = sd[bias].clone()+ sd[bias][d0:] = rows[bridge.OTHER][d0:].to(sd[bias].dtype)++ norm = sd[TEXT + "norm.weight"].float()+ norm[d0:] = -1.0 # chip dims are invisible to the lm_head ...+ for d in code: # ... except the output code and tau+ norm[lay.dim(d)] = tiny.code_gain(lay)+ norm[lay.dim(tau)] = TAU_GAIN+ check_bf16("final norm", norm[d0:])+ sd[TEXT + "norm.weight"] = norm.to(torch.bfloat16)++ dst.mkdir(parents=True, exist_ok=True)+ shard, size, shards = {}, 0, []+ for k in sorted(sd):+ shard[k] = sd[k].contiguous()+ size += sd[k].numel() * 2+ if size > 2**31:+ shards.append(shard)+ shard, size = {}, 0+ shards.append(shard)+ index = {"metadata": {"total_size": sum(t.numel() * 2 for t in sd.values())}, "weight_map": {}}+ for i, sh in enumerate(shards):+ name = f"model-{i + 1:05d}-of-{len(shards):05d}.safetensors"+ save_file(sh, str(dst / name), metadata={"format": "pt"})+ index["weight_map"].update({k: name for k in sh})+ (dst / "model.safetensors.index.json").write_text(json.dumps(index, indent=2))+ (dst / "config.json").write_text(json.dumps(cfg, indent=2))+ extend_tokenizer(src, dst, v0)+ gen_path = dst / "generation_config.json"+ gen = json.loads(gen_path.read_text()) if gen_path.exists() else {}+ if "eos_token_id" not in gen:+ eos = cfg["text_config"].get("eos_token_id", 248044)+ gen.update(eos_token_id=[248046, eos], pad_token_id=eos)+ # After the circuit prints a tool response inside the assistant turn, Qwen sometimes opens a+ # new turn (<|im_start|>user ...) instead of going on. Qwen never writes <|im_start|> in its+ # own output, so the merged model bans it; the circuit never predicts it either.+ gen["bad_words_ids"] = [[IM_START]]+ gen_path.write_text(json.dumps(gen, indent=2))+ meta = {"layout": "v2", "chip_token_offset": v0, "chip_vocab": spec.VOCAB_SIZE,+ "chip_layers": n_chip, "qwen_layers_from": n_chip, "hidden": hidden,+ "chip_dims_offset": d0, "ballast_dims": m, "circuit": c.stats(),+ "slots": alloc.n_slots, "muted": mute_qwen,+ "text_ids": {str(k): v for k, v in text_ids.items()}}+ (dst / "chipformer.json").write_text(json.dumps(meta, indent=2))+ return c, lay+++def main() -> None:+ import argparse++ ap = argparse.ArgumentParser(description="Merge the CHIP-8 circuit into a Qwen3.5 checkpoint")+ ap.add_argument("src", help="Qwen3.5 checkpoint directory")+ ap.add_argument("--out", default="dist/qwen3.5-0.8b-chip8")+ ap.add_argument("--hidden", type=int, default=None)+ ap.add_argument("--ballast", type=int, default=None)+ ap.add_argument("--no-mute", action="store_true", help="leave Qwen's layers active on chip tokens")+ args = ap.parse_args()+ c, lay = build_merged(args.src, args.out, args.hidden, args.ballast, not args.no_mute)+ print(f"{args.out}: hidden {lay.hidden}, {c.n_layers()} chip layers, {c.stats()}")+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/mute.py b/experimental/riscv/chipformer/q35/mute.pynew file mode 100644index 0000000..be78979--- /dev/null+++ b/experimental/riscv/chipformer/q35/mute.py@@ -0,0 +1,98 @@+"""Make Qwen's own layers ignore chip tokens.++On a chip token every RMSNorm output on the ballast dims is a known constant (the normed+ballast, ``Layout.scale * B``), and on a text token it is exactly 0. Weights on one ballast dim can+therefore act on chip tokens alone, leaving text untouched bit for bit:++ Gated DeltaNet a = -1024: softplus(a + dt_bias) = 0, so g = 0 and the state does not decay+ b = -1024: beta = 0, so nothing is written+ z = -1024: silu(z) = 0, so the output is exactly 0+ full attention output gate = -1024: sigmoid = 0, so the output is exactly 0++So Qwen's recurrent state passes through a run of chip tokens unchanged, and chip positions add+nothing to Qwen's residual. Chip positions are still keys in Qwen's full attention (with values+~0); an "anti-sink" key, pointing against the mean query of each KV head (fitted on text), keeps+them from soaking up attention. What remains is a fixed boundary effect of Qwen's 3-token conv+window, independent of the trace length (``report.py``'s post-trace section).+"""++import re++import torch++MUTE = -1024.0+++ROTARY = 64 # dims of each attention head that RoPE rotates; the anti-sink key avoids them+SINK_NORM = 16.0++[email protected]_grad()+def fit_anti_sink(base_dir, device: str = "cpu") -> dict[int, torch.Tensor]:+ """Per full-attention layer, (n_kv_heads, head_dim) keys opposite to the mean text query."""+ from transformers import AutoModelForImageTextToText, AutoTokenizer++ from chipformer.q35.report import PROMPTS, encode++ tok = AutoTokenizer.from_pretrained(base_dir)+ model = AutoModelForImageTextToText.from_pretrained(base_dir, dtype=torch.bfloat16).to(device)+ lm, tc = model.model.language_model, model.config.text_config+ layers = {i: l.self_attn for i, l in enumerate(lm.layers)+ if tc.layer_types[i] == "full_attention"}+ sums = {i: 0 for i in layers}+ count = 0+ hooks = [a.q_norm.register_forward_hook(+ lambda m, inp, out, i=i: sums.__setitem__(i, sums[i] + out.float().flatten(0, -3).sum(0)))+ for i, a in layers.items()]+ for kind, text in PROMPTS:+ ids = encode(tok, kind, text)+ model(input_ids=torch.tensor([ids], device=device))+ count += len(ids)+ for h in hooks:+ h.remove()+ hd, nkv = tc.head_dim, tc.num_key_value_heads+ grp = tc.num_attention_heads // nkv+ out = {}+ for i, a in layers.items():+ mean = sums[i] / count # (heads, head_dim)+ gain = 1.0 + a.k_norm.weight.float()+ keys = torch.zeros(nkv, hd)+ for g in range(nkv):+ u = -mean[g * grp:(g + 1) * grp].mean(0)[ROTARY:].cpu() / gain[ROTARY:].cpu()+ keys[g, ROTARY:] = u / u.norm() * SINK_NORM+ out[i] = keys+ return out+++def mute(sd: dict, tc: dict, n_qwen: int, ballast_dim: int, normed_ballast: float,+ prefix: str = "model.language_model.", sinks: dict | None = None,+ first: int = 0) -> None:+ """Patch Qwen's layers (indices first .. first+n_qwen-1) of ``sd`` in place.++ ``ballast_dim`` is any dim that is a known constant on chip tokens and exactly 0 on text+ (the ballast in the v1 layout, the mu marker after the chip layers' exit in v2);+ ``normed_ballast`` is its value after the RMSNorm. ``sinks`` is keyed by Qwen's own index.+ """+ w = MUTE / normed_ballast+ hd = tc["head_dim"]+ for i in range(n_qwen):+ p = f"{prefix}layers.{first + i}."+ if tc["layer_types"][first + i] == "linear_attention":+ for name in ("in_proj_a", "in_proj_b", "in_proj_z"):+ t = sd[f"{p}linear_attn.{name}.weight"]+ t[:, ballast_dim] = w+ else:+ q = sd[f"{p}self_attn.q_proj.weight"] # rows: per head [query (hd), gate (hd)]+ for h in range(q.shape[0] // (2 * hd)):+ q[h * 2 * hd + hd : (h + 1) * 2 * hd, ballast_dim] = w+ if sinks is not None:+ k = sd[f"{p}self_attn.k_proj.weight"]+ k[:, ballast_dim] = (sinks[i].flatten() / normed_ballast).to(k.dtype)+++def is_muted(sd: dict, ballast_dim: int) -> bool:+ keys = [k for k in sd if re.search(r"linear_attn\.in_proj_b\.weight$", k)]+ return bool(keys) and bool((sd[keys[0]][:, ballast_dim] != 0).all())+++__all__ = ["MUTE", "is_muted", "mute", "torch"]diff --git a/experimental/riscv/chipformer/q35/prims.py b/experimental/riscv/chipformer/q35/prims.pynew file mode 100644index 0000000..93cb165--- /dev/null+++ b/experimental/riscv/chipformer/q35/prims.py@@ -0,0 +1,238 @@+"""Exact gadgets for the q35 IR.++MLP gadgets are lists of unit terms ``(coef, a, b)`` (``coef * a * relu(b)``) so several can share+one node. They keep every positive gate value a bf16 integer (|b| <= 256), which makes silu an+exact relu; the negative side may be any size, since silu(<= -104) is exactly 0.++Values are represented as+ bits list of 0/1 Exprs, LSB first (an integer is their weighted sum, free)+ one-hot list of 0/1 Exprs, exactly one set (bits of the index are linear, free)+ int an Expr holding a small integer (|value| <= 256 inside a gate)+"""++from fractions import Fraction++from chipformer.dsl.graph import Expr+from chipformer.q35.ir import A_BIT, A_REF, REF_COLS, Chan, Graph, K++Terms = list[tuple]+++def cat(term_lists) -> Terms:+ """Concatenate unit-term lists."""+ return [t for ts in term_lists for t in ts]+++def from_bits(bits: list) -> Expr:+ return sum((b * (1 << i) for i, b in enumerate(bits)), Expr())+++def onehot_bits(oh: list, n: int) -> list:+ """Bits of the index of a one-hot."""+ return [sum((e for v, e in enumerate(oh) if v >> i & 1), Expr()) for i in range(n)]+++def onehot_int(oh: list) -> Expr:+ return sum((v * e for v, e in enumerate(oh)), Expr())+++class Circ(Graph):+ # --- unit terms -----------------------------------------------------------++ def t_const(self, c) -> Terms:+ return [(c, self.one, self.one)]++ def t_and(self, lits, coef=1, a=None) -> Terms:+ """coef * a * AND(lits) for 0/1 literals.++ The gate is 4 * (sum - n + 1/2): +2 when all are set, <= -2 otherwise, so a false AND+ is exactly 0 even when the literals carry fp32 noise (silu(-256) == -0).+ """+ a = self.one if a is None else a+ return [(Fraction(coef) / 2, a, 4 * sum(lits, Expr()) - (4 * len(lits) - 2))]++ def t_ge(self, z, theta, lo, hi, coef=1, lits=()) -> Terms:+ """coef * [z >= theta] (AND lits) for an integer z in [lo, hi]."""+ if theta > hi:+ return []+ n = len(lits)+ if theta <= lo and not n:+ return self.t_const(coef)+ if hi - theta + 1 <= 256:+ m = 1 << (hi - lo).bit_length() # > hi - lo, and a bf16 weight+ zz = z + m * (sum(lits, Expr()) - n) if n else z+ return [(coef, self.one, zz - theta + 1), (-coef, self.one, zz - theta)]+ assert theta - lo <= 256, (theta, lo, hi)+ base = self.t_and(lits, coef) if n else self.t_const(coef)+ return base + self.t_lt(z, theta, lo, hi, -coef, lits)++ def t_lt(self, z, theta, lo, hi, coef=1, lits=()) -> Terms:+ """coef * [z < theta] (AND lits) for an integer z in [lo, hi]."""+ if theta <= lo:+ return []+ n = len(lits)+ if theta > hi and not n:+ return self.t_const(coef)+ if theta - lo <= 256:+ m = 1 << (hi - lo).bit_length()+ zz = z - m * (sum(lits, Expr()) - n) if n else z+ return [(coef, self.one, theta - zz), (-coef, self.one, theta - zz - 1)]+ assert hi - theta + 1 <= 256, (theta, lo, hi)+ base = self.t_and(lits, coef) if n else self.t_const(coef)+ return base + self.t_ge(z, theta, lo, hi, -coef, lits)++ def t_in(self, z, a, b, lo, hi, coef=1, lits=()) -> Terms:+ """coef * [a <= z < b] (AND lits)."""+ a, b = max(a, lo), min(b, hi + 1)+ if a >= b:+ return []+ if a == lo:+ return self.t_lt(z, b, lo, hi, coef, lits)+ if b == hi + 1:+ return self.t_ge(z, a, lo, hi, coef, lits)+ if lits:+ return self.t_ge(z, a, lo, hi, coef, lits) + self.t_ge(z, b, lo, hi, -coef, lits)+ lt_cost = (b - lo, a - lo)+ if max(lt_cost) <= 256:+ return self.t_lt(z, b, lo, hi, coef) + self.t_lt(z, a, lo, hi, -coef)+ return self.t_ge(z, a, lo, hi, coef) + self.t_ge(z, b, lo, hi, -coef)++ def t_eq(self, z, v, lo, hi, coef=1, lits=()) -> Terms:+ return self.t_in(z, v, v + 1, lo, hi, coef, lits)++ # --- nodes ----------------------------------------------------------------++ def node(self, terms: Terms, name: str) -> Expr:+ if not terms:+ return Expr()+ return self.ffn(terms, name)++ def and_(self, lits, name: str) -> Expr:+ return self.node(self.t_and(lits), name)++ def onehot_of_bits(self, bits: list, name: str, n: int | None = None) -> list:+ """One-hot of the integer with these bits (values 0..2^len-1, or 0..n-1)."""+ size = n or 1 << len(bits)+ out = []+ for v in range(size):+ lits = [b if v >> i & 1 else self.one - b for i, b in enumerate(bits)]+ out.append(self.and_(lits, f"{name}=={v}"))+ return out++ def nib_onehot(self, z, lo, hi, name: str, div: int = 1) -> list:+ """One-hot of (z // div) mod 16 for an integer z in [lo, hi] (lo >= 0)."""+ out = []+ for x in range(16):+ terms = []+ k0 = lo // (16 * div)+ for k in range(k0, hi // (16 * div) + 1):+ a = div * (16 * k + x)+ terms += self.t_in(z, a, a + div, lo, hi)+ out.append(self.node(terms, f"{name}[{x}]"))+ return out++ def select_bits(self, cases: list[tuple[list, list]], n: int, name: str) -> list:+ """Bits of sum(cond * value) for one-hot conditions; values are bit lists or ints."""+ out = []+ for i in range(n):+ terms = []+ for lits, val in cases:+ if isinstance(val, int):+ if val >> i & 1:+ terms += self.t_and(lits)+ elif i < len(val):+ b = val[i]+ if isinstance(b, int):+ if b:+ terms += self.t_and(lits)+ else:+ terms += self.t_and([*lits, b])+ out.append(self.node(terms, f"{name}.b{i}"))+ return out++ # --- 12-bit arithmetic ----------------------------------------------------++ def add12(self, a_lo, a_hi_oh: list, d, d_max: int, name: str, a_lo_max: int = 255):+ """One-hot nibbles of (A + d) mod 4096, where A = a_lo + 256 * index(a_hi_oh)."""+ lo_hi = a_lo_max + d_max+ L = a_lo + d+ n0 = self.nib_onehot(L, 0, lo_hi, f"{name}.n0")+ n1 = self.nib_onehot(L, 0, lo_hi, f"{name}.n1", div=16)+ n2 = []+ for x in range(16):+ terms = (self.t_lt(L, 256, 0, lo_hi, lits=[a_hi_oh[x]])+ + self.t_ge(L, 256, 0, lo_hi, lits=[a_hi_oh[(x - 1) % 16]]))+ n2.append(self.node(terms, f"{name}.n2[{x}]"))+ return n0, n1, n2++ # --- heads ----------------------------------------------------------------++ def slot_head(self, name, keys: dict, queries: dict, data: dict, ref, a=0,+ outs=None, gate=None, tiny: float = 0.0) -> list:+ """A head whose rows are 0/1 data columns plus a ballast.++ keys: slot -> Chan or pre-activation; queries likewise; data: column (>= 2) -> Chan or+ Expr giving the *bit* (0/1) to store; ref: Chan or Expr, 1 where the row written carries+ the ballast. Returns one dim per data column, holding the read bit.+ """+ def chan(c, scale):+ if isinstance(c, Chan):+ return Chan(c.expr * scale, c.taps)+ return Chan(c * scale)++ def sel(c):+ """Keys and queries: K * (2c - 1), so c = 1 gives K and c <= 0 gives <= -K. A+ false entry is then exactly 0 after silu even with fp32 noise in c, which matters:+ a leak into another row would be rescaled to full size by the output norm."""+ if isinstance(c, Chan):+ return Chan(c.expr * (2 * K) - self.one * K, c.taps)+ return Chan(c * (2 * K) - self.one * K)++ v = {col: chan(c, A_BIT) for col, c in data.items()}+ for col in REF_COLS:+ v[col] = chan(ref, A_REF)+ gate = self.one if gate is None else gate+ cols = sorted(data) if outs is None else outs+ z = {col: K * gate for col in cols}+ return self.head(name, {s: sel(c) for s, c in queries.items()},+ {s: sel(c) for s, c in keys.items()}, v, a, z,+ [(col, 2.0**7) for col in cols], tiny=tiny)+++def sel(c, e):+ """Key / query pre-activation K * (2e - 1) for a 0/1 (or <= 0 when false) form e."""+ if isinstance(e, Chan):+ return Chan(e.expr * (2 * K) - c.one * K, e.taps)+ return Chan(e * (2 * K) - c.one * K)+++def chan(e, scale):+ if isinstance(e, Chan):+ return Chan(e.expr * scale, e.taps)+ return Chan(e * scale)+++def thead(c, name, keys, queries, vheads, tiny=0.0):+ """A head of 0/1 tables: one dict per value head with data {column: 0/1 form or Chan},+ ref (1 where a written row carries the ballast), optional gate (0/1: its outputs), beta+ (write gate) and into (dims to add the outputs onto). Returns each value head's outputs."""+ specs = []+ for x in vheads:+ v = {col: chan(e, A_BIT) for col, e in x["data"].items()}+ for col in REF_COLS:+ v[col] = chan(x["ref"], A_REF)+ gate = c.one if x.get("gate") is None else x["gate"]+ cols = sorted(x["data"])+ specs.append({"v": v, "a": x.get("a", 0), "z": {col: K * gate for col in cols},+ "outs": [(col, 2.0**7) for col in cols], "beta": x.get("beta"),+ "into": x.get("into")})+ first, rest = specs[0], specs[1:]+ outs = c.head(name, {s: sel(c, e) for s, e in queries.items()},+ {s: sel(c, e) for s, e in keys.items()}, first["v"], first["a"], first["z"],+ first["outs"], tiny=tiny, beta=first["beta"], into=first["into"], vx=rest)+ res, i = [], 0+ for x in specs:+ n = len(x["outs"])+ res.append(outs[i:i + n])+ i += n+ return resdiff --git a/experimental/riscv/chipformer/q35/record.py b/experimental/riscv/chipformer/q35/record.pynew file mode 100644index 0000000..3127a35--- /dev/null+++ b/experimental/riscv/chipformer/q35/record.py@@ -0,0 +1,226 @@+"""Record a game played by a q35 checkpoint and render it to video at real-time speed.++ python -m chipformer.q35.record play MODEL tetris --seconds 60 --out run.npz (GPU box)+ python -m chipformer.q35.record render run.npz run.mp4 (needs ffmpeg)++``play`` runs the stock model with greedy decoding through ``fast.Runner``; the host only injects+TICKs every ``ipf`` instructions and key writes chosen by a simple bot from the screen. The trace+is verified record by record before it is saved. ``render`` replays the model's own writes and+shows one frame per TICK, i.e. at the speed the game would run on a real CHIP-8.+"""++import argparse+import json+import subprocess+import time+from pathlib import Path++import numpy as np++from chipformer import spec+from chipformer.ref.emu import apply_writes, display_pixels++ROOT = Path(__file__).resolve().parents[2]+++class TetrisBot:+ """Port of the browser demo's recorder bot: probe rotations, place by a heuristic score."""++ W0, COLS, ROWS, TOP = 27, 10, 31, 6++ def __init__(self, keys):+ self.k = keys+ self.was_top = False+ self.state, self.wait = "idle", 0+ self.stack = [[0] * 10 for _ in range(31)]++ def grid(self, px):+ return [[px[y][self.W0 + x] for x in range(self.COLS)] for y in range(self.ROWS)]++ @staticmethod+ def evaluate(st, cells, x):+ w = max(c[1] for c in cells) + 1+ h = max(c[0] for c in cells) + 1+ if x + w > 10:+ return None+ y = -h++ def fits(yy):+ return all(yy + dy < 31 and (yy + dy < 0 or not st[yy + dy][x + dx])+ for dy, dx in cells)++ while fits(y + 1):+ y += 1+ if any(y + dy < 0 for dy, _ in cells):+ return None+ g = [r[:] for r in st]+ for dy, dx in cells:+ g[y + dy][x + dx] = 1+ lines = 0+ r = 30+ while r >= 0:+ if all(g[r]):+ del g[r]+ g.insert(0, [0] * 10)+ lines += 1+ else:+ r -= 1+ hs = [next((31 - r for r in range(31) if g[r][c]), 0) for c in range(10)]+ holes = 0+ for c in range(10):+ seen = False+ for r in range(31):+ if g[r][c]:+ seen = True+ elif seen:+ holes += 1+ bump = sum(abs(hs[i] - hs[i + 1]) for i in range(9))+ return -0.51 * sum(hs) + 0.76 * lines - 0.72 * holes - 0.18 * bump++ def frame(self, px) -> set:+ """Keys to hold this frame (closed loop on the screen, like the browser bot)."""+ g = self.grid(px)+ top = [(y, x) for y in range(self.TOP) for x in range(10) if g[y][x]]+ spawned = bool(top) and not self.was_top+ self.was_top = bool(top)+ if spawned:+ self.stack = [r[:] if y >= self.TOP else [0] * 10 for y, r in enumerate(g)]+ self.state, self.wait = "probe", 0+ self.shapes = []+ if self.state == "idle":+ return set()+ piece = [(y, x) for y in range(31) for x in range(10)+ if g[y][x] and not self.stack[y][x] and y < 16]+ if self.wait > 0:+ self.wait -= 1+ return set()+ if self.state == "probe": # record each rotation's shape, then pick the best placement+ if len(piece) == 4:+ y0, x0 = min(c[0] for c in piece), min(c[1] for c in piece)+ self.shapes.append(sorted((y - y0, x - x0) for y, x in piece))+ if len(self.shapes) < 4:+ self.wait = 3+ return {self.k["a"]}+ best = None+ for r, cs in enumerate(self.shapes):+ for x in range(10):+ v = self.evaluate(self.stack, cs, x)+ if v is not None and (best is None or v > best[0]):+ best = (v, r, x)+ if best is None:+ self.state = "drop"+ return set()+ _, self.rot, self.target = best+ self.state = "rotate"+ return set()+ if self.state == "rotate":+ if self.rot > 0:+ self.rot -= 1+ self.wait = 3+ return {self.k["a"]}+ self.state = "move"+ if self.state == "move":+ if len(piece) != 4:+ return set()+ x = min(c[1] for c in piece)+ if x < self.target:+ return {self.k["right"]}+ if x > self.target:+ return {self.k["left"]}+ self.state = "drop"+ if self.state == "drop":+ return {self.k["down"]}+ return set()+++def play(model_dir: str, game: str, seconds: float, out: str, max_len: int = 65536) -> None:+ import torch+ from transformers import AutoModelForCausalLM, AutoModelForImageTextToText++ from chipformer.check import verify_trace+ from chipformer.q35.fast import Runner++ meta_path = Path(model_dir) / "chipformer.json"+ merged = meta_path.exists()+ cls = AutoModelForImageTextToText if merged else AutoModelForCausalLM+ model = cls.from_pretrained(model_dir, dtype=torch.bfloat16).cuda().eval()+ off = json.loads(meta_path.read_text())["chip_token_offset"] if merged else 0+ games = {g["id"]: g for g in json.loads((ROOT / "games" / "manifest.json").read_text(+ encoding="utf-8"))["games"]}+ g = games[game]+ runner = Runner(model, max_len=max_len, token_offset=off)+ s = runner.session((ROOT / "games" / g["file"]).read_bytes(), tuple(g["quirks"]))+ bot = TetrisBot(g["keys"]) if game == "tetris" else None+ held: set = set()+ frames = int(seconds * 60)+ t = time.time()+ for f in range(frames):+ want = bot.frame(s.pixels()) if bot else set()+ for k in held - want:+ s.set_key(k, False)+ for k in want - held:+ s.set_key(k, True)+ held = want+ s.tick()+ for _ in range(g["ipf"]):+ s.step()+ if f % 600 == 0:+ el = time.time() - t+ print(f"frame {f}/{frames} {len(s.tokens)} tokens {el:.0f}s", flush=True)+ n, err = verify_trace(s.tokens)+ print(f"verified {n} records, error: {err}, {time.time() - t:.0f}s")+ np.savez_compressed(out, tokens=np.array(s.tokens, dtype=np.uint32), game=game,+ model=str(model_dir), verified=err is None)+++def render(trace: str, out: str, scale: int = 12) -> None:+ from chipformer.trace import parse++ data = np.load(trace)+ tokens = data["tokens"].tolist()+ cells = [0] * spec.NUM_CELLS+ frames, prev = [], None+ for r in parse(tokens, allow_reboot=True):+ if r.kind == "boot":+ cells = [0] * spec.NUM_CELLS+ apply_writes(cells, r.writes)+ if r.kind == "tick":+ px = np.array(display_pixels(cells), dtype=np.uint8)+ shown = px if prev is None else np.maximum(px, prev) # soften CHIP-8 flicker+ frames.append(shown)+ prev = px+ h, w = 32 * scale, 64 * scale+ ink, lit = np.array([25, 34, 31], np.uint8), np.array([221, 245, 158], np.uint8)+ proc = subprocess.Popen(["ffmpeg", "-y", "-loglevel", "error", "-f", "rawvideo",+ "-pix_fmt", "rgb24", "-s", f"{w}x{h}", "-r", "60", "-i", "-",+ "-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "18", out],+ stdin=subprocess.PIPE)+ for px in frames:+ img = np.where(px[..., None].astype(bool), lit, ink)+ img = img.repeat(scale, 0).repeat(scale, 1)+ proc.stdin.write(img.tobytes())+ proc.stdin.close()+ proc.wait()+ print(f"{out}: {len(frames)} frames ({len(frames) / 60:.1f} s at 60 fps)")+++def main() -> None:+ ap = argparse.ArgumentParser()+ sub = ap.add_subparsers(dest="cmd", required=True)+ p = sub.add_parser("play")+ p.add_argument("model")+ p.add_argument("game")+ p.add_argument("--seconds", type=float, default=30)+ p.add_argument("--out", default="run.npz")+ r = sub.add_parser("render")+ r.add_argument("trace")+ r.add_argument("out")+ a = ap.parse_args()+ if a.cmd == "play":+ play(a.model, a.game, a.seconds, a.out)+ else:+ render(a.trace, a.out)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/report.py b/experimental/riscv/chipformer/q35/report.pynew file mode 100644index 0000000..f14ee3e--- /dev/null+++ b/experimental/riscv/chipformer/q35/report.py@@ -0,0 +1,235 @@+"""Check a merged Qwen3.5 + CHIP-8 checkpoint against its base model.++ python -m chipformer.q35.report BASE MERGED [--device cuda] [--out report.json]++1. CHIP-8: teacher-forced predictions on game traces, pure and inside a chat (mixed context).+2. Leakage: the largest chip-token logit at text positions.+3. Text: greedy generations and logits vs the base model, next to the noise floor between two+ attention implementations of the base model itself.+"""++import argparse+import json+import time+from pathlib import Path++import numpy as np+import torch++from chipformer.check import run_machine++ROOT = Path(__file__).resolve().parents[2]+PROMPTS = [+ ("chat", "Explain why the sky is blue in two sentences."),+ ("raw", 'def fibonacci(n):\n """Return the n-th Fibonacci number."""\n'),+ ("chat", "What is 37 * 43? Think step by step."),+ ("chat", "Traduis en français : The weather is lovely today and I want to go for a walk."),+ ("chat", "写一首关于秋天的短诗。"),+ ("chat", "List three differences between TCP and UDP."),+ ("raw", "The CHIP-8 is a virtual machine from the 1970s that"),+ ("raw", "Q: If a train leaves at 3pm travelling 60 km/h, how far has it gone by 5:30pm?\nA:"),+ ("chat", "Write a haiku about a lighthouse."),+ ("chat", "Give me a Python one-liner that reverses the words in a sentence."),+ ("chat", "What causes the seasons on Earth?"),+ ("raw", "In 1969, Apollo 11"),+ ("chat", "Summarise the plot of Romeo and Juliet in three sentences."),+ ("chat", "Solve for x: 3x + 7 = 22."),+ ("raw", "SELECT name, COUNT(*) FROM orders"),+ ("chat", "¿Cuál es la capital de Australia y por qué no es Sídney?"),+]+++def game_trace(gid: str, steps: int, seed: int = 2):+ games = json.loads((ROOT / "games" / "manifest.json").read_text(encoding="utf-8"))["games"]+ g = next(g for g in games if g["id"] == gid)+ rom = (ROOT / "games" / g["file"]).read_bytes()+ return run_machine(rom, steps, ipf=g["ipf"], seed=seed, key_rate=0.3, quirks=tuple(g["quirks"]))+++def load(path, device, impl="sdpa"):+ from transformers import AutoModelForImageTextToText++ return AutoModelForImageTextToText.from_pretrained(+ path, dtype=torch.bfloat16, attn_implementation=impl).to(device).eval()+++def encode(tok, kind, text):+ if kind == "chat":+ return tok.apply_chat_template([{"role": "user", "content": text}],+ add_generation_prompt=True, return_tensors="pt",+ enable_thinking=False)["input_ids"][0].tolist()+ return tok(text).input_ids++[email protected]_grad()+def text_run(model, prompts, device, n_new):+ gens, logits = [], []+ for ids in prompts:+ x = torch.tensor([ids], device=device)+ g = model.generate(x, max_new_tokens=n_new, do_sample=False)+ gens.append(g[0, len(ids):].tolist())+ logits.append(model(input_ids=g).logits[0, :, :248320].float().cpu())+ return gens, logits+++def compare(a, b):+ ga, la = a+ gb, lb = b+ same = [x == y for x, y in zip(ga, gb)]+ first = [next((i for i, (p, q) in enumerate(zip(x, y)) if p != q), None)+ for x, y in zip(ga, gb)]+ dmax, kl, top1 = [], [], []+ for x, y, s in zip(la, lb, same):+ if not s:+ continue+ dmax.append(float((x - y).abs().max()))+ p, q = torch.log_softmax(x, -1), torch.log_softmax(y, -1)+ kl.append(float((p.exp() * (p - q)).sum(-1).mean()))+ top1.append(float((x.argmax(-1) == y.argmax(-1)).float().mean()))+ return {"identical_generations": f"{sum(same)}/{len(same)}", "first_divergence": first,+ "max_abs_dlogit": max(dmax, default=None), "mean_kl": float(np.mean(kl)) if kl else None,+ "top1_agreement": float(np.mean(top1)) if top1 else None}++[email protected]_grad()+def chip_check(model, off, tokens, generated, device, prefix=(), suffix=()):+ ids = list(prefix) + [t + off for t in tokens] + list(suffix)+ logits = model(input_ids=torch.tensor([ids], device=device)).logits[0].float().cpu()+ s = len(prefix)+ pred = logits[s : s + len(tokens)].argmax(-1).numpy() - off+ want = np.array(tokens[1:] + [-1])+ gen = np.array(generated[1:] + [False])+ text_pos = list(range(s)) + list(range(s + len(tokens), len(ids)))+ leak = float(logits[text_pos, off:off + 8708].max()) if text_pos else None+ return {"checked": int(gen.sum()), "mismatches": int((gen & (pred != want)).sum()),+ "min_predicted_logit": float(logits[s : s + len(tokens)].max(-1).values[gen].min()),+ "max_chip_logit_at_text": leak}++[email protected]_grad()+def post_trace(model, tok, off, device, lengths=(0, 2000, 10000)):+ """Does Qwen remember the conversation after a CHIP-8 run in the middle of it?++ Compares the merged model on [prompt, trace, question] with itself on [prompt, question]+ where the question keeps the positions it has after the trace, so only the trace's content+ differs. Reports KL on the answer's next-token distributions and both greedy answers.+ """+ pre = tok.apply_chat_template(+ [{"role": "user", "content": "Remember this: my locker code is 4817 and my cat is called "+ "Biscuit. I'll ask about it later. First, run this CHIP-8 program."},+ {"role": "assistant", "content": "Sure, running it now."}],+ return_tensors="pt", enable_thinking=False)["input_ids"][0].tolist()+ q = tok.apply_chat_template(+ [{"role": "user", "content": "What is my locker code and what is my cat called?"}],+ add_generation_prompt=True, return_tensors="pt", enable_thinking=False)["input_ids"][0]+ q = q.tolist()+ out = {}+ m = game_trace("tetris", 4000, seed=7)+ for n in lengths:+ trace = [t + off for t in m.tokens[:n]]+ if trace and m.tokens[n - 1] != 8705: # end on a STEP+ last = max(i for i in range(n) if m.tokens[i] == 8705)+ trace = trace[: last + 1]+ full = torch.tensor([pre + trace + q], device=device)+ gap = torch.tensor([pre + q], device=device)+ pos = torch.cat([torch.arange(len(pre)), torch.arange(len(q)) + len(pre) + len(trace)])+ a = model.generate(full, max_new_tokens=24, do_sample=False)[0, full.shape[1]:]+ b = model.generate(gap, position_ids=pos[None].to(device), max_new_tokens=24,+ do_sample=False)[0, gap.shape[1]:] if trace else a+ la = model(input_ids=torch.cat([full, a[None]], 1)).logits[0, -len(a) - 1:-1].float()+ pos_b = torch.cat([pos, torch.arange(len(a)) + pos[-1] + 1])[None].to(device)+ lb = model(input_ids=torch.cat([gap, a[None]], 1), position_ids=pos_b).logits[+ 0, -len(a) - 1:-1].float()+ p, r = torch.log_softmax(la[:, :248320], -1), torch.log_softmax(lb[:, :248320], -1)+ out[n] = {"trace_tokens": len(trace), "kl": float((p.exp() * (p - r)).sum(-1).mean()),+ "answer_with_trace": tok.decode(a, skip_special_tokens=True),+ "answer_without": tok.decode(b, skip_special_tokens=True)}+ return out+++PRINT_ROM = bytes([0xA2, 0x0A, 0x60, 4, 0x00, 0xFD, 0x12, 0x06, 0, 0, 4, 8, 1, 7]) # prints 4817++[email protected]_grad()+def bridge_demo(model, tok, meta, device, rom: bytes = PRINT_ROM, n_new: int = 120):+ """One generate(): a chat turn hands over to a CHIP-8 program, which runs, prints its result+ as text, halts, and Qwen continues. Checks the machine part against the reference."""+ from chipformer.q35 import bridge+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ off = meta["chip_token_offset"]+ text_ids = {int(k): v for k, v in meta["text_ids"].items()}+ to_id = {**text_ids}+ pre = tok.apply_chat_template(+ [{"role": "user", "content": "Run the program and tell me the number it prints, "+ "then that number plus one."}],+ add_generation_prompt=True, return_tensors="pt", enable_thinking=False)["input_ids"][0]+ pre = pre.tolist() + tok("Running it now.\n", add_special_tokens=False).input_ids+ boot = boot_tokens(Chip8(rom).cells)+ ids = pre + [t + off for t in boot]+ out = model.generate(torch.tensor([ids], device=device), max_new_tokens=n_new,+ do_sample=False)[0, len(ids):].tolist()+ # reference: the machine records and the print, as model token ids+ bm = bridge.BridgeMachine(rom)+ while not bm.halted:+ bm.step()+ want = [to_id[t] if bridge.is_text(t) else t + off for t in bm.tokens[len(boot):]]+ got_machine = out[:len(want)]+ return {"machine_and_print_exact": got_machine == want,+ "printed": tok.decode(out[:len(want)][-(len(want) - want.index(text_ids[bridge.NL])):]),+ "continuation": tok.decode(out[len(want):], skip_special_tokens=True),+ "n_machine_tokens": len(want)}+++def main() -> None:+ from transformers import AutoTokenizer++ ap = argparse.ArgumentParser()+ ap.add_argument("base")+ ap.add_argument("merged")+ ap.add_argument("--device", default="cuda" if torch.cuda.is_available() else "cpu")+ ap.add_argument("--new", type=int, default=64)+ ap.add_argument("--out", default=None)+ args = ap.parse_args()+ meta = json.loads((Path(args.merged) / "chipformer.json").read_text())+ off = meta["chip_token_offset"]+ tok = AutoTokenizer.from_pretrained(args.merged)+ prompts = [encode(tok, k, t) for k, t in PROMPTS]+ rep = {"meta": meta}++ t = time.time()+ model = load(args.merged, args.device)+ rep["params_B"] = sum(p.numel() for p in model.parameters()) / 1e9+ for gid in ("brix", "tetris", "space-invaders"):+ try:+ m = game_trace(gid, 400)+ except StopIteration:+ continue+ rep[f"chip_{gid}"] = chip_check(model, off, m.tokens, m.generated, args.device)+ m = game_trace("tetris", 200, seed=5)+ pre = encode(tok, "chat", "Run this CHIP-8 program for me.")+ post = tok("\nThe machine is running Tetris now. What should I do next?",+ add_special_tokens=False).input_ids+ rep["chip_mixed"] = chip_check(model, off, m.tokens, m.generated, args.device, pre, post)+ rep["post_trace"] = post_trace(model, tok, off, args.device)+ if meta.get("layout") == "v2":+ rep["bridge"] = bridge_demo(model, tok, meta, args.device)+ merged = text_run(model, prompts, args.device, args.new)+ rep["sample_merged"] = tok.decode(merged[0][2])+ del model+ base = load(args.base, args.device)+ ref = text_run(base, prompts, args.device, args.new)+ rep["sample_base"] = tok.decode(ref[0][2])+ del base+ eager = text_run(load(args.base, args.device, "eager"), prompts, args.device, args.new)+ rep["noise_floor_sdpa_vs_eager"] = compare(ref, eager)+ rep["base_vs_merged"] = compare(ref, merged)+ rep["seconds"] = time.time() - t+ print(json.dumps(rep, indent=2, ensure_ascii=False))+ if args.out:+ Path(args.out).write_text(json.dumps(rep, indent=2, ensure_ascii=False), encoding="utf-8")+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/spike.py b/experimental/riscv/chipformer/q35/spike.pynew file mode 100644index 0000000..91590a3--- /dev/null+++ b/experimental/riscv/chipformer/q35/spike.py@@ -0,0 +1,208 @@+"""Phase 0 spike: check that the real Qwen3.5 modules can hold exact machine state.++One Qwen3.5 decoder layer (Gated DeltaNet + MLP) with hand-set weights, fed `inputs_embeds`+directly. The residual stream layout below stands in for the chip dims of the merged model.++Every chip token carries a ballast of 2^40 in two dims. That pins the RMSNorm scale to exactly+2^-36, so a residual value x reaches the projections as x * 2^-36, and the weights carry the+2^36 back. Text tokens carry no ballast and leave the chip state untouched.++Heads (all 16 key heads, one value head each):+ 0..13 RAM: write the token's bits to (whead, wslot), read (rhead, rslot)+ 14 RAM whose state is cleared on RST tokens (display CLS)+ 15 counter: x = 2^-w, w = chip tokens since the last CNTW token+"""++import torch+from transformers import Qwen3_5TextConfig, Qwen3_5TextModel++D = 512+B0, B1 = 0, 1 # ballast+ISW, RST, CNTW = 2, 3, 4 # write flag, display clear, counter reset+WSLOT, WHEAD, BITS = 8, 136, 152+RSLOT, RHEAD = 160, 288+OUT_BITS, OUT_FOUND, OUT_CNT = 304, 312, 313+S_DIM, C_DIM, AND_DIM, NM = 320, 356, 392, 36 # counter decode stages, AND gadget+GARBAGE = 400 # dims 400.. stand in for Qwen's own dims++BALLAST = 2.0**40+FEAT = 2.0**36 # residual value 1 -> normed 2^-36; weights carry this factor back+ONE = 16.0 # the normed ballast+K = 128.0 # silu(K n) == K n for integers n >= 1, and -0 for n <= -1+A_REF = 2.0**21 # ballast columns of a value row+NH, HD = 16, 128+RAM_HEADS, CLS_HEAD, CNT_HEAD = range(14), 14, 15+++def config():+ return Qwen3_5TextConfig(+ vocab_size=16,+ hidden_size=D,+ intermediate_size=128,+ # HF's cache needs one attention layer to count positions; it has zero weights here+ num_hidden_layers=2,+ layer_types=["linear_attention", "full_attention"],+ num_attention_heads=2,+ num_key_value_heads=1,+ head_dim=256,+ linear_num_key_heads=NH,+ linear_num_value_heads=NH,+ linear_key_head_dim=HD,+ linear_value_head_dim=HD,+ tie_word_embeddings=False,+ )+++def build(dtype):+ model = Qwen3_5TextModel(config()).eval()+ for p in model.parameters():+ p.data.zero_()+ layer = model.layers[0]+ la = layer.linear_attn+ qkv = la.in_proj_qkv.weight.data # (3*NH*HD, D)+ z, b, a = la.in_proj_z.weight.data, la.in_proj_b.weight.data, la.in_proj_a.weight.data+ out = la.out_proj.weight.data # (D, NH*HD)+ q_off, k_off, v_off = 0, NH * HD, 2 * NH * HD++ def row(w, terms): # terms: {dim: coef}; the ballast dim carries constants+ for d, c in terms.items():+ w[d] += c / ONE if d == B0 else c * FEAT++ for h in range(NH):+ b[h, B0] = 128.0 / ONE # beta = sigmoid(128) = 1+ z[h * HD : (h + 1) * HD, B0] = K / ONE # silu(z) = 128 on chip tokens, 0 on text+ la.dt_bias.data.fill_(-110.0) # softplus(a - 110) = 0 at a = 0: decay exactly 1+ la.A_log.data.zero_()+ la.norm.weight.data.fill_(1.0)+ la.conv1d.weight.data[:, 0, -1] = 1.0 # current-token tap only++ for h in [*RAM_HEADS, CLS_HEAD]:+ for s in range(HD):+ row(qkv[q_off + h * HD + s], {RSLOT + s: K, RHEAD + h: K, B0: -K})+ row(qkv[k_off + h * HD + s], {WSLOT + s: K, WHEAD + h: K, ISW: K, B0: -2 * K})+ v = v_off + h * HD+ row(qkv[v + 0], {B0: A_REF})+ row(qkv[v + 1], {B0: A_REF})+ for i in range(8):+ row(qkv[v + 2 + i], {BITS + i: K})+ # normed bit column = 2^-11 (bf16 snapped); times silu(z) = 128 -> 2^-4+ out[OUT_BITS + i, h * HD + 2 + i] = 16.0+ out[OUT_FOUND, h * HD] = 2.0**-10 # normed ballast column 8, times 128+ row(a[CLS_HEAD], {RST: 238.0}) # softplus(128) = 128: decay exp(-128) = 0++ # counter: row 0 = count (written at CNTW), row 1 = ref (refreshed on every other token)+ h = CNT_HEAD+ row(qkv[q_off + h * HD + 0], {B0: K})+ row(qkv[q_off + h * HD + 1], {B0: K})+ row(qkv[k_off + h * HD + 0], {CNTW: K})+ row(qkv[k_off + h * HD + 1], {B0: K, CNTW: -K})+ v = v_off + h * HD+ row(qkv[v + 0], {CNTW: K})+ row(qkv[v + 1], {B0: 2.0**20, CNTW: -(2.0**20)})+ row(qkv[v + 2], {B0: 2.0**20, CNTW: -(2.0**20)})+ row(a[h], {B0: 110.0, CNTW: 128.0}) # softplus(0) = ln 2 per token, softplus(128) at reset+ # normed count column = 8 * 2^-w * 128 / 2^20 = 2^-(w+10), times 128 -> 2^-(w+3)+ out[OUT_CNT, h * HD] = 8.0++ # MLPs: silu(K s) / K is an exact relu(s) when K s is a bf16 integer multiple of K.+ # Layer 0: s_m = relu(5 - 6 * 2^m * x) and an AND of three bits. Layer 1: c_m = [w > m].+ m0, m1 = model.layers[0].mlp, model.layers[1].mlp+ for m in range(NM):+ row(m0.gate_proj.weight.data[m], {B0: 5 * K, OUT_CNT: -6 * 2.0**m * K})+ row(m0.up_proj.weight.data[m], {B0: 1 / K})+ m0.down_proj.weight.data[S_DIM + m, m] = 1.0+ for j, c in enumerate((0.0, -1.0)):+ u = 2 * m + j+ row(m1.gate_proj.weight.data[u], {S_DIM + m: K, B0: c * K})+ row(m1.up_proj.weight.data[u], {B0: (1 - 2 * j) / K})+ m1.down_proj.weight.data[C_DIM + m, u] = 1.0+ u = NM+ row(m0.gate_proj.weight.data[u], {OUT_BITS: K, OUT_BITS + 1: K, OUT_FOUND: K, B0: -2 * K})+ row(m0.up_proj.weight.data[u], {B0: 1 / K})+ m0.down_proj.weight.data[AND_DIM, u] = 1.0+ return model.to(dtype)+++def make_sequence(n, seed=0, text_frac=0.1):+ """Random tokens plus the expected chip outputs from a plain Python model of the heads."""+ g = torch.Generator().manual_seed(seed)+ r = lambda hi: int(torch.randint(hi, (1,), generator=g))+ x = torch.zeros(n, D, dtype=torch.float64)+ want = {} # token -> {dim: value}+ ram = {}+ cnt_w = None+ for t in range(n):+ if torch.rand(1, generator=g) < text_frac:+ x[t, GARBAGE:] = torch.randn(D - GARBAGE, generator=g, dtype=torch.float64) * 1e3+ continue+ x[t, B0] = x[t, B1] = BALLAST+ x[t, GARBAGE:] = torch.randn(D - GARBAGE, generator=g, dtype=torch.float64) * 1e3+ rst = r(20) == 0+ cntw = r(25) == 0+ if rst:+ x[t, RST] = 1+ for key in [k for k in ram if k[0] == CLS_HEAD]:+ del ram[key]+ if r(2):+ hd, s, val = r(15), r(HD), r(256)+ x[t, ISW] = 1+ x[t, WSLOT + s] = x[t, WHEAD + hd] = 1+ for i in range(8):+ x[t, BITS + i] = (val >> i) & 1+ ram[hd, s] = val+ exp = {}+ if r(3):+ hd, s = r(15), r(HD)+ x[t, RSLOT + s] = x[t, RHEAD + hd] = 1+ val = ram.get((hd, s))+ for i in range(8):+ exp[OUT_BITS + i] = 0.0 if val is None else float((val >> i) & 1)+ exp[OUT_FOUND] = 0.0 if val is None else 1.0+ exp[AND_DIM] = float(val is not None and val & 3 == 3)+ if cntw:+ x[t, CNTW] = 1+ cnt_w = 0+ elif cnt_w is not None:+ cnt_w += 1+ if cnt_w <= 100: # records are at most 82 tokens; 2^-w stays a normal bf16+ exp[OUT_CNT] = 2.0**-cnt_w+ for m in range(NM):+ exp[C_DIM + m] = float(cnt_w > m)+ else:+ exp[OUT_CNT] = 0.0+ want[t] = exp+ return x, want+++def run(model, x, mode, split=None):+ """Residual after the layer, for prefill (chunked kernel) or prefill+decode (recurrent)."""+ dtype = next(model.parameters()).dtype+ x = x.to(dtype)[None]+ outs = []+ hook = model.layers[-1].register_forward_hook(lambda m, i, o: outs.append(o[0] if isinstance(o, tuple) else o))+ try:+ with torch.no_grad():+ if mode == "prefill":+ model(inputs_embeds=x, use_cache=False)+ else:+ split = split or 1+ o = model(inputs_embeds=x[:, :split], use_cache=True)+ cache = o.past_key_values+ for t in range(split, x.shape[1]):+ o = model(inputs_embeds=x[:, t : t + 1], past_key_values=cache, use_cache=True)+ cache = o.past_key_values+ finally:+ hook.remove()+ return torch.cat([o[0] for o in outs], 0).to(torch.float64)+++def compare(res, want, x):+ """(max abs error on expected outputs, max abs change on text tokens' chip dims, count)."""+ err, n = 0.0, 0+ for t, exp in want.items():+ for d, v in exp.items():+ err = max(err, abs(float(res[t, d]) - v))+ n += 1+ text = [t for t in range(x.shape[0]) if t not in want]+ leak = float((res[text, :GARBAGE] - x[text, :GARBAGE]).abs().max()) if text else 0.0+ return err, leak, ndiff --git a/experimental/riscv/chipformer/q35/spikes_pos.py b/experimental/riscv/chipformer/q35/spikes_pos.pynew file mode 100644index 0000000..b12c7da--- /dev/null+++ b/experimental/riscv/chipformer/q35/spikes_pos.py@@ -0,0 +1,78 @@+"""Spike: does freezing RoPE positions on chip tokens stop Qwen repeating a line after an inline call?++ python -m chipformer.q35.spikes_pos MERGED --ids gsmhard-82 gsmhard-306 --max-new 1500++Decodes each problem twice with a manual greedy loop: positions advancing on every token (what+HF generate does), and positions frozen on chip tokens (Qwen's text sees contiguous positions).+Reports the most repeated line that holds an inline call.+"""++import argparse+import json+import re+from pathlib import Path++import torch++[email protected]_grad()+def decode(model, prompt, chip_lo, chip_hi, max_new, freeze, stop):+ from transformers import DynamicCache++ cache = DynamicCache(config=model.config)+ pos, p = [], 0+ for t in prompt:+ pos.append(p)+ if not (freeze and chip_lo <= t < chip_hi):+ p += 1+ out = model(input_ids=torch.tensor([prompt], device="cuda"),+ position_ids=torch.tensor([pos], device="cuda"), past_key_values=cache,+ use_cache=True)+ gen = []+ for _ in range(max_new):+ t = int(out.logits[0, -1].float().argmax())+ gen.append(t)+ if t in stop:+ break+ pos_t = p+ if not (freeze and chip_lo <= t < chip_hi):+ p += 1+ out = model(input_ids=torch.tensor([[t]], device="cuda"),+ position_ids=torch.tensor([[pos_t]], device="cuda"), past_key_values=cache,+ use_cache=True)+ return gen+++def main():+ from chipformer.q35 import evals+ from chipformer.q35.demo import load+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ ap = argparse.ArgumentParser()+ ap.add_argument("merged")+ ap.add_argument("--ids", nargs="+", default=["gsmhard-82"])+ ap.add_argument("--cond", default="merged_inline")+ ap.add_argument("--max-new", type=int, default=1500)+ args = ap.parse_args()+ meta = json.loads((Path(args.merged) / "chipformer.json").read_text())+ off, nchip = meta["chip_token_offset"], meta["chip_vocab"]+ bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ model, tok = load(args.merged)+ data = {r["id"]: r for r in evals.load_data("gsmhard", 200)}+ stop = {248046, 248044, 248045}+ for i in args.ids:+ prompt = evals.build_prompt(tok, data[i]["question"], args.cond, bios)+ for freeze in (False, True):+ gen = decode(model, prompt, off, off + nchip, args.max_new, freeze, stop)+ shown = re.sub(r"(<\|chip[^|]*\|>)+", "[m]", tok.decode(gen))+ lines = [ln for ln in shown.split("\n") if "[m]" in ln]+ rep = max((lines.count(ln) for ln in lines), default=0)+ print(json.dumps({"id": i, "freeze": freeze, "n": len(gen), "fires": shown.count("[m]"),+ "max_line_repeat": rep, "gold": data[i].get("gold"),+ "tail": shown[-240:]}, ensure_ascii=False), flush=True)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/spikes_replay.py b/experimental/riscv/chipformer/q35/spikes_replay.pynew file mode 100644index 0000000..d7b9cc1--- /dev/null+++ b/experimental/riscv/chipformer/q35/spikes_replay.py@@ -0,0 +1,103 @@+"""Spike: replay a merged model's generation through the circuit IR and compare, token by token.++ python -m chipformer.q35.spikes_replay MERGED --data gsmhard --idx 28 --cond merged_inline++Generates greedily with HF generate (batch 1), maps the tokens to the circuit's alphabet (chip+tokens, text classes, OTHER), evaluates the IR on that stream, and reports every position the+circuit owns where the model's token differs from the IR's prediction, plus each inline fire.+"""++import argparse+import json+import re+from pathlib import Path++import numpy as np+import torch+++def class_map(tok, meta):+ from chipformer.q35 import bridge as B++ m = {int(i): int(cls) for cls, i in meta["text_ids"].items()}+ for cls, strings in B.EXTRA_TEXT.items():+ for s in strings:+ (i,) = tok.encode(s, add_special_tokens=False)+ m[i] = cls+ return m+++def main():+ from chipformer.q35 import bridge as B+ from chipformer.q35 import check as Q+ from chipformer.q35 import chip8_q35, evals+ from chipformer.q35.demo import load+ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import Chip8+ from chipformer.trace import boot_tokens++ ap = argparse.ArgumentParser()+ ap.add_argument("merged")+ ap.add_argument("--data", default="gsmhard")+ ap.add_argument("--idx", type=int, nargs="+", default=[28])+ ap.add_argument("--cond", default="merged_inline")+ ap.add_argument("--max-new", type=int, default=1500)+ ap.add_argument("--n", type=int, default=200)+ ap.add_argument("--ids", help="replay these saved prompt+generated ids (.npy) instead")+ ap.add_argument("--prompt-len", type=int, help="with --ids: the prompt is this many ids")+ args = ap.parse_args()+ meta = json.loads((Path(args.merged) / "chipformer.json").read_text())+ off, nchip = meta["chip_token_offset"], meta["chip_vocab"]+ bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]+ if args.ids:+ from transformers import AutoTokenizer++ model, tok = None, AutoTokenizer.from_pretrained(args.merged)+ else:+ model, tok = load(args.merged)+ cmap = class_map(tok, meta)+ circuit = chip8_q35.build()+ data = evals.load_data(args.data, args.n)+ for i in args.idx:+ prompt = evals.build_prompt(tok, data[i]["question"], args.cond, bios)+ if args.ids:+ saved = np.load(args.ids).tolist()+ if args.prompt_len:+ prompt = saved[:args.prompt_len]+ assert saved[:len(prompt)] == prompt, "saved ids do not start with this prompt"+ gen = saved[len(prompt):]+ else:+ x = torch.tensor([prompt], device="cuda")+ with torch.no_grad():+ out = model.generate(x, max_new_tokens=args.max_new, do_sample=False)+ gen = out[0, len(prompt):].tolist()+ ids = prompt + gen+ stream = [t - off if off <= t < off + nchip else cmap.get(t, B.OTHER) for t in ids]+ pred, m = Q.predictions(circuit, stream)+ from chipformer.q35 import ir++ owner = ir.ev(circuit.owner, m)+ bad = [p for p in range(len(prompt) - 1, len(ids) - 1)+ if owner[p] == 1 and pred[p] != stream[p + 1]]+ text = tok.decode(gen)+ shown = re.sub(r"(<\|chip[^|]*\|>)+", "[m]", text)+ fires = re.findall(r"([\d,]+)\s*(\\times|\*|×|\+|-)\s*([\d,]+)\s*=\[m\]\s*(\d*)", shown)+ wrong = []+ for a, op, b, r in fires:+ a_, b_ = int(a.replace(",", "")), int(b.replace(",", ""))+ want = {"+": a_ + b_, "-": a_ - b_}.get(op, a_ * b_)+ if r != str(want):+ wrong.append((a, op, b, r, want))+ print(json.dumps({"idx": i, "n_gen": len(gen), "ir_mismatch_at_owned": len(bad),+ "first": [(p - len(prompt), int(stream[p + 1]), int(pred[p]))+ for p in bad[:5]],+ "fires": len(fires), "wrong_fires": wrong}), flush=True)+ if bad:+ p = bad[0]+ print("context:", repr(tok.decode(ids[max(len(prompt), p - 40):p + 2])), flush=True)+ if not args.ids:+ np.save(Path(args.merged).parent / f"replay_{args.data}_{i}.npy", np.array(ids))+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/text_eval.py b/experimental/riscv/chipformer/q35/text_eval.pynew file mode 100644index 0000000..3047d4b--- /dev/null+++ b/experimental/riscv/chipformer/q35/text_eval.py@@ -0,0 +1,625 @@+"""Text benchmarks: is a merged Qwen3.5 + CHIP-8 checkpoint as capable as its base on pure text?++ python -m chipformer.q35.text_eval run MODEL --name NAME [--attn sdpa|eager] [--tokenizer BASE]+ python -m chipformer.q35.text_eval table [dist/evals/text] [--ref base]++``run`` evaluates one checkpoint and writes ``OUT/NAME/TASK.json`` (per-item records; a task whose+file exists is skipped, so a killed run resumes). ``table`` compares every run against ``--ref``.+All runs share one tokenizer (pass the base's) and identical batches, so items line up.++Tasks (all prompts in Qwen's chat template with thinking off, except the ARC/HellaSwag contexts):+ mmlu cais/mmlu, 1000 seeded test questions, 0-shot; prediction = argmax over A/B/C/D+ logits at the first answer token+ arc ARC-Challenge test (1172), continuation log-likelihood, acc and acc_norm+ hellaswag 1000 seeded validation items, continuation log-likelihood, acc_norm+ gsm8k 200 seeded test problems, greedy, no tools, final answer in \\boxed{}+ ifeval google/IFEval (541), greedy, scored by lm-eval's checker (strict and loose)+ niah needle (a 7-digit "magic number") in a wikitext-103 haystack at 8k/16k/32k tokens,+ chunked prefill; greedy answer and the teacher-forced log-prob of the needle+ ppl 8 disjoint 32k-token wikitext-103 windows: per-token NLL by position and greedy+ next-token agreement (a dense, long-context fidelity measure)++The noise floor is the base model under another attention implementation (eager vs sdpa), which+changes nothing but the rounding inside Qwen's full-attention layers.+"""++import argparse+import itertools+import json+import math+import random+import re+import sys+import time+from pathlib import Path++import numpy as np+import torch++TASKS = ("mmlu", "arc", "hellaswag", "gsm8k", "ifeval", "niah", "ppl")+TEXT_VOCAB = 248320 # Qwen's own vocabulary; the merged checkpoint's chip tokens come after it+TOKEN_BUDGET = 4096 # tokens per log-likelihood batch+GEN_BATCH = 16+CHUNK = 1024 # prefill chunk for the long-context check, same for every model+++# --- model ------------------------------------------------------------------------------------+++def load(path, attn, mem_gb):+ from transformers import AutoModelForImageTextToText++ if mem_gb:+ total = torch.cuda.get_device_properties(0).total_memory / 2**30+ torch.cuda.set_per_process_memory_fraction(min(1.0, mem_gb / total))+ model = AutoModelForImageTextToText.from_pretrained(+ path, dtype=torch.bfloat16, attn_implementation=attn)+ del model.model.visual # text only; frees ~0.7 GB+ return model.to("cuda").eval()+++def chat(tok, text):+ return tok.apply_chat_template([{"role": "user", "content": text}], add_generation_prompt=True,+ return_tensors="pt", enable_thinking=False)["input_ids"][0].tolist()+++def batches_by_budget(lengths, budget=TOKEN_BUDGET, max_b=64):+ """Indices sorted by length, grouped so that batch * max_len <= budget."""+ order = sorted(range(len(lengths)), key=lambda i: -lengths[i])+ out, cur = [], []+ for i in order:+ if cur and (len(cur) + 1) * lengths[cur[0]] > budget or len(cur) == max_b:+ out.append(cur)+ cur = []+ cur.append(i)+ return out + [cur] if cur else out++[email protected]_grad()+def target_logprobs(model, seqs, queries, pad):+ """For each sequence, log-probs (full vocabulary) of chosen tokens at chosen positions.++ ``queries[i]`` is a list of (position, [token ids]); returns, per sequence, a list of+ (logprobs of those ids, argmax id) per query. Right padding, so real positions are exact.+ """+ out = [None] * len(seqs)+ for b in batches_by_budget([len(s) for s in seqs]):+ n = max(len(seqs[i]) for i in b)+ ids = torch.full((len(b), n), pad, dtype=torch.long)+ mask = torch.zeros((len(b), n), dtype=torch.long)+ for r, i in enumerate(b):+ ids[r, :len(seqs[i])] = torch.tensor(seqs[i])+ mask[r, :len(seqs[i])] = 1+ h = model.model(input_ids=ids.cuda(), attention_mask=mask.cuda(),+ use_cache=False).last_hidden_state+ rows = [(r, p, toks) for r, i in enumerate(b) for p, toks in queries[i]]+ sel = h[torch.tensor([r for r, _, _ in rows]), torch.tensor([p for _, p, _ in rows])]+ vals, tops = [], []+ for c in range(0, len(rows), 256): # bounded memory for the vocabulary-wide softmax+ lp = torch.log_softmax(model.lm_head(sel[c:c + 256]).float(), -1)+ tops += lp.argmax(-1).tolist()+ rr = [k - c for k in range(c, min(c + 256, len(rows))) for _ in rows[k][2]]+ tt = [t for k in range(c, min(c + 256, len(rows))) for t in rows[k][2]]+ vals += lp[rr, tt].tolist()+ k = v = 0+ for i in b:+ res = []+ for _, toks in queries[i]:+ res.append((vals[v:v + len(toks)], tops[k]))+ k, v = k + 1, v + len(toks)+ out[i] = res+ return out++[email protected]_grad()+def generate(model, prompts, max_new, eos, pad, batch=GEN_BATCH):+ """Greedy, batched with left padding; batches of similar prompt length, same for every+ model. Returns the new tokens of each prompt, cut after the first EOS."""+ order = sorted(range(len(prompts)), key=lambda i: -len(prompts[i]))+ out = [None] * len(prompts)+ for s in range(0, len(order), batch):+ b = order[s:s + batch]+ n = max(len(prompts[i]) for i in b)+ ids = torch.full((len(b), n), pad, dtype=torch.long)+ mask = torch.zeros((len(b), n), dtype=torch.long)+ for r, i in enumerate(b):+ ids[r, n - len(prompts[i]):] = torch.tensor(prompts[i])+ mask[r, n - len(prompts[i]):] = 1+ model.model.rope_deltas = None+ g = model.generate(input_ids=ids.cuda(), attention_mask=mask.cuda(), max_new_tokens=max_new,+ do_sample=False, temperature=None, top_p=None, top_k=None,+ eos_token_id=eos, pad_token_id=pad)[:, n:].tolist()+ for r, i in enumerate(b):+ toks = g[r]+ cut = next((j for j, t in enumerate(toks) if t in eos), None)+ out[i] = toks if cut is None else toks[:cut + 1]+ print(f" gen {s + len(b)}/{len(prompts)}", flush=True)+ return out+++def gen_record(tok, toks, eos):+ text = tok.decode([t for t in toks if t < TEXT_VOCAB], skip_special_tokens=True)+ return {"text": text, "tokens": toks, "n": len(toks), "finished": bool(toks) and toks[-1] in eos,+ "chip_tokens": sum(t >= TEXT_VOCAB for t in toks)}+++# --- tasks ------------------------------------------------------------------------------------+++def task_mmlu(model, tok, ctx, n=1000, seed=0):+ from datasets import load_dataset++ ds = load_dataset("cais/mmlu", "all", split="test")+ idx = sorted(random.Random(seed).sample(range(len(ds)), n))+ letters = [tok.encode(c, add_special_tokens=False) for c in "ABCD"]+ assert all(len(x) == 1 for x in letters)+ letters = [x[0] for x in letters]+ seqs, items = [], []+ for i in idx:+ d = ds[i]+ q = (f"The following is a multiple choice question about {d['subject'].replace('_', ' ')}."+ f"\n\n{d['question'].strip()}\n"+ + "".join(f"{c}. {t}\n" for c, t in zip("ABCD", d["choices"]))+ + "\nAnswer with the letter of the correct option only.")+ seqs.append(chat(tok, q))+ items.append({"idx": i, "subject": d["subject"], "gold": d["answer"]})+ res = target_logprobs(model, seqs, [[(len(s) - 1, letters)] for s in seqs], ctx["pad"])+ for it, ((lp, top),) in zip(items, res):+ it.update(lp=lp, pred=int(np.argmax(lp)), top_is_letter=top in letters)+ it["correct"] = it["pred"] == it["gold"]+ return items+++def choice_items(model, tok, ctx, docs):+ """docs: (context string, [continuation strings], gold). Continuation log-likelihoods."""+ seqs, queries, meta = [], [], []+ for c, conts, gold in docs:+ cids = tok.encode(c, add_special_tokens=False)+ meta.append((gold, [len(x) for x in conts]))+ for x in conts:+ xids = tok.encode(x, add_special_tokens=False)+ seqs.append(cids + xids)+ queries.append([(len(cids) - 1 + j, [t]) for j, t in enumerate(xids)])+ res = target_logprobs(model, seqs, queries, ctx["pad"])+ items, k = [], 0+ for gold, chars in meta:+ lp, greedy = [], []+ for _ in chars:+ lp.append(sum(r[0][0] for r in res[k]))+ greedy.append(all(r[1] == q[1][0] for r, q in zip(res[k], queries[k])))+ k += 1+ norm = [v / n for v, n in zip(lp, chars)]+ items.append({"gold": gold, "lp": lp, "pred": int(np.argmax(lp)),+ "pred_norm": int(np.argmax(norm)), "greedy": greedy})+ items[-1]["correct"] = items[-1]["pred"] == gold+ items[-1]["correct_norm"] = items[-1]["pred_norm"] == gold+ return items+++def task_arc(model, tok, ctx):+ from datasets import load_dataset++ ds = load_dataset("allenai/ai2_arc", "ARC-Challenge", split="test")+ docs = [(f"Question: {d['question']}\nAnswer:", [" " + t for t in d["choices"]["text"]],+ d["choices"]["label"].index(d["answerKey"])) for d in ds]+ items = choice_items(model, tok, ctx, docs)+ for d, it in zip(ds, items):+ it["id"] = d["id"]+ return items+++def _hs(text):+ text = text.strip().replace(" [title]", ". ")+ return re.sub(r"\[.*?\]", "", text).replace(" ", " ")+++def task_hellaswag(model, tok, ctx, n=1000, seed=0):+ from datasets import load_dataset++ ds = load_dataset("Rowan/hellaswag", split="validation")+ idx = sorted(random.Random(seed).sample(range(len(ds)), n))+ docs = []+ for i in idx:+ d = ds[i]+ c = _hs(d["activity_label"] + ": " + d["ctx_a"] + " " + d["ctx_b"].capitalize())+ docs.append((c, [" " + _hs(e) for e in d["endings"]], int(d["label"])))+ items = choice_items(model, tok, ctx, docs)+ for i, it in zip(idx, items):+ it["idx"] = i+ return items+++def gsm_answer(text):+ boxed = re.findall(r"\\boxed\{((?:[^{}]|\{[^{}]*\})*)\}", text)+ s = boxed[-1] if boxed else text+ nums = re.findall(r"-?\d[\d,]*(?:\.\d+)?", s.replace("$", ""))+ return nums[-1].replace(",", "").rstrip(".") if nums else None+++def same_number(a, b):+ try:+ return a is not None and abs(float(a) - float(b)) < 1e-6+ except ValueError:+ return False+++def task_gsm8k(model, tok, ctx, n=200, seed=0, max_new=1024):+ from datasets import load_dataset++ ds = load_dataset("openai/gsm8k", "main", split="test")+ idx = sorted(random.Random(seed).sample(range(len(ds)), n))+ prompts = [chat(tok, ds[i]["question"] + "\nPlease reason step by step, and put your final "+ "answer within \\boxed{}.") for i in idx]+ gens = generate(model, prompts, max_new, ctx["eos"], ctx["pad"])+ items = []+ for i, g in zip(idx, gens):+ gold = ds[i]["answer"].split("####")[-1].strip().replace(",", "")+ r = gen_record(tok, g, ctx["eos"])+ r.update(idx=i, gold=gold, pred=gsm_answer(r["text"]))+ r["correct"] = same_number(r["pred"], gold)+ items.append(r)+ return items+++def ifeval_utils():+ """lm-eval's IFEval checker, imported without lm_eval/__init__ (which pulls in the whole+ harness). Needs langdetect, immutabledict, nltk (punkt_tab)."""+ import importlib.util+ import types++ root = Path(importlib.util.find_spec("lm_eval").submodule_search_locations[0])+ for name, p in (("lm_eval", root), ("lm_eval.tasks", root / "tasks")):+ if name not in sys.modules:+ m = types.ModuleType(name)+ m.__path__ = [str(p)]+ sys.modules[name] = m+ from lm_eval.tasks.ifeval import utils++ return utils+++def task_ifeval(model, tok, ctx, max_new=1280):+ from datasets import load_dataset++ utils = ifeval_utils()+ ds = load_dataset("google/IFEval", split="train")+ gens = generate(model, [chat(tok, d["prompt"]) for d in ds], max_new, ctx["eos"], ctx["pad"])+ items = []+ for d, g in zip(ds, gens):+ r = gen_record(tok, g, ctx["eos"])+ s = utils.process_results(d, [r["text"]])+ r.update(key=d["key"], strict=s["prompt_level_strict_acc"],+ loose=s["prompt_level_loose_acc"], inst_strict=s["inst_level_strict_acc"],+ inst_loose=s["inst_level_loose_acc"])+ r["correct"] = r["strict"]+ items.append(r)+ return items+++CITIES = ["Lisbon", "Nairobi", "Osaka", "Quito", "Tbilisi", "Winnipeg", "Hanoi", "Bergen",+ "Cusco", "Dakar", "Perth", "Tallinn"]+++def haystack_ids(tok):+ from datasets import load_dataset++ ds = load_dataset("Salesforce/wikitext", "wikitext-103-raw-v1", split="test")+ text = "".join(t for t in ds["text"] if t.strip() and not t.strip().startswith("="))+ return tok.encode(text, add_special_tokens=False)++[email protected]_grad()+def prefill(model, ids, keep=1):+ """Chunked prefill with a fresh cache; logits (text vocab) of the last ``keep`` positions."""+ from transformers import DynamicCache++ model.model.rope_deltas = None # stale after a batched generate(); text-only deltas are 0+ cache = DynamicCache(config=model.config)+ n = len(ids) - keep+ cuts = list(range(0, n, CHUNK)) + [n, len(ids)]+ for a, b in itertools.pairwise(cuts):+ x = torch.tensor([ids[a:b]], device="cuda")+ out = model(input_ids=x, past_key_values=cache, use_cache=True, logits_to_keep=keep)+ return out.logits[0, :, :TEXT_VOCAB].float(), cache++[email protected]_grad()+def greedy_from(model, logits, cache, n, eos):+ toks = []+ t = int(logits[-1].argmax())+ for _ in range(n):+ toks.append(t)+ if t in eos:+ break+ out = model(input_ids=torch.tensor([[t]], device="cuda"), past_key_values=cache,+ use_cache=True)+ t = int(out.logits[0, -1].argmax()) # full vocabulary: a chip token would show up+ return toks+++def task_niah(model, tok, ctx, lengths=(8192, 16384, 32768), depths=(0.1, 0.3, 0.5, 0.7, 0.9),+ seeds=4):+ hay = haystack_ids(tok)+ items = []+ for L in lengths:+ for depth in depths:+ for s in range(seeds):+ rng = random.Random(L * 100 + int(depth * 10) * 10 + s)+ city, num = rng.choice(CITIES), str(rng.randrange(10**6, 10**7))+ question = (f"What is the special magic number for {city} mentioned in the text? "+ "Answer with the number only.")+ head = "Read the text below and answer the question that follows.\n\n"+ over = len(chat(tok, head + "\n\n" + question)) + 20+ off = rng.randrange(0, len(hay) - L)+ body = hay[off:off + L - over]+ p = int(depth * len(body))+ left, right = tok.decode(body[:p]), tok.decode(body[p:])+ cut = left.rfind(". ")+ if cut > len(left) - 2000: # put the needle at a sentence boundary+ left, right = left[:cut + 2], left[cut + 2:] + right+ needle = f"The special magic number for {city} is {num}. "+ ids = chat(tok, head + left + needle + right + "\n\n" + question)+ ans = tok.encode(num, add_special_tokens=False)+ logits, _ = prefill(model, ids + ans, keep=len(ans) + 1)+ lp = torch.log_softmax(logits, -1)+ gold_lp = float(sum(lp[j, t] for j, t in enumerate(ans)))+ logits, cache = prefill(model, ids)+ g = greedy_from(model, logits, cache, 16, ctx["eos"])+ del cache+ r = gen_record(tok, g, ctx["eos"])+ r.update(length=len(ids), target=L, depth=depth, seed=s, city=city, gold=num,+ gold_logprob=gold_lp, correct=num in r["text"])+ items.append(r)+ print(f" niah L={len(ids)} d={depth} s={s} ok={r['correct']} "+ f"lp={gold_lp:.3f} {r['text']!r}", flush=True)+ return items++[email protected]_grad()+def task_ppl(model, tok, ctx, length=32768, docs=8):+ """Long-context language modelling: per-token NLL and greedy next token over ``docs``+ disjoint ``length``-token windows of wikitext-103 test, chunked prefill."""+ from transformers import DynamicCache++ hay = haystack_ids(tok)+ items = []+ for d in range(docs):+ ids = hay[d * length:(d + 1) * length]+ model.model.rope_deltas = None+ cache = DynamicCache(config=model.config)+ nll, top = [], []+ for a in range(0, length, CHUNK):+ x = torch.tensor([ids[a:a + CHUNK]], device="cuda")+ h = model.model(input_ids=x, past_key_values=cache, use_cache=True).last_hidden_state[0]+ for c in range(0, h.shape[0], 256):+ lp = torch.log_softmax(model.lm_head(h[c:c + 256]).float(), -1)+ p0 = a + c+ tgt = torch.tensor(ids[p0 + 1:p0 + 1 + lp.shape[0]], device="cuda")+ nll += (-lp[torch.arange(len(tgt)), tgt]).tolist()+ top += lp.argmax(-1)[:len(tgt)].tolist()+ del cache+ items.append({"offset": d * length, "nll": [round(v, 5) for v in nll], "top1": top,+ "target": ids[1:], "correct": float(np.mean(np.array(top) == ids[1:]))})+ print(f" ppl doc {d}: ppl {math.exp(np.mean(nll)):.3f} top1 {items[-1]['correct']:.4f}",+ flush=True)+ return items+++PPL_BUCKETS = ((0, 2048), (2048, 8192), (8192, 16384), (16384, 32768))+++def ppl_table(data, names, ref, summary):+ """Rows for the long-context perplexity task: PPL per position bucket, signed mean dNLL vs+ ``ref`` with a bootstrap CI over 1024-token blocks, |dNLL|, greedy next-token agreement."""+ lines = []+ for lo, hi in PPL_BUCKETS:+ cells = []+ for n in names:+ if "ppl" not in data[n]:+ cells.append("-")+ continue+ v = np.concatenate([np.array(it["nll"][lo:hi]) for it in data[n]["ppl"]])+ s = {"ppl": float(np.exp(v.mean())), "tokens": len(v)}+ cell = f"{s['ppl']:.3f}"+ if n != ref:+ r = [np.array(it["nll"][lo:hi]) for it in data[ref]["ppl"]]+ m = [np.array(it["nll"][lo:hi]) for it in data[n]["ppl"]]+ blocks = np.array([(y[k:k + 1024] - x[k:k + 1024]).mean() for x, y in zip(r, m)+ for k in range(0, len(x), 1024)])+ boot = np.random.default_rng(0).choice(blocks, (10000, len(blocks))).mean(1)+ dl, dh = np.percentile(boot, [2.5, 97.5])+ d = np.concatenate([y - x for x, y in zip(r, m)])+ agree = np.concatenate([np.array(a["top1"][lo:hi]) == np.array(b["top1"][lo:hi])+ for a, b in zip(data[ref]["ppl"], data[n]["ppl"])])+ s.update(dnll=float(d.mean()), dnll_ci95=[float(dl), float(dh)],+ abs_dnll=float(np.abs(d).mean()), top1_agreement=float(agree.mean()))+ cell += (f"<br>dNLL {d.mean():+.2e} [{dl:+.1e}, {dh:+.1e}], mean abs dNLL "+ f"{np.abs(d).mean():.2e}, top-1 agree {100 * agree.mean():.2f}%")+ summary.setdefault(n, {}).setdefault("ppl", {})[f"{lo}-{hi}"] = s+ cells.append(cell)+ lines.append(f"| ppl | positions {lo}-{hi} | " + " | ".join(cells) + " |")+ return lines+++def stop_tokens(tok):+ """End of turn and end of text stop generation (as in the merged generation_config.json);+ the base snapshot ships no generation config."""+ eos = [tok.convert_tokens_to_ids("<|im_end|>"), tok.convert_tokens_to_ids("<|endoftext|>")]+ return {"eos": eos, "pad": eos[1]}+++# --- run --------------------------------------------------------------------------------------+++def run(args):+ from transformers import AutoTokenizer++ tok = AutoTokenizer.from_pretrained(args.tokenizer or args.model)+ ctx = stop_tokens(tok)+ out = Path(args.out) / args.name+ out.mkdir(parents=True, exist_ok=True)+ model = None+ for task in args.tasks.split(","):+ f = out / f"{task}.json"+ if f.exists():+ print(f"{args.name}/{task}: done already", flush=True)+ continue+ if model is None:+ model = load(args.model, args.attn, args.mem_gb)+ print(f"{args.name}/{task} ...", flush=True)+ t = time.time()+ torch.cuda.reset_peak_memory_stats()+ items = globals()[f"task_{task}"](model, tok, ctx)+ rec = {"model": args.model, "attn": args.attn, "task": task, "seconds": time.time() - t,+ "peak_gb": torch.cuda.max_memory_allocated() / 2**30, "items": items}+ f.write_text(json.dumps(rec, ensure_ascii=False))+ acc = np.mean([it["correct"] for it in items])+ print(f"{args.name}/{task}: acc {acc:.4f} n={len(items)} {rec['seconds']:.0f}s "+ f"peak {rec['peak_gb']:.1f} GB", flush=True)+++# --- statistics -------------------------------------------------------------------------------+++def wilson(k, n, z=1.96):+ if n == 0:+ return (float("nan"),) * 2+ p = k / n+ c = (p + z * z / (2 * n)) / (1 + z * z / n)+ h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)+ return c - h, c + h+++def paired_diff(a, b, n_boot=10000, seed=0):+ """Mean of a - b with a paired bootstrap 95% CI, and the exact McNemar p-value."""+ a, b = np.asarray(a, float), np.asarray(b, float)+ rng = np.random.default_rng(seed)+ idx = rng.integers(0, len(a), (n_boot, len(a)))+ d = (a[idx] - b[idx]).mean(1)+ n01, n10 = int(((a == 1) & (b == 0)).sum()), int(((a == 0) & (b == 1)).sum())+ m = n01 + n10+ p = min(1.0, 2 * sum(math.comb(m, k) for k in range(min(n01, n10) + 1)) / 2**m) if m else 1.0+ lo, hi = np.percentile(d, [2.5, 97.5])+ return float((a - b).mean()), float(lo), float(hi), p+++def choice_kl(p_lp, q_lp):+ """KL(p || q) between the normalised choice distributions of two runs, per item."""+ p = torch.log_softmax(torch.tensor(p_lp, dtype=torch.float64), -1)+ q = torch.log_softmax(torch.tensor(q_lp, dtype=torch.float64), -1)+ return float((p.exp() * (p - q)).sum())+++def first_divergence(x, y):+ return next((i for i, (p, q) in enumerate(zip(x, y)) if p != q),+ None if len(x) == len(y) else min(len(x), len(y)))+++METRICS = { # task -> (metric name, item key) list; the first is the headline+ "mmlu": [("acc", "correct")],+ "arc": [("acc_norm", "correct_norm"), ("acc", "correct")],+ "hellaswag": [("acc_norm", "correct_norm"), ("acc", "correct")],+ "gsm8k": [("acc", "correct")],+ "ifeval": [("prompt_strict", "strict"), ("prompt_loose", "loose")],+ "niah": [("acc", "correct")],+}+++def compare_items(task, ref, run):+ """Fine-grained agreement of ``run`` with ``ref`` beyond accuracy."""+ out = {}+ if task in ("mmlu", "arc", "hellaswag"):+ out["same_pred"] = [a["pred"] == b["pred"] for a, b in zip(ref, run)]+ out["mean_choice_kl"] = float(np.mean([choice_kl(a["lp"], b["lp"])+ for a, b in zip(ref, run)]))+ out["mean_abs_dlogprob"] = float(np.mean([abs(x - y) for a, b in zip(ref, run)+ for x, y in zip(a["lp"], b["lp"])]))+ elif task == "niah":+ out["same_pred"] = [a["correct"] == b["correct"] for a, b in zip(ref, run)]+ out["mean_abs_dlogprob"] = float(np.mean([abs(a["gold_logprob"] - b["gold_logprob"])+ for a, b in zip(ref, run)]))+ out["identical_text"] = float(np.mean([a["tokens"] == b["tokens"] for a, b in zip(ref, run)]))+ else:+ key = "pred" if task == "gsm8k" else "strict"+ out["same_pred"] = [a[key] == b[key] for a, b in zip(ref, run)]+ out["identical_text"] = float(np.mean([a["tokens"] == b["tokens"] for a, b in zip(ref, run)]))+ fd = [first_divergence(a["tokens"], b["tokens"]) for a, b in zip(ref, run)]+ fd = [x for x in fd if x is not None]+ out["median_first_divergence"] = float(np.median(fd)) if fd else None+ out["unfinished"] = int(sum(not b["finished"] for b in run))+ out["chip_tokens"] = int(sum(b["chip_tokens"] for b in run))+ return out+++def table(args):+ root = Path(args.dir)+ names = [args.ref] + [p.name for p in sorted(root.iterdir())+ if p.is_dir() and p.name != args.ref]+ data = {n: {f.stem: json.loads(f.read_text(encoding="utf-8"))["items"]+ for f in (root / n).glob("*.json")} for n in names}+ summary, lines = {}, []+ lines.append("| task | metric | " + " | ".join(names) + " |")+ lines.append("|---|---|" + "---|" * len(names))+ for task in TASKS:+ if task not in data[args.ref] or task == "ppl":+ continue+ ref = data[args.ref][task]+ for metric, key in METRICS[task]:+ cells = []+ for n in names:+ if task not in data[n]:+ cells.append("-")+ continue+ v = [bool(it[key]) for it in data[n][task]]+ lo, hi = wilson(sum(v), len(v))+ s = {"acc": float(np.mean(v)), "ci95": [lo, hi], "n": len(v)}+ cell = f"{100 * s['acc']:.1f} [{100 * lo:.1f}, {100 * hi:.1f}]"+ if n != args.ref:+ d, dlo, dhi, p = paired_diff(v, [bool(it[key]) for it in ref])+ s.update(diff=d, diff_ci95=[dlo, dhi], mcnemar_p=p)+ cell += f"<br>diff {100 * d:+.1f} [{100 * dlo:+.1f}, {100 * dhi:+.1f}] p={p:.2f}"+ summary.setdefault(n, {}).setdefault(task, {})[metric] = s+ cells.append(cell)+ lines.append(f"| {task} | {metric} (n={len(ref)}) | " + " | ".join(cells) + " |")+ cells = []+ for n in names:+ if n == args.ref or task not in data[n]:+ cells.append("ref" if n == args.ref else "-")+ continue+ c = compare_items(task, ref, data[n][task])+ same = c.pop("same_pred")+ lo, hi = wilson(sum(same), len(same))+ c["agreement"] = {"frac": float(np.mean(same)), "ci95": [lo, hi]}+ summary[n][task]["vs_ref"] = c+ extra = ", ".join(f"{k} {v:.3g}" if isinstance(v, float) else f"{k} {v}"+ for k, v in c.items() if k != "agreement")+ cells.append(f"{100 * np.mean(same):.1f}% [{100 * lo:.1f}, {100 * hi:.1f}]<br>{extra}")+ lines.append(f"| {task} | agreement with {args.ref} | " + " | ".join(cells) + " |")+ if "ppl" in data[args.ref]:+ lines += ppl_table(data, names, args.ref, summary)+ md = "\n".join(lines)+ print(md)+ (root / "summary.json").write_text(json.dumps(summary, indent=2))+ (root / "table.md").write_text(md + "\n", encoding="utf-8")+++def main() -> None:+ ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])+ sub = ap.add_subparsers(dest="cmd", required=True)+ r = sub.add_parser("run")+ r.add_argument("model")+ r.add_argument("--name", required=True)+ r.add_argument("--attn", default="sdpa")+ r.add_argument("--tokenizer", default=None, help="shared tokenizer (the base checkpoint)")+ r.add_argument("--tasks", default=",".join(TASKS))+ r.add_argument("--out", default="dist/evals/text")+ r.add_argument("--mem-gb", type=float, default=20.0, help="per-process GPU memory cap")+ t = sub.add_parser("table")+ t.add_argument("dir", nargs="?", default="dist/evals/text")+ t.add_argument("--ref", default="base")+ args = ap.parse_args()+ run(args) if args.cmd == "run" else table(args)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/q35/tiny.py b/experimental/riscv/chipformer/q35/tiny.pynew file mode 100644index 0000000..91eba01--- /dev/null+++ b/experimental/riscv/chipformer/q35/tiny.py@@ -0,0 +1,132 @@+"""A standalone Qwen3.5 model that holds only the chip circuit.++Same architecture class and DeltaNet head shapes as Qwen3.5-0.8B, so the circuit that runs here+merges into the real model unchanged. HF's cache needs one full-attention layer to count+positions; it is appended last with zero weights, so it adds nothing.+"""+++import torch++from chipformer.q35.emit import Layout, allocate, check_bf16, embedding_rows, emit_layers+from chipformer.q35.ir import Graph++BASE = { # Qwen3.5-0.8B text config, minus the sizes we pick+ "hidden_act": "silu",+ "intermediate_size": 3584,+ "linear_conv_kernel_dim": 4,+ "linear_key_head_dim": 128,+ "linear_value_head_dim": 128,+ "linear_num_key_heads": 16,+ "linear_num_value_heads": 16,+ "num_attention_heads": 8,+ "num_key_value_heads": 2,+ "head_dim": 256,+ "rms_norm_eps": 1e-6,+ "attn_output_gate": True,+ "rope_parameters": {"rope_type": "default", "rope_theta": 10000000,+ "partial_rotary_factor": 0.25, "mrope_section": [11, 11, 10],+ "mrope_interleaved": True},+ "max_position_embeddings": 262144,+ "tie_word_embeddings": True,+}+++def pick_layout(n_dims: int) -> Layout:+ """Smallest hidden = m * 4^k (a multiple of 64) that fits m ballast dims and n_dims."""+ best = None+ for m in range(3, 16): # >= 3 ballast dims: constants are split over them+ k = 0+ while m * 4**k < m + n_dims or (m * 4**k) % 64:+ k += 1+ h = m * 4**k+ if best is None or h < best.hidden:+ best = Layout(hidden=h, m=m)+ return best+++CODE_GAIN = 2.0**20 # final-norm gain on the output code (the exit leaves KAPPA * +-1 there)+++def code_gain(lay: Layout | None = None) -> float:+ return CODE_GAIN+++def build(g: Graph, vocab: list[int], code_dims: list[int], dtype=torch.bfloat16,+ clearable: set[int] | None = None, share: bool = False):+ """Qwen3_5ForCausalLM whose token i is chip token vocab[i]; lm_head reads ``code_dims``.++ With ``clearable`` (dims whose slots may be handed over), dims share residual slots.+ """+ from transformers import Qwen3_5ForCausalLM, Qwen3_5TextConfig++ alloc = None+ keep = set(getattr(g, "keep", code_dims))+ if share:+ alloc = allocate(g, clearable, keep, BASE["intermediate_size"])+ lay = pick_layout(alloc.n_slots + 1)+ lay.slots = alloc.slots+ else:+ lay = pick_layout(len(g.dims))+ n_chip = g.n_layers()+ n_vocab = -(-len(vocab) // 64) * 64+ tc = dict(BASE, hidden_size=lay.hidden, vocab_size=n_vocab, num_hidden_layers=n_chip + 1,+ layer_types=["linear_attention"] * n_chip + ["full_attention"])+ sd = emit_layers(g, lay, tc, 0, "model.", alloc, keep if hasattr(g, "keep") else None)+ emb = torch.zeros(n_vocab, lay.hidden)+ emb[: len(vocab)] = embedding_rows(g, lay, vocab, lay.hidden)+ sd["model.embed_tokens.weight"] = emb+ norm = torch.full((lay.hidden,), -1.0) # (1 + w) = 0: lm_head sees only the code dims+ for d in code_dims:+ norm[lay.dim(d)] = code_gain(lay) # gain 1 + w, close enough to a power of 2+ check_bf16("final norm", norm)+ sd["model.norm.weight"] = norm+ config = Qwen3_5TextConfig(**tc)+ model = Qwen3_5ForCausalLM(config)+ for p in model.parameters():+ p.data.zero_()+ missing, unexpected = model.load_state_dict(sd, strict=False)+ assert not unexpected, unexpected[:5]+ assert all(k.startswith(f"model.layers.{n_chip}.") or k == "lm_head.weight" for k in missing), \+ [k for k in missing if not k.startswith(f"model.layers.{n_chip}.")][:5]+ model.tie_weights()+ model.layout = lay+ return model.to(dtype).eval()+++def sample_traces() -> list[list[int]]:+ """Short traces covering every opcode and quirk, for deciding which dims are clearable."""+ from chipformer.check import run_machine+ from chipformer.fuzz import random_program, random_quirks++ out = []+ for seed in range(6):+ m = run_machine(random_program(seed), 300, ipf=7, seed=seed, key_rate=0.3,+ quirks=random_quirks(seed))+ out.append(m.tokens)+ return out+++def build_chip8(dtype=torch.bfloat16, share_slots: bool = True):+ from chipformer.q35 import bridge, chip8_q35++ c = chip8_q35.build()+ vocab = list(range(bridge.TEXT_BASE + bridge.N_TEXT))+ model = build(c, vocab, [next(iter(e.terms)) for e in c.code], dtype=dtype,+ share=share_slots)+ return c, model+++def main() -> None:+ import argparse++ ap = argparse.ArgumentParser(description="Save CHIP-8 as a standalone Qwen3.5 checkpoint")+ ap.add_argument("--out", default="dist/q35-tiny")+ args = ap.parse_args()+ c, model = build_chip8()+ model.save_pretrained(args.out)+ print(f"{args.out}: {c.stats()}, hidden {model.layout.hidden}")+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/ref/__init__.py b/experimental/riscv/chipformer/ref/__init__.pynew file mode 100644index 0000000..ffe231d--- /dev/null+++ b/experimental/riscv/chipformer/ref/__init__.py@@ -0,0 +1 @@+"""Reference (non-neural) CHIP-8 implementation used as ground truth."""diff --git a/experimental/riscv/chipformer/ref/emu.py b/experimental/riscv/chipformer/ref/emu.pynew file mode 100644index 0000000..28018f8--- /dev/null+++ b/experimental/riscv/chipformer/ref/emu.py@@ -0,0 +1,285 @@+"""Reference CHIP-8 interpreter operating directly on the flat cell array.++Every state change is expressed as a list of ``(addr, value)`` writes computed+from the state *before* the instruction, then applied. The trace format is+therefore exactly the emulator's semantics: replaying the writes reproduces+the state.++Quirks are machine state: six flag cells (spec.QUIRKS) with the meaning of+Octo's flags of the same name, set when the ROM is loaded (docs/quirks.md):+ - shift: 8XY6/8XYE shift VX in place (off: VX = VY shifted)+ - loadstore: FX55/FX65 leave I unchanged (off: I += X + 1)+ - clip: sprites clip at the screen edges (off: they wrap around)+ - logic: 8XY1/2/3 reset VF to 0+ - jump: BXNN jumps to XNN + VX (off: BNNN jumps to NNN + V0)+ - vforder: for 8XY4-E the result is written after VF (wins when X = F)+Fixed behaviour: sprites start at (VX % 64, VY % 32); FX0A completes as soon+as any key is held (lowest key wins); the stack pointer wraps modulo 16.+"""++from chipformer import spec+from chipformer.spec import (+ CLS,+ DISPLAY,+ DT,+ KEYS,+ NULL,+ PC,+ Q_CLIP,+ Q_JUMP,+ Q_LOADSTORE,+ Q_LOGIC,+ Q_SHIFT,+ Q_VFORDER,+ RNG,+ SP,+ ST,+ STACK,+ I,+ V,+)++Writes = list[tuple[int, int]]+++def fetch(cells: list[int]) -> int:+ pc = cells[PC]+ return (cells[pc] << 8) | cells[(pc + 1) & 0xFFF]+++def opcode_class(op: int) -> str:+ """Human-readable opcode family, e.g. '8XY4' or 'DXYN'."""+ hi = op >> 12+ if op == 0x00E0:+ return "00E0"+ if op == 0x00EE:+ return "00EE"+ if hi == 0x0:+ return "0NNN"+ if hi == 0x8:+ return f"8XY{op & 0xF:X}"+ if hi in (0xE, 0xF):+ return f"{hi:X}X{op & 0xFF:02X}"+ return {1: "1NNN", 2: "2NNN", 3: "3XNN", 4: "4XNN", 5: "5XY0", 6: "6XNN", 7: "7XNN",+ 9: "9XY0", 0xA: "ANNN", 0xB: "BNNN", 0xC: "CXNN", 0xD: "DXYN"}[hi]+++class Chip8:+ def __init__(self, rom: bytes, quirks: "str | tuple[str, ...] | None" = None):+ if len(rom) > spec.RAM_SIZE - spec.PROGRAM_START:+ raise ValueError(f"ROM too large: {len(rom)} bytes")+ self.cells = [0] * spec.NUM_CELLS+ self.cells[spec.FONT_BASE:spec.FONT_BASE + len(spec.FONT)] = spec.FONT+ self.cells[spec.PROGRAM_START:spec.PROGRAM_START + len(rom)] = rom+ self.cells[PC] = spec.PROGRAM_START+ self.cells[RNG] = spec.RNG_SEED+ for a, v in spec.quirk_cells(quirks).items():+ self.cells[a] = v++ # --- state transitions --------------------------------------------------++ def apply(self, writes: Writes) -> None:+ apply_writes(self.cells, writes)++ def step(self) -> Writes:+ w = instruction_writes(self.cells)+ self.apply(w)+ return w++ def tick(self) -> Writes:+ w = tick_writes(self.cells)+ self.apply(w)+ return w++ def set_key(self, key: int, down: bool) -> Writes:+ w = [(KEYS + key, int(down))]+ self.apply(w)+ return w++ # --- views --------------------------------------------------------------++ def state_writes(self) -> Writes:+ return state_writes(self.cells)++ def pixels(self) -> list[list[int]]:+ return display_pixels(self.cells)+++def apply_writes(cells: list[int], writes: Writes) -> None:+ for a, v in writes:+ assert 0 <= v <= spec.MAX_VALUE, (a, v)+ if a == CLS:+ cells[DISPLAY:DISPLAY + spec.DISPLAY_H * spec.DISPLAY_ROW_BYTES] = (+ [0] * (spec.DISPLAY_H * spec.DISPLAY_ROW_BYTES))+ cells[a] = v+++def state_writes(cells: list[int]) -> Writes:+ """A state dump: replaying it on a zeroed state reproduces ``cells``.++ Only nonzero cells are written (reads of never-written cells return 0).+ CLS always comes first, so a "latest CLS" exists and the display bytes+ written after it survive.+ """+ return [(CLS, cells[CLS])] + [+ (a, cells[a]) for a in range(spec.NUM_CELLS) if a not in (CLS, NULL) and cells[a]]+++def display_pixels(cells: list[int]) -> list[list[int]]:+ rows = []+ for y in range(spec.DISPLAY_H):+ row = []+ for b in range(spec.DISPLAY_ROW_BYTES):+ byte = cells[DISPLAY + y * spec.DISPLAY_ROW_BYTES + b]+ row.extend((byte >> (7 - i)) & 1 for i in range(8))+ rows.append(row)+ return rows+++def tick_writes(cells: list[int]) -> Writes:+ return [(DT, max(cells[DT] - 1, 0)), (ST, max(cells[ST] - 1, 0))]+++def instruction_writes(c: list[int]) -> Writes:+ """Writes for one instruction, computed purely from the current state.++ The PC write is always last, so every earlier write sees the old PC.+ """+ pc = c[PC]+ op = fetch(c)+ nxt = (pc + 2) & 0xFFF+ skip = (pc + 4) & 0xFFF+ x, y, n = (op >> 8) & 0xF, (op >> 4) & 0xF, op & 0xF+ nn, nnn = op & 0xFF, op & 0xFFF+ vx, vy = c[V + x], c[V + y]+ hi = op >> 12+ w: Writes = []++ if op == 0x00E0:+ w = [(CLS, 0)]+ elif op == 0x00EE:+ sp = (c[SP] - 1) & 0xF+ w = [(SP, sp)]+ nxt = c[STACK + sp]+ elif hi == 0x0:+ pass # 0NNN: machine-code routine, ignored+ elif hi == 0x1:+ nxt = nnn+ elif hi == 0x2:+ sp = c[SP]+ w = [(STACK + sp, nxt), (SP, (sp + 1) & 0xF)]+ nxt = nnn+ elif hi == 0x3:+ nxt = skip if vx == nn else nxt+ elif hi == 0x4:+ nxt = skip if vx != nn else nxt+ elif hi == 0x5 and n == 0:+ nxt = skip if vx == vy else nxt+ elif hi == 0x9 and n == 0:+ nxt = skip if vx != vy else nxt+ elif hi == 0x6:+ w = [(V + x, nn)]+ elif hi == 0x7:+ w = [(V + x, (vx + nn) & 0xFF)]+ elif hi == 0x8:+ w = alu_writes(c, x, n, vx, vy)+ elif hi == 0xA:+ w = [(I, nnn)]+ elif hi == 0xB:+ nxt = (nnn + (vx if c[Q_JUMP] else c[V])) & 0xFFF+ elif hi == 0xC:+ s = spec.lfsr_step(c[RNG])+ w = [(V + x, s & nn & 0xFF), (RNG, s)]+ elif hi == 0xD:+ w = draw_writes(c, vx, vy, n)+ elif hi == 0xE and nn == 0x9E:+ nxt = skip if c[KEYS + (vx & 0xF)] else nxt+ elif hi == 0xE and nn == 0xA1:+ nxt = skip if not c[KEYS + (vx & 0xF)] else nxt+ elif hi == 0xF:+ ix = c[I]+ if nn == 0x07:+ w = [(V + x, c[DT])]+ elif nn == 0x0A:+ pressed = [k for k in range(16) if c[KEYS + k]]+ if pressed:+ w = [(V + x, pressed[0])]+ else:+ w = [(V + x, vx)]+ nxt = pc+ elif nn == 0x15:+ w = [(DT, vx)]+ elif nn == 0x18:+ w = [(ST, vx)]+ elif nn == 0x1E:+ w = [(I, (ix + vx) & 0xFFF)]+ elif nn == 0x29:+ w = [(I, spec.FONT_BASE + 5 * (vx & 0xF))]+ elif nn == 0x33:+ w = [((ix + j) & 0xFFF, d) for j, d in enumerate((vx // 100, vx // 10 % 10, vx % 10))]+ elif nn == 0x55:+ w = [((ix + j) & 0xFFF, c[V + j]) for j in range(x + 1)]+ elif nn == 0x65:+ w = [(V + j, c[(ix + j) & 0xFFF]) for j in range(x + 1)]+ if nn in (0x55, 0x65) and not c[Q_LOADSTORE]:+ w.append((I, (ix + x + 1) & 0xFFF))++ w.append((PC, nxt))+ return w+++def alu_writes(c: list[int], x: int, n: int, vx: int, vy: int) -> Writes:+ vf = V + 0xF+ if n in (0x1, 0x2, 0x3):+ result = {0x1: vx | vy, 0x2: vx & vy, 0x3: vx ^ vy}[n]+ return [(V + x, result)] + ([(vf, 0)] if c[Q_LOGIC] else [])+ src = vx if c[Q_SHIFT] else vy+ if n == 0x0:+ return [(V + x, vy)]+ if n == 0x4:+ s = vx + vy+ result, flag = s & 0xFF, s >> 8+ elif n == 0x5:+ result, flag = (vx - vy) & 0xFF, int(vx >= vy)+ elif n == 0x6:+ result, flag = src >> 1, src & 1+ elif n == 0x7:+ result, flag = (vy - vx) & 0xFF, int(vy >= vx)+ elif n == 0xE:+ result, flag = (src << 1) & 0xFF, src >> 7+ else:+ return []+ if c[Q_VFORDER]:+ return [(vf, flag), (V + x, result)]+ return [(V + x, result), (vf, flag)]+++def draw_writes(c: list[int], vx: int, vy: int, n: int) -> Writes:+ """DXYN as a fixed-length record: two display writes per sprite row+ (left and right byte the row can touch), then VF.++ With clipping, writes that fall off-screen go to NULL so the record length+ depends only on N, which keeps the transformer's micro-step bookkeeping+ trivial. Without it, rows and bytes wrap around the screen.+ """+ clip = c[Q_CLIP]+ x0, y0 = vx % spec.DISPLAY_W, vy % spec.DISPLAY_H+ b, o = x0 >> 3, x0 & 7+ w: Writes = []+ collision = 0+ for i in range(n):+ s = c[(c[I] + i) & 0xFFF]+ parts = ((b, s >> o), (b + 1, (s << (8 - o)) & 0xFF))+ row = y0 + i+ for col, bits in parts:+ if not clip:+ row, col = row % spec.DISPLAY_H, col % spec.DISPLAY_ROW_BYTES+ if row < spec.DISPLAY_H and col < spec.DISPLAY_ROW_BYTES:+ a = DISPLAY + row * spec.DISPLAY_ROW_BYTES + col+ old = c[a]+ w.append((a, old ^ bits))+ collision |= old & bits+ else:+ w.append((NULL, 0))+ w.append((V + 0xF, int(collision != 0)))+ return wdiff --git a/experimental/riscv/chipformer/ref/play.py b/experimental/riscv/chipformer/ref/play.pynew file mode 100644index 0000000..43fddfb--- /dev/null+++ b/experimental/riscv/chipformer/ref/play.py@@ -0,0 +1,72 @@+"""Play a ROM on the reference emulator.++ uv run --extra play python -m chipformer.ref.play roms/pong.ch8++Keypad layout (CHIP-8 -> keyboard):+ 1 2 3 C 1 2 3 4+ 4 5 6 D -> Q W E R+ 7 8 9 E A S D F+ A 0 B F Z X C V+"""++import argparse+import sys+from pathlib import Path++from chipformer import spec+from chipformer.trace import Machine, replay++KEYMAP = {+ "1": 0x1, "2": 0x2, "3": 0x3, "4": 0xC,+ "q": 0x4, "w": 0x5, "e": 0x6, "r": 0xD,+ "a": 0x7, "s": 0x8, "d": 0x9, "f": 0xE,+ "z": 0xA, "x": 0x0, "c": 0xB, "v": 0xF,+}+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("rom", type=Path)+ ap.add_argument("--ipf", type=int, default=10, help="instructions per 60 Hz frame")+ ap.add_argument("--scale", type=int, default=12)+ ap.add_argument("--check", action="store_true", help="replay the trace on exit and verify")+ args = ap.parse_args()++ import pygame++ m = Machine(args.rom.read_bytes())+ pygame.init()+ screen = pygame.display.set_mode((spec.DISPLAY_W * args.scale, spec.DISPLAY_H * args.scale))+ pygame.display.set_caption(f"chipformer reference - {args.rom.name}")+ clock = pygame.time.Clock()+ keycodes = {pygame.key.key_code(k): v for k, v in KEYMAP.items()}++ running = True+ while running:+ for ev in pygame.event.get():+ if ev.type == pygame.QUIT or (ev.type == pygame.KEYDOWN and ev.key == pygame.K_ESCAPE):+ running = False+ elif ev.type in (pygame.KEYDOWN, pygame.KEYUP) and ev.key in keycodes:+ m.set_key(keycodes[ev.key], ev.type == pygame.KEYDOWN)+ m.tick()+ for _ in range(args.ipf):+ m.step()+ screen.fill((16, 16, 24))+ for y, row in enumerate(m.emu.pixels()):+ for x, p in enumerate(row):+ if p:+ screen.fill((230, 230, 210),+ (x * args.scale, y * args.scale, args.scale, args.scale))+ pygame.display.flip()+ clock.tick(60)+ pygame.quit()++ print(f"trace: {len(m.tokens):,} tokens")+ if args.check:+ ok = replay(m.tokens) == m.emu.cells+ print("replay check:", "OK" if ok else "MISMATCH")+ sys.exit(0 if ok else 1)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/ref/profile.py b/experimental/riscv/chipformer/ref/profile.pynew file mode 100644index 0000000..cc46146--- /dev/null+++ b/experimental/riscv/chipformer/ref/profile.py@@ -0,0 +1,50 @@+"""Opcode histogram and trace statistics for a ROM under random input.++ uv run python -m chipformer.ref.profile roms/pong.ch8 --steps 200000+"""++import argparse+import random+from collections import Counter+from pathlib import Path++from chipformer.ref.emu import fetch, opcode_class+from chipformer.trace import Machine+++def profile(rom: bytes, steps: int, ipf: int = 10, seed: int = 0):+ rng = random.Random(seed)+ m = Machine(rom)+ boot_len = len(m.tokens)+ ops: Counter[str] = Counter()+ for i in range(steps):+ if i % ipf == 0:+ m.tick()+ if rng.random() < 0.1:+ m.set_key(rng.randrange(16), rng.random() < 0.5)+ ops[opcode_class(fetch(m.emu.cells))] += 1+ m.step()+ return m, ops, boot_len+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("rom", type=Path)+ ap.add_argument("--steps", type=int, default=100_000)+ ap.add_argument("--ipf", type=int, default=10)+ args = ap.parse_args()++ m, ops, boot_len = profile(args.rom.read_bytes(), args.steps, args.ipf)+ total = sum(ops.values())+ print(f"{'opcode':8} {'count':>9} {'share':>7}")+ for op, cnt in ops.most_common():+ print(f"{op:8} {cnt:9,} {cnt / total:7.2%}")+ gen = len(m.tokens) - boot_len+ print(f"\nboot prompt: {boot_len:,} tokens")+ print(f"generated: {gen:,} tokens over {total:,} instructions "+ f"({gen / total:.2f} tokens/instruction incl. ticks and input)")+ print(f"at {args.ipf} instr/frame, 60 Hz needs ~{gen / total * args.ipf * 60:,.0f} tok/s")+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/__init__.py b/experimental/riscv/chipformer/rv/__init__.pynew file mode 100644index 0000000..021b197--- /dev/null+++ b/experimental/riscv/chipformer/rv/__init__.py@@ -0,0 +1 @@+"""RV32IM inside Qwen3.5's Gated DeltaNet layers (P0 feasibility, see docs/rv32.md)."""diff --git a/experimental/riscv/chipformer/rv/asm.py b/experimental/riscv/chipformer/rv/asm.pynew file mode 100644index 0000000..e99a938--- /dev/null+++ b/experimental/riscv/chipformer/rv/asm.py@@ -0,0 +1,269 @@+"""A small two-pass RV32IM assembler for tests (gcc in WSL is the real toolchain).++ label: define a label+ .text / .data / .rodata switch section (text at ROM_BASE, data at DATA_BASE, rodata at+ DROM_BASE)+ .word 1, 2, label 32-bit words .byte / .half likewise+ .space N N zero bytes .align N to 2^N bytes+ ; or # comment++Instructions: RV32IM and Zba (sh1add / sh2add / sh3add) with ABI register names, plus the+pseudo-instructions+li, la, mv, not, neg, seqz, snez, nop, j, jr, ret, call, tail, beqz, bnez, blez, bgez, bltz,+bgtz, bgt, ble, bgtu, bleu.+"""+# ruff: noqa: B023 (the per-pass helpers close over the pass's own text/data/sec)++import re++from chipformer.rv.spec import DATA_BASE, DROM_BASE, ROM_BASE++ABI = ["zero", "ra", "sp", "gp", "tp", "t0", "t1", "t2", "s0", "s1"] + [f"a{i}" for i in range(8)] \+ + [f"s{i}" for i in range(2, 12)] + ["t3", "t4", "t5", "t6"]+REG = {n: i for i, n in enumerate(ABI)} | {f"x{i}": i for i in range(32)} | {"fp": 8}++R_OPS = { # funct7, funct3 (opcode 0x33)+ "add": (0, 0), "sub": (0x20, 0), "sll": (0, 1), "slt": (0, 2), "sltu": (0, 3), "xor": (0, 4),+ "srl": (0, 5), "sra": (0x20, 5), "or": (0, 6), "and": (0, 7),+ "mul": (1, 0), "mulh": (1, 1), "mulhsu": (1, 2), "mulhu": (1, 3), "div": (1, 4),+ "divu": (1, 5), "rem": (1, 6), "remu": (1, 7),+ "sh1add": (0x10, 2), "sh2add": (0x10, 4), "sh3add": (0x10, 6)}+I_OPS = {"addi": 0, "slti": 2, "sltiu": 3, "xori": 4, "ori": 6, "andi": 7}+SH_OPS = {"slli": (0, 1), "srli": (0, 5), "srai": (0x20, 5)}+LOADS = {"lb": 0, "lh": 1, "lw": 2, "lbu": 4, "lhu": 5}+STORES = {"sb": 0, "sh": 1, "sw": 2}+BRANCHES = {"beq": 0, "bne": 1, "blt": 4, "bge": 5, "bltu": 6, "bgeu": 7}+CSRS = {"cycle": 0xC00, "time": 0xC01, "instret": 0xC02, "cycleh": 0xC80, "instreth": 0xC82,+ "mhartid": 0xF14, "misa": 0x301, "mscratch": 0x340}+++def enc_r(f7, rs2, rs1, f3, rd, op):+ return f7 << 25 | rs2 << 20 | rs1 << 15 | f3 << 12 | rd << 7 | op+++def enc_i(imm, rs1, f3, rd, op):+ assert -2048 <= imm < 2048, imm+ return (imm & 0xFFF) << 20 | rs1 << 15 | f3 << 12 | rd << 7 | op+++def enc_s(imm, rs2, rs1, f3, op=0x23):+ assert -2048 <= imm < 2048, imm+ imm &= 0xFFF+ return (imm >> 5) << 25 | rs2 << 20 | rs1 << 15 | f3 << 12 | (imm & 31) << 7 | op+++def enc_b(off, rs2, rs1, f3):+ assert -4096 <= off < 4096 and off % 2 == 0, off+ o = off & 0x1FFF+ return ((o >> 12 & 1) << 31 | (o >> 5 & 63) << 25 | rs2 << 20 | rs1 << 15 | f3 << 12+ | (o >> 1 & 15) << 8 | (o >> 11 & 1) << 7 | 0x63)+++def enc_u(imm20, rd, op):+ return (imm20 & 0xFFFFF) << 12 | rd << 7 | op+++def enc_j(off, rd):+ assert -(1 << 20) <= off < 1 << 20 and off % 2 == 0, off+ o = off & 0x1FFFFF+ return ((o >> 20 & 1) << 31 | (o >> 1 & 0x3FF) << 21 | (o >> 11 & 1) << 20+ | (o >> 12 & 0xFF) << 12 | rd << 7 | 0x6F)+++def hi_lo(v):+ """lui/addi split of a 32-bit constant."""+ v &= 0xFFFFFFFF+ lo = v & 0xFFF+ lo = lo - 4096 if lo >= 2048 else lo+ return ((v - lo) >> 12) & 0xFFFFF, lo+++class Program:+ def __init__(self, text: list[int], data: bytes, labels: dict[str, int],+ data_base: int = DATA_BASE, rodata: bytes = b""):+ self.text, self.data, self.labels, self.data_base = text, data, labels, data_base+ self.rodata = rodata # read-only data from DROM_BASE++ @property+ def entry(self) -> int:+ return self.labels.get("_start", ROM_BASE)+++def assemble(src: str) -> Program:+ lines = []+ for raw in src.splitlines():+ line = re.split(r"[;#]", raw)[0].strip()+ while line:+ m = re.match(r"([A-Za-z_.$][\w.$]*):\s*(.*)", line)+ if not m:+ break+ lines.append(("label", m[1]))+ line = m[2].strip()+ if line:+ op, _, rest = line.partition(" ")+ args = [a.strip() for a in rest.split(",")] if rest.strip() else []+ lines.append((op.lower(), args))+ labels: dict[str, int] = {}+ for pass_ in range(2):+ text, data, rodata = [], bytearray(), bytearray()+ sec = "text"++ def here():+ if sec == "text":+ return ROM_BASE + 4 * len(text)+ return DATA_BASE + len(data) if sec == "data" else DROM_BASE + len(rodata)++ def buf():+ return data if sec == "data" else rodata++ def val(s):+ s = s.strip()+ m = re.fullmatch(r"%(hi|lo)\((.+)\)", s)+ if m:+ h, lo = hi_lo(val(m[2]))+ return h if m[1] == "hi" else lo+ if s in labels:+ return labels[s]+ if re.fullmatch(r"[A-Za-z_.$][\w.$]*", s):+ if pass_:+ raise ValueError(f"unknown label {s}")+ return here() # (a forward label in pass 0: any in-range placeholder)+ return int(s, 0)++ def reg(s):+ return REG[s.strip()]++ def mem(s): # "off(reg)"+ m = re.fullmatch(r"(.*)\((\w+)\)", s.strip())+ return (val(m[1]) if m[1].strip() else 0), reg(m[2])++ def emit(*words):+ assert sec == "text", "instructions belong in .text"+ text.extend(w & 0xFFFFFFFF for w in words)++ for op, args in lines:+ if op == "label":+ labels[args] = here()+ continue+ if op in (".text", ".data", ".rodata"):+ sec = op[1:]+ continue+ if op in (".word", ".half", ".byte"):+ n = {".word": 4, ".half": 2, ".byte": 1}[op]+ for a in args:+ v = val(a) & ((1 << 8 * n) - 1)+ if sec != "text":+ buf().extend(v.to_bytes(n, "little"))+ else:+ assert n == 4+ text.append(v)+ continue+ if op == ".space":+ buf().extend(bytes(val(args[0])))+ continue+ if op == ".align":+ a = 1 << val(args[0])+ if sec != "text":+ buf().extend(bytes(-len(buf()) % a))+ continue+ if op == ".globl":+ continue+ pc = here()+ a = args+ if op in R_OPS:+ f7, f3 = R_OPS[op]+ emit(enc_r(f7, reg(a[2]), reg(a[1]), f3, reg(a[0]), 0x33))+ elif op in ("bmac", "bmsb", "bld", "bget"): # rd, rs1, rs2 (operands in rs2)+ f3 = {"bmac": 0, "bmsb": 1, "bld": 2, "bget": 3}[op]+ emit(enc_r(0, reg(a[2]), reg(a[1]), f3, reg(a[0]), 0x0B))+ elif op == "bscan": # rd, rs1+ emit(enc_r(0, 0, reg(a[1]), 4, reg(a[0]), 0x0B))+ elif op == "bprd": # dD, k+ emit(enc_i(val(a[1]), 0, 0, val(a[0]), 0x2B))+ elif op in I_OPS:+ emit(enc_i(val(a[2]), reg(a[1]), I_OPS[op], reg(a[0]), 0x13))+ elif op in SH_OPS:+ f7, f3 = SH_OPS[op]+ emit(enc_r(f7, val(a[2]) & 31, reg(a[1]), f3, reg(a[0]), 0x13))+ elif op in LOADS:+ off, rs1 = mem(a[1])+ emit(enc_i(off, rs1, LOADS[op], reg(a[0]), 0x03))+ elif op in STORES:+ off, rs1 = mem(a[1])+ emit(enc_s(off, reg(a[0]), rs1, STORES[op]))+ elif op in BRANCHES:+ emit(enc_b(val(a[2]) - pc, reg(a[1]), reg(a[0]), BRANCHES[op]))+ elif op in ("bgt", "ble", "bgtu", "bleu"):+ real = {"bgt": "blt", "ble": "bge", "bgtu": "bltu", "bleu": "bgeu"}[op]+ emit(enc_b(val(a[2]) - pc, reg(a[0]), reg(a[1]), BRANCHES[real]))+ elif op in ("beqz", "bnez", "blez", "bgez", "bltz", "bgtz"):+ r, t = reg(a[0]), val(a[1]) - pc+ emit({"beqz": enc_b(t, 0, r, 0), "bnez": enc_b(t, 0, r, 1),+ "blez": enc_b(t, r, 0, 5), "bgez": enc_b(t, 0, r, 5),+ "bltz": enc_b(t, 0, r, 4), "bgtz": enc_b(t, r, 0, 4)}[op])+ elif op == "lui":+ emit(enc_u(val(a[1]), reg(a[0]), 0x37))+ elif op == "auipc":+ emit(enc_u(val(a[1]), reg(a[0]), 0x17))+ elif op == "jal":+ if len(a) == 1:+ emit(enc_j(val(a[0]) - pc, 1))+ else:+ emit(enc_j(val(a[1]) - pc, reg(a[0])))+ elif op == "jalr":+ if len(a) == 1:+ emit(enc_i(0, reg(a[0]), 0, 1, 0x67))+ elif "(" in a[1]:+ off, rs1 = mem(a[1])+ emit(enc_i(off, rs1, 0, reg(a[0]), 0x67))+ else:+ emit(enc_i(val(a[2]) if len(a) > 2 else 0, reg(a[1]), 0, reg(a[0]), 0x67))+ elif op == "j":+ emit(enc_j(val(a[0]) - pc, 0))+ elif op == "jr":+ emit(enc_i(0, reg(a[0]), 0, 0, 0x67))+ elif op == "ret":+ emit(enc_i(0, 1, 0, 0, 0x67))+ elif op in ("call", "tail"): # auipc + jalr (always two words)+ off = val(a[0]) - pc+ hi, lo = hi_lo(off)+ link = 1 if op == "call" else 0+ tmp = 1 if op == "call" else 6+ emit(enc_u(hi, tmp, 0x17), enc_i(lo, tmp, 0, link, 0x67))+ elif op == "li":+ v = val(a[1])+ if -2048 <= v < 2048:+ emit(enc_i(v, 0, 0, reg(a[0]), 0x13))+ else:+ hi, lo = hi_lo(v)+ emit(enc_u(hi, reg(a[0]), 0x37), enc_i(lo, reg(a[0]), 0, reg(a[0]), 0x13))+ elif op == "la": # lui + addi (absolute: the code is not relocated)+ hi, lo = hi_lo(val(a[1]))+ emit(enc_u(hi, reg(a[0]), 0x37), enc_i(lo, reg(a[0]), 0, reg(a[0]), 0x13))+ elif op == "mv":+ emit(enc_i(0, reg(a[1]), 0, reg(a[0]), 0x13))+ elif op == "not":+ emit(enc_i(-1, reg(a[1]), 4, reg(a[0]), 0x13))+ elif op == "neg":+ emit(enc_r(0x20, reg(a[1]), 0, 0, reg(a[0]), 0x33))+ elif op == "seqz":+ emit(enc_i(1, reg(a[1]), 3, reg(a[0]), 0x13))+ elif op == "snez":+ emit(enc_r(0, reg(a[1]), 0, 3, reg(a[0]), 0x33))+ elif op == "nop":+ emit(enc_i(0, 0, 0, 0, 0x13))+ elif op == "ecall":+ emit(0x00000073)+ elif op == "ebreak":+ emit(0x00100073)+ elif op == "fence":+ emit(0x0FF0000F)+ elif op == "csrr": # csrrs rd, csr, x0+ emit(CSRS.get(a[1], 0) << 20 | 2 << 12 | reg(a[0]) << 7 | 0x73+ if a[1] in CSRS else val(a[1]) << 20 | 2 << 12 | reg(a[0]) << 7 | 0x73)+ elif op == "csrw": # csrrw x0, csr, rs+ csr = CSRS[a[0]] if a[0] in CSRS else val(a[0])+ emit(csr << 20 | reg(a[1]) << 15 | 1 << 12 | 0x73)+ else:+ raise ValueError(f"unknown instruction {op} {args}")+ return Program(text, bytes(data), labels, rodata=bytes(rodata))diff --git a/experimental/riscv/chipformer/rv/bridge.py b/experimental/riscv/chipformer/rv/bridge.pynew file mode 100644index 0000000..b02cbe0--- /dev/null+++ b/experimental/riscv/chipformer/rv/bridge.py@@ -0,0 +1,185 @@+"""The P3 bridge between Qwen's text and the RV32 machine (the reference the circuit must match).++Qwen calls a tool with its native syntax:++ <tool_call>\\n<function=calc>\\n<parameter=expression>\\n123*456\\n</parameter>\\n...</tool_call>++- Between ``<tool_call>`` and ``</tool_call>`` every text token is captured: row i of the text+ buffer (TEXT_ADDR + 16 i, i = 1 .. CAP_MAX) holds the i-th token's UTF-8 bytes, NUL-padded.+ The buffer is cleared at ``<tool_call>``.+- At ``</tool_call>`` the circuit takes over: it predicts GO, and the machine runs from where it+ parked (the firmware's ``exit`` ecall, HALT, is the park; GO resumes after it).+- Every putchar is a BYTE record followed by its echo: the Qwen token of that byte (or, for+ 256.., a special token such as ``<tool_response>``), which Qwen reads as ordinary text. HALT+ is echoed as a newline, and Qwen predicts again from that newline on.+- The circuit owns (predicts) the positions from ``</tool_call>`` through the last echo but+ one; Qwen never sees the chip tokens of a run (decoding with transparent chip tokens).++A token of the circuit's stream is a chip token (< spec.VOCAB) or ``spec.TEXT_TOKEN | qwen_id``.+"""++from functools import cached_property+from pathlib import Path++from chipformer.rv import ref+from chipformer.rv import spec as S++# special tokens the firmware prints with putchar(256 + k)+SPECIALS = ["<tool_response>", "</tool_response>", "<think>", "</think>"]+++def bytes_to_unicode() -> dict[int, str]:+ """GPT-2's byte-level BPE alphabet: byte -> the character that stands for it."""+ bs = list(range(ord("!"), ord("~") + 1)) + list(range(ord("¡"), ord("¬") + 1)) \+ + list(range(ord("®"), ord("ÿ") + 1))+ cs = bs[:]+ n = 0+ for b in range(256):+ if b not in bs:+ bs.append(b)+ cs.append(256 + n)+ n += 1+ return dict(zip(bs, map(chr, cs)))+++class Text:+ """Qwen's vocabulary as the bridge sees it: each token's bytes, and the printable tokens."""++ def __init__(self, tokenizer_json: str | Path) -> None:+ from tokenizers import Tokenizer++ self.tok = Tokenizer.from_file(str(tokenizer_json))+ vocab = self.tok.get_vocab(with_added_tokens=True)+ self.n = max(vocab.values()) + 1+ dec = {c: b for b, c in bytes_to_unicode().items()}+ added = {a.content for a in self.tok.get_added_tokens_decoder().values()} \+ if hasattr(self.tok, "get_added_tokens_decoder") else set()+ self.bytes = [b""] * self.n+ for s, i in vocab.items():+ if s in added or not all(ch in dec for ch in s):+ self.bytes[i] = s.encode()+ else:+ self.bytes[i] = bytes(dec[ch] for ch in s)+ ids = {s: i for s, i in vocab.items()}+ self.TCO, self.TCC = ids["<tool_call>"], ids["</tool_call>"]+ u = bytes_to_unicode()+ self.printable = [ids[u[b]] for b in range(256)] + [ids[s] for s in SPECIALS]+ self.pindex = {q: p for p, q in enumerate(self.printable)}+ self.NL = self.printable[10]++ def printable_index(self, qid: int) -> int | None:+ return self.pindex.get(qid)++ def row_bytes(self, qid: int) -> bytes:+ """A token's text-buffer row: its first TEXT_ROW bytes, NUL-padded."""+ b = self.bytes[qid] if 0 <= qid < self.n else b""+ return b[:S.TEXT_ROW].ljust(S.TEXT_ROW, bytes(1))++ def encode(self, s: str) -> list[int]:+ return self.tok.encode(s, add_special_tokens=False).ids++ def decode_bytes(self, qids: list[int]) -> bytes:+ return b"".join(self.bytes[q] for q in qids)++ @cached_property+ def byte_token(self) -> dict[int, int]:+ return {b: q for b, q in enumerate(self.printable[:256])}+++def default_tokenizer() -> Path:+ """A Qwen3.5 tokenizer.json from the local HF cache."""+ import glob+ import os++ hub = Path(os.environ.get("HF_HOME", Path.home() / ".cache" / "huggingface")) / "hub"+ for name in ("Qwen3.5-4B", "Qwen3.5-0.8B", "Qwen3.5-9B"):+ hits = sorted(glob.glob(str(hub / f"models--Qwen--{name}" / "snapshots" / "*" /+ "tokenizer.json")))+ if hits:+ return Path(hits[-1])+ raise FileNotFoundError("no Qwen3.5 tokenizer.json in the HF cache")+++def text_token(qid: int) -> int:+ return S.TEXT_TOKEN | qid+++def park(m: "ref.Machine", max_records: int = 10**6) -> list[int]:+ """Run a freshly loaded firmware to its first park (exit ecall); returns that trace."""+ toks = m.run(max_records)+ assert m.halted and m.fault is None, (m.fault, m.out)+ m.halted = False+ return toks+++class Session:+ """The token stream of a conversation with the machine, and who predicts each next token.++ ``toks[i]`` is the i-th token; ``own[i]`` is True where the circuit predicts ``toks[i + 1]``.+ The machine starts parked (its state is the snapshot the host injects)."""++ def __init__(self, text: Text, machine: "ref.Machine") -> None:+ self.T, self.m = text, machine+ machine.text = [bytes(S.TEXT_ROW)] * S.TEXT_ROWS+ self.toks: list[int] = []+ self.own: list[bool] = []+ self.armed, self.cap = False, 0+ self.runs: list[list[int]] = [] # each run's echoed Qwen tokens++ def _push(self, t: int, owner: bool) -> None:+ self.toks.append(t)+ self.own.append(owner)++ def history(self, qids: list[int]) -> None:+ """Prefilled text (an earlier turn, its tool responses already written out): captured+ like any text, but no run follows its </tool_call>s (the circuit owns those positions+ and predicts GO, but the stream goes on with the history's own tokens)."""+ self.qwen(qids, run=False)++ def qwen(self, qids: list[int], max_records: int = 10**6, run: bool = True) -> None:+ """Tokens Qwen writes (a run follows each </tool_call>)."""+ for q in qids:+ if q == self.T.TCO:+ self.armed, self.cap = True, 0+ self.m.text = [bytes(S.TEXT_ROW)] * S.TEXT_ROWS+ elif q == self.T.TCC:+ self.armed = False+ elif self.armed:+ self.cap += 1+ rows = list(self.m.text)+ rows[min(self.cap, S.CAP_MAX)] = self.T.row_bytes(q)+ self.m.text = rows+ self._push(text_token(q), q == self.T.TCC)+ if q == self.T.TCC and run:+ self.run(max_records)++ def run(self, max_records: int = 10**6) -> list[int]:+ """GO, the machine's records until HALT, with every putchar and the HALT echoed."""+ m, echo = self.m, []+ self._push(S.tok("GO"), True)+ m.halted = False+ n = 0+ while n < max_records:+ rec = m.record()+ n += 1+ for t in rec:+ self._push(t, True)+ fam, p = S.family(t)+ if fam == "BYTE":+ q = self.T.printable[p] if p < len(self.T.printable) else self.T.NL+ echo.append(q)+ self._push(text_token(q), True)+ if rec[-1] == S.tok("HALT"):+ echo.append(self.T.NL)+ self._push(text_token(self.T.NL), False)+ break+ else:+ raise RuntimeError("the run did not halt")+ if m.fault is None:+ m.halted = False # parked: the next GO resumes after the exit ecall+ self.runs.append(echo)+ return echo++ def qwen_text(self) -> list[int]:+ """The Qwen tokens of the stream (what Qwen sees: chip tokens are transparent)."""+ return [t - S.TEXT_TOKEN for t in self.toks if S.is_text(t)]diff --git a/experimental/riscv/chipformer/rv/calc.py b/experimental/riscv/chipformer/rv/calc.pynew file mode 100644index 0000000..89b80e9--- /dev/null+++ b/experimental/riscv/chipformer/rv/calc.py@@ -0,0 +1,226 @@+"""The calc firmware (env/calc): exact decimal integer expressions on the digit file.++ python -m chipformer.rv.calc --cases 10000 # reference machine vs Python+ python -m chipformer.rv.calc --expr "12345678*87654321"++``machine(expr)`` is the firmware's snapshot with the expression in calc_in (RAM 0x100);+``expected(expr)`` is Python's answer in the firmware's output format; ``cases(n, seed)`` draws+random problems per operation (add, sub, mul, div, mod, gcd, pow, mixed expressions).+"""++import argparse+import random+import time+from math import gcd+from pathlib import Path++from chipformer.rv import elf, ref+from chipformer.rv import spec as S++ELF = Path(__file__).parents[2] / "tests" / "data" / "rv" / "calc.elf"+CALC_IN, MAXD = 0x100, 62+++def machine(expr: str, path=ELF) -> ref.Machine:+ m = ref.Machine.from_program(elf.program(path))+ b = expr.encode() + b"\0"+ assert len(b) <= 256+ for i, x in enumerate(b):+ a = CALC_IN + i+ w = a >> 2+ m.ram[w] = m.ram[w] & ~(0xFF << 8 * (a & 3)) | x << 8 * (a & 3)+ return m+++def _floordiv(a, b):+ if b == 0:+ raise ZeroDivisionError+ return a // b+++def _mod(a, b):+ if b == 0:+ raise ZeroDivisionError+ return a % b+++def expected_decimal(expr: str) -> str:+ """A decimal expression: + - * exact (other operators need integers), printed without+ trailing zeros after the point."""+ import decimal+ import re++ if re.search(r"[/%^]|gcd", expr) or "**" in expr:+ return "error\n"+ src = re.sub(r"\d+\.\d+|\d+", lambda m: f"D('{m[0]}')", expr)+ with decimal.localcontext() as ctx:+ ctx.prec = 400+ try:+ v = eval(src, {"__builtins__": {}}, {"D": decimal.Decimal}) # noqa: S307+ except SyntaxError:+ return "error\n"+ s = format(v, "f")+ if "." in s:+ s = s.rstrip("0").rstrip(".")+ if s in ("-0", ""):+ s = "0"+ return "error\n" if len(s.replace("-", "").replace(".", "").lstrip("0")) > MAXD else f"{s}\n"+++class _Err(Exception):+ pass+++def _value(node):+ """An integer expression's value as a Fraction: / is exact, but a non-integer value may only+ be the final result (or negated); // and % floor, ** needs an exponent 0..999."""+ import ast+ from fractions import Fraction++ def ints(*vs):+ if any(v.denominator != 1 for v in vs):+ raise _Err+ return [int(v) for v in vs]++ if isinstance(node, ast.Constant) and type(node.value) is int:+ return Fraction(node.value)+ if isinstance(node, ast.UnaryOp) and isinstance(node.op, (ast.USub, ast.UAdd)):+ v = _value(node.operand)+ return -v if isinstance(node.op, ast.USub) else v+ if isinstance(node, ast.Call) and getattr(node.func, "id", None) == "gcd" and \+ len(node.args) == 2 and not node.keywords:+ return Fraction(gcd(*ints(*map(_value, node.args))))+ if not isinstance(node, ast.BinOp):+ raise _Err+ a, b = ints(_value(node.left), _value(node.right))+ op = type(node.op)+ if op in (ast.Div, ast.FloorDiv, ast.Mod) and b == 0:+ raise _Err+ if op is ast.Pow and not 0 <= b <= 999:+ raise _Err+ f = {ast.Add: lambda: a + b, ast.Sub: lambda: a - b, ast.Mult: lambda: a * b,+ ast.Div: lambda: Fraction(a, b), ast.FloorDiv: lambda: a // b, ast.Mod: lambda: a % b,+ ast.Pow: lambda: a**b}.get(op)+ if f is None:+ raise _Err+ return Fraction(f())+++def expected(expr: str) -> str:+ """Python's value (/ exact: an integer or a reduced fraction p/q; // and % floor, ** / ^+ powers, gcd) or "error", as the firmware prints it."""+ import ast++ if "." in expr:+ return expected_decimal(expr)+ try:+ v = _value(ast.parse(expr.replace("^", "**"), mode="eval").body)+ except (_Err, SyntaxError):+ return "error\n"+ s = str(v) # Fraction: "p" or "p/q"+ return "error\n" if any(len(x) > MAXD for x in s.lstrip("-").split("/")) else f"{s}\n"+++def _n(rng, d):+ return str(rng.randint(10 ** (d - 1) if d > 1 else 0, 10**d - 1))+++OPS = ["add", "sub", "mul", "div", "mod", "gcd", "pow", "expr", "frac"]+++def case(rng: random.Random, op: str) -> str:+ da, db = rng.randint(1, 24), rng.randint(1, 24)+ a, b = _n(rng, da), _n(rng, db)+ sa = "-" if rng.random() < 0.2 else ""+ sb = "-" if rng.random() < 0.2 else ""+ if op == "add":+ return f"{sa}{a}+{b}"+ if op == "sub":+ return f"{a}-{sb}{b}" if sb else f"{sa}{a}-{b}"+ if op == "mul":+ return f"{sa}{a}*{_n(rng, rng.randint(1, 24))}"+ if op in ("div", "mod"):+ a = _n(rng, rng.randint(1, 30))+ b = _n(rng, rng.randint(1, max(1, min(len(a), 15))))+ return f"{sa}{a}{'//' if op == 'div' else '%'}{'(-' + b + ')' if sb else b}"+ if op == "gcd":+ g = int(_n(rng, rng.randint(1, 6)))+ x, y = int(_n(rng, rng.randint(1, 12))) * g, int(_n(rng, rng.randint(1, 12))) * g+ return f"gcd({x},{y})"+ if op == "pow":+ base = _n(rng, rng.randint(1, 4))+ e = rng.randint(0, max(0, MAXD // len(base) - 1))+ return f"{sa}{base}**{e}" if not sa else f"({sa}{base})**{e}"+ if op == "frac": # exact division: often a reduced fraction, sometimes an integer+ g = int(_n(rng, rng.randint(1, 8)))+ x, y = int(_n(rng, rng.randint(1, 20))) * g, int(_n(rng, rng.randint(1, 12))) * g+ if rng.random() < 0.3:+ x = y * int(_n(rng, rng.randint(1, 8)))+ return rng.choice([f"{sa}{x}/{y}", f"({x}+{y})/{sb}{y}", f"{x}*{g}/({y}-{g})"])+ # a random expression of 2-4 operands+ k = rng.randint(2, 4)+ nums = [_n(rng, rng.randint(1, 8)) for _ in range(k)]+ s = nums[0]+ for x in nums[1:]:+ o = rng.choice(["+", "-", "*", "//", "%"])+ s = f"({s}){o}{x}" if rng.random() < 0.3 else f"{s}{o}{x}"+ return s+++def cases(n: int, seed: int = 0) -> list[tuple[str, str]]:+ rng = random.Random(seed)+ return [(op, case(rng, op)) for i in range(n) for op in [OPS[i % len(OPS)]]]+++def run(expr: str) -> tuple[str, list[int]]:+ m = machine(expr)+ toks = m.run(10**7)+ assert m.halted and m.fault is None, (expr, m.fault)+ return m.out.decode(), toks+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--cases", type=int, default=0)+ ap.add_argument("--seed", type=int, default=0)+ ap.add_argument("--expr", default=None)+ ap.add_argument("--free-run", action="store_true", help="the circuit's executor on the GPU")+ ap.add_argument("--batch", type=int, default=32)+ args = ap.parse_args()+ if args.free_run:+ from functools import partial++ from chipformer.rv import core, fuzz++ c = core.build()+ exprs = [e for _, e in cases(args.cases, args.seed)]+ exprs.sort(key=lambda e: len(machine(e).run(10**7))) # (a batch runs to its longest)+ res = fuzz.free_run(c, [partial(machine, e) for e in exprs], args.batch)+ print(f"calc free-running: {res}", flush=True)+ return+ if args.expr:+ out, toks = run(args.expr)+ st = ref.trace_stats(toks)+ print(repr(out), expected(args.expr) == out, st)+ return+ t = time.time()+ per = {op: [0, 0, 0] for op in OPS} # cases, instructions, tokens+ bad = 0+ for op, e in cases(args.cases, args.seed):+ out, toks = run(e)+ if out != expected(e):+ bad += 1+ print("MISMATCH", e, repr(out), repr(expected(e)))+ st = ref.trace_stats(toks)+ per[op][0] += 1+ per[op][1] += st["instructions"]+ per[op][2] += st["tokens"]+ print(f"{args.cases} cases, {bad} mismatches ({time.time() - t:.0f} s)")+ for op, (n, i, tk) in per.items():+ if n:+ print(f" {op:5s} {n:5d} cases {i / n:8.0f} instructions {tk / n:8.0f} tokens")+ _ = S+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/chat.py b/experimental/riscv/chipformer/rv/chat.pynew file mode 100644index 0000000..fa69e7e--- /dev/null+++ b/experimental/riscv/chipformer/rv/chat.py@@ -0,0 +1,392 @@+"""Talk to a merged rv1 checkpoint: Qwen writes, the RV32 machine runs inside the same model.++ python -m chipformer.rv.chat dist/q4-rv1 --firmware tests/data/rv/c-calc.elf \\+ --ask "What is 123456789 * 987654321?"++Decoding is greedy and transparent to chip tokens (as BatchDecoder's fp + tc): a chip token+runs through the chip layers only, so Qwen's layers, caches and RoPE positions see the text+alone; a text token runs through every layer. At every position the next token is the+circuit's when its output code is set (the owner), else Qwen's argmax over its own vocabulary.+The firmware (an ELF, parked at its first exit ecall) is a state snapshot written into the chip+layers' recurrent states after the prompt's prefill (rv/firmware.py, rv/hf.py).++``--check`` replays the whole stream through the IR evaluator and the reference machine: every+circuit-owned token must equal the IR's prediction and the reference's next token.+"""++import argparse+import json+import time+from pathlib import Path++import numpy as np+import torch++from chipformer.rv import bridge as BR+from chipformer.rv import core, elf, hf, ref+from chipformer.rv import spec as S++CALC_TOOL = {"type": "function", "function": {+ "name": "calc",+ "description": "Evaluate an arithmetic expression exactly, with numbers of up to 62 digits: "+ "+ - * and parentheses on integers or decimals; on integers also / (exact: "+ "an integer or a reduced fraction p/q), // (floor division), % , ** and "+ "gcd(a,b). Returns the exact value.",+ "parameters": {"type": "object", "properties": {+ "expression": {"type": "string",+ "description": "e.g. 123456789*987654321-42 or 1.25*-3.5 or 84/18"}},+ "required": ["expression"]}}}+++class RvChat:+ def __init__(self, path, device: str | None = None, fast: bool = True) -> None:+ from transformers import AutoTokenizer, Qwen3_5ForCausalLM++ from chipformer.q35.fast import exact_matmuls+ from chipformer.q35.ir import Target++ exact_matmuls()+ self.path = Path(path)+ self.meta = json.loads((self.path / "chipformer.json").read_text())+ self.dev = device or ("cuda" if torch.cuda.is_available() else "cpu")+ self.model = Qwen3_5ForCausalLM.from_pretrained(self.path, dtype=torch.bfloat16)+ self.model = self.model.to(self.dev).eval()+ self.lm = self.model.model+ self.tok = AutoTokenizer.from_pretrained(self.path)+ self.T = BR.Text(self.path / "tokenizer.json")+ cfg = self.model.config+ self.c = core.build(Target(cfg.linear_num_key_heads, cfg.linear_num_value_heads,+ cfg.intermediate_size), self.T)+ self.n_chip, self.v0 = self.meta["chip_layers"], self.meta["chip_token_offset"]+ self.code_dims = torch.tensor(self.meta["code_dims"], device=self.dev)+ self.eos = {248046, 248044}+ self.fast = fast and self.dev == "cuda"+ if self.fast:+ self._build_fast()++ # --- the fast chip path ----------------------------------------------------------------+ def _build_fast(self) -> None:+ """A second view of the chip layers: a text model of the n chip layers (the same module+ objects) plus one zero full-attention layer, its linear maps compacted to their nonzero+ blocks (fast.compact; exact), with its own static cache, run as one CUDA graph per token+ (torch.compile reduce-overhead). Chip tokens run only here; text tokens run here and then+ through Qwen's layers with the main cache."""+ import copy++ from transformers.models.qwen3_5 import modeling_qwen3_5 as q++ from chipformer.q35 import fast as F++ cfg = copy.deepcopy(self.model.config)+ cfg.num_hidden_layers = self.n_chip + 1+ cfg.layer_types = ["linear_attention"] * self.n_chip + ["full_attention"]+ self.ccfg = cfg+ dummy = q.Qwen3_5DecoderLayer(cfg, self.n_chip).to(self.dev, torch.bfloat16)+ for p in dummy.parameters():+ p.data.zero_()+ with torch.device("meta"):+ cm = q.Qwen3_5TextModel(cfg)+ cm.embed_tokens = self.lm.embed_tokens+ cm.layers = torch.nn.ModuleList([*self.lm.layers[: self.n_chip], dummy])+ cm.norm = torch.nn.Identity()+ cm.rotary_emb = q.Qwen3_5TextRotaryEmbedding(config=cfg).to(self.dev)+ self.cm = cm.eval()++ class _Wrap(torch.nn.Module): # fast.compact's view of it+ def __init__(self, inner):+ super().__init__()+ self.model = inner++ self.compact = F.compact(_Wrap(cm), qwen=False, n_chip=self.n_chip)+ self.graph = None # captured after the first prefill (the cache buffers are static)++ def _chip_cache(self):+ """The chip view's cache, allocated once: its state buffers keep their addresses (HF+ updates them in place), so one CUDA graph serves every conversation."""+ from transformers import DynamicCache++ if getattr(self, "ccache", None) is None:+ self.ccache = DynamicCache(config=self.ccfg)+ else:+ for L in range(self.n_chip):+ lay = self.ccache.layers[L]+ lay.conv_states[0].zero_()+ lay.recurrent_states[0].zero_()+ lay.has_previous_state[0] = False+ return self.ccache++ @torch.no_grad()+ def _chip_body(self):+ h = self.cm.embed_tokens(self.g_ids)+ for L in range(self.n_chip):+ h = self.cm.layers[L](h, position_embeddings=None, attention_mask=None,+ position_ids=None, past_key_values=self.ccache)+ self.g_out.copy_(h)++ @torch.no_grad()+ def _capture(self):+ """One CUDA graph of the chip layers' single-token decode step. Warm-up runs change the+ cache, so the state is saved and put back around them."""+ self.g_ids = torch.zeros((1, 1), dtype=torch.long, device=self.dev)+ self.g_out = torch.zeros((1, 1, self.model.config.hidden_size), dtype=torch.bfloat16,+ device=self.dev)+ lays = [self.ccache.layers[L] for L in range(self.n_chip)]+ saved = [(lay.conv_states[0].clone(), lay.recurrent_states[0].clone()) for lay in lays]+ s = torch.cuda.Stream()+ s.wait_stream(torch.cuda.current_stream())+ with torch.cuda.stream(s):+ for _ in range(3):+ self._chip_body()+ torch.cuda.current_stream().wait_stream(s)+ self.graph = torch.cuda.CUDAGraph()+ with torch.cuda.graph(self.graph):+ self._chip_body()+ for lay, (cs, rs) in zip(lays, saved):+ lay.conv_states[0].copy_(cs)+ lay.recurrent_states[0].copy_(rs)++ @torch.no_grad()+ def _chip_step(self, ids: int, g: int, cache, compiled: bool = True):+ if not compiled or self.graph is None:+ x = torch.tensor([[ids]], device=self.dev)+ out = self.cm(input_ids=x, past_key_values=cache, use_cache=True,+ cache_position=torch.tensor([g], device=self.dev))+ return out.last_hidden_state[0, -1].clone()+ self.g_ids.fill_(ids)+ self.graph.replay()+ return self.g_out[0, -1].clone()++ @torch.no_grad()+ def _qwen_step(self, hc, cache, pos: int):+ """Qwen's layers on a text token whose residual after the chip layers is hc."""+ h = hc.view(1, 1, -1)+ p4 = torch.full((4, 1, 1), pos, device=self.dev, dtype=torch.long)+ pe = self.lm.rotary_emb(h, p4[1:])+ for i in range(self.n_chip, len(self.lm.layers)):+ h = self.lm.layers[i](h, position_embeddings=pe, attention_mask=None,+ position_ids=p4[0], past_key_values=cache)+ return self.model.lm_head(self.lm.norm(h))[0, -1]++ # --- one position ----------------------------------------------------------------------+ @torch.no_grad()+ def _layers(self, ids, cache, pos: int | None):+ """Run one token through the chip layers (pos None) or the whole stack (text at RoPE+ position pos); returns (residual after the chip layers, logits or None)."""+ h = self.lm.embed_tokens(torch.tensor([[ids]], device=self.dev))+ pe = tp = None+ if pos is not None:+ p4 = torch.full((4, 1, 1), pos, device=self.dev, dtype=torch.long)+ tp = p4[0]+ pe = self.lm.rotary_emb(h, p4[1:])+ hc = None+ n = self.n_chip if pos is None else len(self.lm.layers)+ for i in range(n):+ h = self.lm.layers[i](h, position_embeddings=pe, attention_mask=None,+ position_ids=tp, past_key_values=cache)+ if i == self.n_chip - 1:+ hc = h+ logits = None if pos is None else self.model.lm_head(self.lm.norm(h))[0, -1]+ return hc[0, -1], logits++ def circuit_token(self, hc) -> int | None:+ """The model id the circuit predicts at a position, None where Qwen predicts."""+ code = hc[self.code_dims].float()+ if not bool((code != 0).any()):+ return None+ bits = (code > 0).long().tolist()+ nf = core.NFAM+ fid = sum(b << i for i, b in enumerate(bits[:nf]))+ p = sum(b << i for i, b in enumerate(bits[nf:]))+ t = core.decode_code(fid, p, self.T)+ if t < 0:+ raise RuntimeError(f"the circuit predicts no token (family {fid}, payload {p})")+ return t - S.TEXT_TOKEN if S.is_text(t) else self.v0 + t++ # --- a conversation --------------------------------------------------------------------+ def prompt_ids(self, messages, tools=(CALC_TOOL,)) -> list[int]:+ out = self.tok.apply_chat_template(messages, tools=list(tools) or None,+ add_generation_prompt=True, enable_thinking=False,+ tokenize=True)+ if not isinstance(out, list):+ out = out["input_ids"]+ return [int(x) for x in out]++ @torch.no_grad()+ def generate(self, ids: list[int], machine: "ref.Machine", max_new: int = 4000,+ max_chip: int = 60000, check_head: bool = False, verbose: bool = False,+ lock: "BR.Session | None" = None) -> dict:+ """Greedy decoding after the prompt ``ids`` with the parked ``machine`` loaded.+ ``lock`` (a Session on a copy of the machine): stop at the first token that differs from+ the reference's."""+ x = torch.tensor([ids], device=self.dev)+ out = self.model(input_ids=x, use_cache=True)+ cache = out.past_key_values # (fast: Qwen's layers only)+ core.snapshot(self.c, machine)+ if self.fast:+ ccache = self._chip_cache()+ h = self.cm.embed_tokens(x) # the prompt through the chip layers only+ for L in range(self.n_chip):+ h = self.cm.layers[L](h, position_embeddings=None, attention_mask=None,+ position_ids=None, past_key_values=ccache)+ hf.inject(self.c, ccache)+ if self.graph is None:+ self._capture()+ g = len(ids) # the chip view's cache position (every token)+ else:+ hf.inject(self.c, cache)+ logits = out.logits[0, -1]+ pos = len(ids) # the next text token's RoPE position+ gen, n_text, n_chip_tok = [], 0, 0+ stream_hc = []+ # the prompt's last position: Qwen predicts (nothing captured yet)+ nxt = int(torch.argmax(logits[: self.v0]))+ t0 = time.time()+ mismatch = 0+ t_chip = t_text = 0.0+ expect, lock_error = [], None+ while n_text < max_new and n_chip_tok < max_chip:+ gen.append(nxt)+ if lock is not None: # check against the reference as we go+ st = nxt - self.v0 if nxt >= self.v0 else BR.text_token(nxt)+ if expect:+ e = expect.pop(0)+ if st != e:+ lock_error = {"at": len(gen) - 1, "got": S.name(st) if st < S.VOCAB+ else self.tok.decode([nxt]),+ "want": S.name(e) if e < S.VOCAB+ else self.tok.decode([e - S.TEXT_TOKEN]),+ "recent": [S.name(t - self.v0) if t >= self.v0 else+ self.tok.decode([t]) for t in gen[-12:]]}+ break+ elif st < S.VOCAB:+ lock_error = {"at": len(gen) - 1, "got": S.name(st), "want": "text"}+ break+ else:+ k = len(lock.toks)+ lock.qwen([nxt])+ expect = list(lock.toks[k + 1:])+ if nxt in self.eos:+ break+ if verbose and len(gen) % 500 == 0:+ tail = [S.name(t - self.v0) if t >= self.v0 else repr(self.tok.decode([t]))+ for t in gen[-8:]]+ print(f" {len(gen)} tokens ({n_text} text, {n_chip_tok} chip) "+ f"{time.time() - t0:.0f}s: {' '.join(tail)}", flush=True)+ ts = time.time()+ if nxt >= self.v0: # a chip token+ n_chip_tok += 1+ if self.fast:+ hc = self._chip_step(nxt, g, ccache)+ g += 1+ else:+ hc, _ = self._layers(nxt, cache, None)+ nxt = self.circuit_token(hc)+ t_chip += time.time() - ts+ if nxt is None:+ raise RuntimeError("the circuit gave up a chip position")+ else:+ n_text += 1+ if self.fast:+ hc = self._chip_step(nxt, g, ccache)+ g += 1+ logits = self._qwen_step(hc, cache, pos)+ else:+ hc, logits = self._layers(nxt, cache, pos)+ pos += 1+ ct = self.circuit_token(hc)+ q = int(torch.argmax(logits))+ if ct is not None and check_head and q != ct:+ mismatch += 1+ nxt = ct if ct is not None else int(torch.argmax(logits[: self.v0]))+ t_text += time.time() - ts+ if check_head:+ stream_hc.append(hc[self.code_dims].float().cpu().numpy())+ dt = time.time() - t0+ return {"tokens": gen, "text_tokens": n_text, "chip_tokens": n_chip_tok,+ "seconds": dt, "head_mismatch": mismatch, "lock_error": lock_error,+ "chip_seconds": round(t_chip, 2), "text_seconds": round(t_text, 2)}++ def to_stream(self, ids: list[int]) -> list[int]:+ """Model ids -> the circuit's token stream (chip tokens, TEXT_TOKEN | qwen id)."""+ return [i - self.v0 if i >= self.v0 else BR.text_token(i) for i in ids]++ def text_of(self, ids: list[int]) -> str:+ return self.tok.decode([i for i in ids if i < self.v0], skip_special_tokens=False)+++def replay(chat: RvChat, prompt: list[int], gen: list[int], firmware) -> dict:+ """Check a generated stream: the reference machine (fed Qwen's own tokens) must produce the+ same chip tokens and echoes, and the IR circuit must predict every circuit-owned token."""+ from chipformer.q35 import ir++ m = ref.Machine.from_program(elf.program(firmware))+ BR.park(m)+ ses = BR.Session(chat.T, m)+ stream = chat.to_stream(prompt + gen)+ # the prompt is prefilled history (its system prompt shows the tool-call format, so it holds+ # <tool_call> .. </tool_call> tokens: the circuit owns those </tool_call> positions, but no+ # run follows); then the reference is fed Qwen's tokens and appends the runs itself+ ses.history(prompt)+ skip = {x for x in range(len(prompt)) if prompt[x] == chat.T.TCC}+ i = 0+ pos = len(prompt)+ while pos < len(stream):+ t = stream[pos]+ if not S.is_text(t):+ raise AssertionError(f"chip token {S.name(t)} at {pos} where the reference has none")+ k = len(ses.toks)+ ses.qwen([t - S.TEXT_TOKEN])+ pos += 1+ added = ses.toks[k + 1:]+ if added: # a run: the stream must continue with exactly the reference's tokens+ got = stream[pos:pos + len(added)]+ if got != added:+ j = next((x for x in range(min(len(got), len(added))) if got[x] != added[x]),+ min(len(got), len(added)))+ return {"ok": False, "where": pos + j, "got": got[j:j + 5], "want": added[j:j + 5]}+ pos += len(added)+ i += 1+ m2 = ref.Machine.from_program(elf.program(firmware))+ BR.park(m2)+ core.snapshot(chat.c, m2)+ carry, mats = {}, []+ for a in range(0, len(stream), 2048):+ mats.append(ir.evaluate(chat.c, stream[a:a + 2048], carry=carry))+ mat = np.concatenate(mats)+ pred = core.predictions(chat.c, mat)+ own = np.rint(ir.ev(chat.c.owner, mat)).astype(int)+ bad_own = int(sum(own[x] != int(ses.own[x]) for x in range(len(stream))))+ bad = int(sum(1 for x in range(len(stream) - 1)+ if ses.own[x] and x not in skip and pred[x] != stream[x + 1]))+ return {"ok": bad == 0 and bad_own == 0, "positions": len(stream),+ "owned": int(sum(ses.own)) - len(skip), "wrong_predictions": bad,+ "owner_mismatch": bad_own, "runs": len(ses.runs)}+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("model")+ ap.add_argument("--firmware", default="tests/data/rv/calc-bridge.elf")+ ap.add_argument("--ask", default="Use the calc tool: what is 123456789 * 987654321?")+ ap.add_argument("--max-new", type=int, default=600)+ ap.add_argument("--check", action="store_true")+ ap.add_argument("--slow", action="store_true", help="every layer through the HF modules")+ args = ap.parse_args()+ chat = RvChat(args.model, fast=not args.slow)+ m = ref.Machine.from_program(elf.program(args.firmware))+ BR.park(m)+ ids = chat.prompt_ids([{"role": "user", "content": args.ask}])+ m2 = ref.Machine.from_program(elf.program(args.firmware))+ BR.park(m2)+ lock = BR.Session(chat.T, m2)+ lock.history(ids)+ r = chat.generate(ids, m, args.max_new, check_head=True, verbose=True, lock=lock)+ print(chat.text_of(r["tokens"]))+ print(f"[{r['text_tokens']} text + {r['chip_tokens']} chip tokens in {r['seconds']:.1f}s, "+ f"head mismatches {r['head_mismatch']}, lock error {r['lock_error']}]")+ if args.check:+ print(replay(chat, ids, r["tokens"], args.firmware))+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/core.py b/experimental/riscv/chipformer/rv/core.pynew file mode 100644index 0000000..679480f--- /dev/null+++ b/experimental/riscv/chipformer/rv/core.py@@ -0,0 +1,1304 @@+"""The RV32IM + Zba core as a q35 circuit (spec.py has the trace format, ref.py the ROM word).++Every position computes the instruction that the next token belongs to: the state it reads is+the state after the position's own commit, so at a record's last token the circuit already works+on the next instruction, and inside a record it recomputes the current one (nothing changes+until the record's last token). Writes take their key from t-1 (a conv tap), their write gate+beta from the token at t, and their value from token payloads (lags 0..3) or from dims the+circuit computed at t-1 below the writing layer.++ L1 attn anchor (PC head), count (2^-n), j (tokens of the record, 2^-j), v (tokens since the+ last QS / QD)+ mlp thermometers+ L2 mlp PC word w = anchor + n; link word w + 1; nd (quotient digits so far)+ L3-4 attn ROM, two sublayers of 16 key heads x 2 value heads: the 4-byte pre-decoded word w+ mlp uop / register one-hots, raw field bits (the immediates are linear in them)+ L5 attn register file (3 read ports: RS1, RS2, RD slot; writes commit from t-1); the quotient+ head (the record's quotient digits, right-aligned by nd)+ mlp adder stage 1 (4 operand modes), logic ops, shifter stage 1, multiplier stage 1+ (A x B, or Q x B for the remainder), store data, DIV signs / sizes+ L6 mlp adder stage 2, shifter stage 2, multiplier stage 2; DIV: |A| >> 4s, Q x |B| and+ m x |B| products, the signed quotient+ L7-8 attn RAM, two sublayers (32 KB of words)+ mlp multiplier stages 3-4, loads, SB/SH merge, compares, faults, the Zba and branch+ target adders; DIV: product splits, the 15 compare columns+ L9 attn DROM (128 KB of read-only bytes)+ mlp multiplier stage 5, DROM bytes to bits; DIV: compare lookahead+ L10 mlp DIV: the 15 compare signs (the next quotient digit); rd value select and size+ L11 mlp the output code (the exit)++DIV / REM (radix 16, one digit per token): at the digit position for digit k (s = 7 - k) with the+digits so far Q (right-aligned), q_k = #{m in 1..15 : |A| >> 4s >= (16 Q + m) |B|}; each test is+one sign of D - m |B| (D = (|A| >> 4s) - 16 Q |B|) computed mod 2^40 from un-normalized columns+(products of nibbles) in three MLP stages. The remainder is A + Q B or A - Q B (by the result's+sign) through the multiplier.+"""++from contextlib import contextmanager++import numpy as np++from chipformer.dsl.graph import Expr+from chipformer.q35.ir import A_BIT, A_REF, HD, REF_COLS, Chan, K, Target+from chipformer.q35.prims import Circ, chan, from_bits, sel, thead+from chipformer.rv import ref as RF+from chipformer.rv import spec as S++T4B = Target(key_heads=16, value_heads=32, intermediate=9216) # Qwen3.5-4B's layers+# output family ids: the chip families and controls, then Q (P3: a printable Qwen token, the echo+# of a putchar; its payload indexes the bridge's printable tokens)+OUT_NAMES = list(S.FAMILIES) + S.CONTROL_NAMES + ["Q"]+FAM_ID = {n: i for i, n in enumerate(OUT_NAMES)}+NFAM = (len(OUT_NAMES) - 1).bit_length() # family id bits in the output code+NPAY = 16 # payload bits in the output code+OUT_W = 2.0**7 # head output weight of a one-hot read (as thead)+JMAX = 66 # longest record: a digit-file row op (63 digits, then its value)+CODE_AMP = 128 # a token's code in its (tied) embedding row: CODE_AMP * (2 * bit - 1)+KAPPA = 2.0**-20 # the predicted code after the exit: KAPPA * (2 * bit - 1)+TAU_TEXT = 2.0**-40 # (P3) tau on text tokens; -1 on chip tokens (see q35.merge)+PAY_W = {"V": 15, "F0": 15, "F1": 15, "F2": 2, "J": 16, "R": 16, "BYTE": 9, "FLT": 3, "HEX": 4,+ "QS": 4, "QD": 7, "BD": 8}+++def code_bits(fam: str, p: int) -> list[int]:+ fid = FAM_ID[fam]+ return [fid >> i & 1 for i in range(NFAM)] + [p >> i & 1 for i in range(NPAY)]+++def token_code(t: int) -> list[int]:+ """The output-code bits of a chip token: family id (NFAM) then payload (16)."""+ return code_bits(*S.family(t))+++def famof(t: int) -> tuple:+ """(family, payload) of a chip token; (None, 0) for a text token (P3)."""+ return S.family(t) if t < S.VOCAB else (None, 0)+++def lits_of(bits, v):+ """0/1 literals that are all 1 exactly when the bits spell v."""+ return [b if v >> i & 1 else 1 - b for i, b in enumerate(bits)]+++def andpre(lits) -> Expr:+ """sum(lits) - (n - 1): 1 when all literals are 1, <= 0 otherwise (a key / query / z form)."""+ return sum(lits, Expr()) - (len(lits) - 1)+++def col_offsets(bounds: list[tuple[int, int]], top: int = 255) -> list[int]:+ """Offsets off_i = 16 g_i - g_(i-1) (so sum off_i 16^i is a multiple of 16^n) that put every+ column (positive part <= pos_i, negative part <= neg_i) into [0, top]."""+ reach, hist = {0: None}, []+ for pos, neg in bounds:+ nxt = {}+ for gp in reach:+ for g in range(64):+ if neg <= 16 * g - gp <= top - pos and g not in nxt:+ nxt[g] = gp+ assert nxt, "no column offsets"+ hist.append(nxt)+ reach = nxt+ g, offs = min(reach), []+ for i in range(len(bounds) - 1, -1, -1):+ gp = hist[i][g]+ offs.append(16 * g - gp)+ g = gp+ return offs[::-1]+++class RvCore(Circ):+ def __init__(self, target: Target | None = None, text=None) -> None:+ """``text`` (P3, bridge.Text): the Qwen vocabulary the bridge reads and prints; None+ builds the standalone machine."""+ super().__init__(target or T4B)+ f = self.token_feature+ self.text = text++ def fam_is(nm):+ return lambda t: int(famof(t)[0] == nm)++ def pay_bit(nm, i):+ return lambda t: (famof(t)[1] >> i & 1) if famof(t)[0] == nm else 0++ names = list(S.FAMILIES) + S.CONTROL_NAMES+ paid = ("V", "F0", "F1", "F2", "J", "R", "QD", "BD") + (("BYTE",) if text else ())+ self.fam = {n: f(f"is{n}", fam_is(n), full=True) for n in names}+ self.pay = {n: [f(f"{n}.b{i}", pay_bit(n, i), full=True) for i in range(w)]+ for n, w in PAY_W.items() if n in paid}++ def next_digit(k):+ """The token says the next token is quotient digit k: QS(k) or QD(k - 1, q)."""+ def fn(t):+ nm, p = famof(t)+ return int(nm == "QS" and p == k or nm == "QD" and p >> 4 == k - 1)+ return fn++ self.kk = [f(f"digit=={k}", next_digit(k), full=True) for k in range(8)]+ self.final = sum((self.fam[n] for n in S.FINAL), Expr())+ self.tables = {} # name -> head node (for snapshots)++ def code_of(t):+ """A token's own output code, or None (a text token the circuit never prints)."""+ if t < S.VOCAB:+ return token_code(t)+ p = text.printable_index(t - S.TEXT_TOKEN) if text else None+ return None if p is None else code_bits("Q", p)++ # the token's own code (read by the tied output head; cleared before the exit writes the+ # predicted one)+ self.code = [f(f"code{i}", lambda t, i=i: 0 if (cb := code_of(t)) is None+ else CODE_AMP * (2 * cb[i] - 1), full=True)+ for i in range(NFAM + NPAY)]+ if text:+ tt = S.TEXT_TOKEN+ self.is_text = f("is_text", lambda t: int(t >= tt), full=True)+ self.mu = f("mu", lambda t: int(t < S.VOCAB), full=True) # Qwen mutes on chip tokens+ self.tau = f("tau", lambda t: TAU_TEXT if t >= tt else -1, full=True)+ self.isTCO = f("isTCO", lambda t: int(t == tt + text.TCO), full=True)+ self.isTCC = f("isTCC", lambda t: int(t == tt + text.TCC), full=True)++ def tbyte(k, i):+ return lambda t: (text.row_bytes(t - tt)[k] >> i & 1) if t >= tt else 0++ # a text token's UTF-8 bytes (the first TEXT_ROW), bit by bit+ self.tb = [[f(f"tb{k}.b{i}", tbyte(k, i), full=True) for i in range(8)]+ for k in range(S.TEXT_ROW)]++ floor = 0 # no node goes below this level (see at_least)++ def ffn(self, terms, name="ffn", into=None, min_level=0):+ return super().ffn(terms, name, into, max(min_level, self.floor))++ @contextmanager+ def at_least(self, sub: int):+ """Place the nodes made inside at sublayer >= sub (to move work out of a full MLP)."""+ old, self.floor = self.floor, max(self.floor, sub - 1)+ try:+ yield+ finally:+ self.floor = old++ def fn_bits(self, z, lo: int, hi: int, f, n: int, name: str, lits=()) -> list:+ """Bits of f(z) for an integer z in [lo, hi] (AND lits), by thresholds at its flips."""+ terms = [[] for _ in range(n)]+ for b in range(n):+ if f(lo) >> b & 1:+ terms[b] += self.t_and(list(lits)) if lits else self.t_const(1)+ for theta in range(lo + 1, hi + 1):+ dd = (f(theta) >> b & 1) - (f(theta - 1) >> b & 1)+ if dd:+ terms[b] += self.t_ge(z, theta, lo, hi, dd, lits=list(lits))+ return terms++ def zeros(self, n: int, name: str) -> list:+ """Fresh dims that start at 0 (heads add onto them)."""+ return [self.token_feature(f"{name}[{i}]", lambda t: 0, full=True) for i in range(n)]++ def table(self, name, queries, keys, vheads, levels="bit", tiny=0.0):+ """A head with explicit per-column values / gates: vheads are dicts with v {col: Chan},+ z {col: Expr}, outs [(col, weight)], into [Expr] (one per out), beta, a."""+ first, rest = vheads[0], vheads[1:]+ outs = self.head(name, {s: sel(self, e) for s, e in queries.items()},+ {s: sel(self, e) for s, e in keys.items()}, first["v"], first.get("a", 0),+ first["z"], first["outs"], tiny=tiny, beta=first.get("beta"),+ into=first.get("into"),+ vx=[{"v": x["v"], "a": x.get("a", 0), "z": x["z"], "outs": x["outs"],+ "beta": x.get("beta"), "into": x.get("into")} for x in rest])+ node = self.nodes[-1]+ node.levels = levels+ self.tables[name] = node+ return outs++ def int_bits(self, z: Expr, hi: int, name: str, n: int) -> list:+ """Bits of an integer 0..hi (thresholds at every flip of each bit)."""+ out = []+ for j in range(n):+ terms = []+ for theta in range(1, hi + 1):+ d = (theta >> j & 1) - ((theta - 1) >> j & 1)+ if d:+ terms += self.t_ge(z, theta, 0, hi, d)+ out.append(self.node(terms, f"{name}.b{j}"))+ return out++ def counter(self, name, reset, step):+ """A 2^-k counter head: k = the `step` tokens since the latest `reset` token (0 there)."""+ one = self.one+ ((x,),) = thead(self, name, {0: reset, 1: one - reset}, {0: one - reset, 1: one - reset},+ [{"data": {2: reset}, "ref": one - reset,+ "a": 110 * step + 238 * reset}], tiny=2.0**-90)+ self.tables[name] = self.nodes[-1]+ self.nodes[-1].levels = "decay" # a snapshot mid-count holds 2^-k+ return x++ def thermo(self, x, reset, n, name):+ """[k > m] for m < n, from x = 2^-k (0 at reset tokens)."""+ one = self.one+ return [self.node([(1, one - reset, 7 * one - 8 * 2**m * x),+ (-1, one - reset, 6 * one - 8 * 2**m * x)], f"{name}>{m}")+ for m in range(n)]++ def split(self, z: Expr, lo: int, hi: int, name: str) -> Expr:+ """floor(z / 16) of an integer z in [lo, hi] (0 <= lo)."""+ terms = self.t_const(lo // 16) if lo // 16 else []+ for t in range(lo // 16 + 1, hi // 16 + 1):+ terms += self.t_ge(z, 16 * t, lo, hi)+ return self.node(terms, name)+++def adder_stage2(c, g, p_, d, n_nib, name):+ """Result bits of a carry-lookahead add from per-nibble generate / propagate / digit bits;+ also the carry out of the top nibble."""+ one = c.one++ def carry_sources(k):+ """Mutually exclusive literal sets, one of them true iff a carry enters nibble k."""+ return [[g[j]] + p_[j + 1:k] for j in range(k)]++ r = []+ for k in range(n_nib):+ for b in range(4):+ terms = c.t_and([d[k][b]])+ for src in carry_sources(k):+ low = d[k][:b]+ terms += c.t_and(src + low + [one - d[k][b]])+ terms += c.t_and(src + low + [d[k][b]], -1)+ r.append(c.node(terms, f"{name}.r{4 * k + b}"))+ cout = c.node([t for src in carry_sources(n_nib) for t in c.t_and(src)], f"{name}.cout")+ return r, cout+++def stage1_digits(c, sums, name, hi):+ """Generate, propagate and digit bits of nibble sums; sums[k] = [(int Expr, literals)]."""+ g, p_, d = [], [], []+ for k, forms in enumerate(sums):+ gt, pt, dt = [], [], [[] for _ in range(4)]+ for s, lits in forms:+ gt += c.t_ge(s, 16, 0, hi, lits=lits)+ pt += c.t_eq(s, 15, 0, hi, lits=lits)+ for b in range(4):+ for theta in range(1, hi + 1):+ dd = (theta % 16 >> b & 1) - ((theta - 1) % 16 >> b & 1)+ if dd:+ dt[b] += c.t_ge(s, theta, 0, hi, dd, lits=lits)+ g.append(c.node(gt, f"{name}.g{k}"))+ p_.append(c.node(pt, f"{name}.p{k}"))+ d.append([c.node(dt[b], f"{name}.d{k}.{b}") for b in range(4)])+ return g, p_, d+++def add_nibbles(c, a_bits, b_bits, n_nib, lits, name):+ """A plain carry-lookahead add of two bit vectors (n_nib nibbles), gated by literals."""+ sums = [[(from_bits(a_bits[4 * k:4 * k + 4]) + from_bits(b_bits[4 * k:4 * k + 4]), lits)]+ for k in range(n_nib)]+ g, p_, d = stage1_digits(c, sums, name, 30)+ r, _ = adder_stage2(c, g, p_, d, n_nib, name)+ return r+++def build(target: Target | None = None, text=None) -> RvCore:+ """The circuit; with ``text`` (P3) also the bridge to Qwen's text: a tool call's text goes+ into a buffer the program reads (TEXT_ADDR), ``</tool_call>`` starts a run (GO), putchar+ output is echoed as Qwen tokens, and the circuit owns the positions of a run."""+ c = RvCore(target, text)+ no_text = c.is_text if text else 0 # text tokens do not count as tokens of a record+ one, fam, pay, kk = c.one, c.fam, c.pay, c.kk+ isV, isJR = fam["V"], fam["J"] + fam["R"]+ isSS = fam["SS0"] + fam["SS1"]+ isQ = fam["QS"] + fam["QD"]+ dp = sum(kk, Expr()) # the next token is a quotient digit++ # ---------------------------------------------------------------- L1: anchor, counters+ jr_bits = [pay["J"][i] + pay["R"][i] for i in range(16)]+ (anchor,) = thead(c, "pc", {0: one}, {0: one},+ [{"data": {2 + i: jr_bits[i] for i in range(16)}, "ref": one, "beta": isJR}])+ c.tables["pc"] = c.nodes[-1]+ xn = c.counter("count", isJR, c.final - isJR)+ rec_start = c.final + fam["GO"]+ xj = c.counter("j", rec_start, one - rec_start - no_text)+ v_reset = rec_start + isQ+ xv = c.counter("vcnt", v_reset, one - v_reset - no_text)+ cnt = c.thermo(xn, isJR, S.N_MAX, "n")+ jt = c.thermo(xj, rec_start, JMAX, "j")+ vt = c.thermo(xv, v_reset, 3, "v")+ n_int = sum(cnt, Expr())+ at_max = cnt[S.N_MAX - 1] # n == 63: the refresh comes first++ def jeq(v):+ return (one if v == 0 else jt[v - 1]) - (jt[v] if v < JMAX else 0)++ j0, j1, j2, j3 = (jeq(v) for v in range(4))+ v0, v1, v2 = one - vt[0], vt[0] - vt[1], vt[1] - vt[2]++ if text: # ------------------------------------------------ P3 bridge state+ isGO, isHALT = fam["GO"], fam["HALT"]+ # running: from GO to HALT; armed: inside a tool call (its text is captured)+ ((running,),) = thead(c, "run", {0: one}, {0: one},+ [{"data": {2: isGO}, "ref": one, "beta": isGO + isHALT}])+ c.tables["run"] = c.nodes[-1]+ ((armed,),) = thead(c, "arm", {0: one}, {0: one},+ [{"data": {2: c.isTCO}, "ref": one, "beta": c.isTCO + c.isTCC}])+ c.tables["arm"] = c.nodes[-1]+ capw = c.and_([c.is_text, armed, one - c.isTCO], "cap.w") # a captured text token+ xcap = c.counter("cap", c.isTCO, capw) # its index (1, 2, ..) since <tool_call>+ capt = c.thermo(xcap, c.isTCO, S.CAP_MAX, "cap")+ caprow = {i: capt[i - 1] - (capt[i] if i < S.CAP_MAX else 0)+ for i in range(1, S.CAP_MAX + 1)}++ # ---------------------------------------------------------------- L2: PC word, link, nd+ def add_small(abits, extra, nb, name):+ """Bits 0..nb-1 of (anchor + n + extra) for n <= 63: low 6 bits by thresholds, the rest+ by the carry into bit 6."""+ L = from_bits(abits[:6]) + n_int + extra+ lo, hi = 0, 63 + 63 + extra+ bits = []+ for j in range(6):+ terms = []+ for theta in range(lo + 1, hi + 1):+ d = (theta >> j & 1) - ((theta - 1) >> j & 1)+ if d:+ terms += c.t_ge(L, theta, lo, hi, d)+ bits.append(c.node(terms, f"{name}.b{j}"))+ for j in range(6, nb):+ below = abits[6:j]+ if j < 16:+ terms = c.t_and([abits[j]])+ terms += c.t_ge(L, 64, lo, hi, 1, lits=below + [one - abits[j]])+ terms += c.t_ge(L, 64, lo, hi, -1, lits=below + [abits[j]])+ else: # the carry out of bit 15+ terms = c.t_ge(L, 64, lo, hi, 1, lits=below)+ bits.append(c.node(terms, f"{name}.b{j}"))+ return bits++ w = add_small(anchor, 0, 16, "w")+ w1 = add_small(anchor, 1, 17, "w+1") # the link: PC + 4 = ROM_BASE + 4 * (w + 1)+ assert S.ROM_BASE == 1 << 18 and S.ROM_WORDS == 1 << 16+ pc_bits = [Expr(), Expr()] + w + [one] + [Expr()] * 13 # PC = 0x40000 + 4 w+ link = [Expr()] * 2 + w1[:16] + [one - w1[16], w1[16]] + [Expr()] * 12+ # digits so far: nd = j - 1 - v inside a DIV record (QS at j = 0, then the digits)+ nd_int = sum(jt, Expr()) - 1 - sum(vt, Expr())+ nd_is = {v: c.node(c.t_eq(nd_int, v, -4, JMAX), f"nd=={v}") for v in range(1, 9)}++ # ---------------------------------------------------------------- L3-4: ROM fetch+ # word w: key head w[15:11] (two sublayers of 16), value head w[10], row pair (p, p + 64)+ # with p = w[9:4], half w[3] (row p: columns 2..33, row p + 64: 34..65), word w[2:0], 4 byte+ # columns (b0..b3 of the ROM word)+ fdims = c.zeros(4, "rom.b")+ for h in range(S.ROM_WORDS >> 11):+ hl = lits_of(w[11:16], h)+ q = {}+ for p in range(64):+ e = andpre(lits_of(w[4:10], p) + hl)+ q[p], q[p + 64] = e, e+ vhs = []+ for x in range(2):+ z, outs, into = {}, [], []+ for half in range(2):+ for k in range(8):+ ze = K * andpre(lits_of([w[3]], half) + lits_of(w[0:3], k)+ + lits_of([w[10]], x))+ for b in range(4):+ col = 2 + 32 * half + 4 * k + b+ z[col] = ze+ outs.append((col, OUT_W))+ into.append(fdims[b])+ vhs.append({"v": {}, "z": z, "outs": outs, "into": into})+ c.table(f"rom{h}", q, {}, vhs, levels="byte")+ B0, B1, B2, B3 = (256 * d for d in fdims) # the word's bytes++ # ---------------------------------------------------------------- L4: decode+ def uop_terms(u, lits=()):+ v = RF.U[u]+ return c.t_eq(B0, v, 0, 255, lits=list(lits)) + c.t_eq(B0, v + 128, 0, 255, lits=list(lits))++ uops = {u: c.node(uop_terms(u), f"uop={u}") for u in RF.UOPS}++ def any_of(*names):+ return sum((uops[n] for n in names), Expr())++ s0 = c.node(c.t_ge(B0, 128, 0, 255), "rom.S0")+ bits1, bits2, bits3 = (c.int_bits(Bx, 255, f"rom.b{i}", 8)+ for i, Bx in ((1, B1), (2, B2), (3, B3)))+ spare = [s0] + bits1[0:3] + bits2[0:3] + bits3[0:3] # S0..S9+ RDb, RS1b, RS2b = bits1[3:8], bits2[3:8], bits3[3:8]+ rd = {r: c.node(c.t_in(B1, 8 * r, 8 * r + 8, 0, 255), f"rd={r}") for r in range(1, 32)}+ rs1 = {r: c.node(c.t_in(B2, 8 * r, 8 * r + 8, 0, 255), f"rs1={r}") for r in range(1, 32)}+ rs2 = {r: c.node(c.t_in(B3, 8 * r, 8 * r + 8, 0, 255), f"rs2={r}") for r in range(1, 32)}+ rdw_any = sum(rd.values(), Expr())+ imm = RS2b + spare[0:7] + [spare[6]] * 20 # I-type (and stores): sign-extended imm12+ boff = RDb + spare[0:6] + [spare[5]] * 5 # branch: word offset, sign-extended to 16 bits+ bbad, jbad = spare[6], spare[6]+ jtgt = RS1b + RS2b + spare[0:6] # JAL: target word index+ upper = RS1b + RS2b + spare[0:10] # LUI / AUIPC: value bits 12..31+ jl = [Expr() if not link[i].terms else+ c.node(uop_terms("JAL", [link[i]]) + uop_terms("JALR", [link[i]]), f"link.b{i}")+ for i in range(32)]+ divc = any_of(*RF.DIV_OPS)+ signed = any_of("DIV", "REM")+ branches = any_of(*RF.BRANCH_OPS)+ stores = any_of("SB", "SH", "SW")+ bprd, bget = uops["BPRD"], uops["BGET"]+ bnrow = any_of(*RF.BN_ROW)+ k7 = RS2b + spare[0:2] # BPRD's digit index++ # ---------------------------------------------------------------- L5: register file+ P = pay["V"]+ isF1, isF2 = fam["F1"], fam["F2"]+ vcols = {} # bit -> list of value Chans (the commit's sources; exactly one is live)+ for i in range(32):+ cs = []+ if i < 15:+ cs.append(Chan(pay["F0"][i]))+ cs.append(Chan(-1 * P[i] + isF1 + isV, (1, -1, 0, 0))) # V at t-1 when t is F1+ cs.append(Chan(-1 * P[i] + isF2 + isV, (1, 0, -1, 0))) # V at t-2 when t is F2+ elif i < 30:+ cs.append(Chan(pay["F1"][i - 15] + pay["F0"][14]))+ cs.append(Chan(-1 * P[i - 15] + isF2 + isV, (1, -1, 0, 0)))+ else:+ cs.append(Chan(pay["F2"][i - 30] + pay["F1"][14] + pay["F0"][14]))+ if jl[i].terms:+ cs.append(Chan(jl[i], (0, 1, 0, 0))) # the link, computed at t-1+ vcols[i] = cs+ reg_beta = fam["F0"] + fam["F1"] + fam["F2"] + fam["J"]+ nw = stores + branches # the RD slot holds a store's data register / a branch offset+ reg_keys = {r: Chan(rd[r] - nw, (0, 1, 0, 0)) for r in range(1, 32)}+ ports = []+ for pname, rq in (("rs1", rs1), ("rs2", rs2), ("rd", rd)):+ dims = c.zeros(32, f"x.{pname}")+ v, z, outs, into, col = {}, {}, [], [], 2+ c.reg_cols = {}+ for i in range(32):+ for ch in vcols[i]:+ v[col] = chan(ch, A_BIT)+ z[col] = K * one+ outs.append((col, OUT_W))+ into.append(dims[i])+ c.reg_cols.setdefault(i, []).append(col)+ col += 1+ for rc in REF_COLS:+ v[rc] = chan(one, A_REF)+ c.table(f"regs.{pname}", rq, reg_keys,+ [{"v": v, "z": z, "outs": outs, "into": into, "beta": reg_beta}])+ ports.append(dims)+ A, B, C = ports++ # the quotient heads: value head d (4 key heads x 2) holds the record's d-th digit in one row,+ # written at its QD token (beta: QD and j == d + 2), in one copy per alignment nd' > d+ # (nibble nd' - 1 - d); one-hot reads gated by the current nd give the digits so far+ # right-aligned (the whole |Q| after the last). (One-hot reads only: a multi-hot read of+ # often-rewritten rows picks up fp32 residues in fla's chunk kernel.)+ qal = c.zeros(32, "quo")+ for h in range(4):+ vhs = []+ for d in (2 * h, 2 * h + 1):+ v, z, outs, into, col = {}, {}, [], [], 2+ for ndp in range(d + 1, 9):+ for b in range(4):+ v[col] = chan(Chan(pay["QD"][b]), A_BIT)+ z[col] = K * nd_is[ndp]+ outs.append((col, OUT_W))+ into.append(qal[4 * (ndp - 1 - d) + b])+ col += 1+ for rc in REF_COLS:+ v[rc] = chan(one, A_REF)+ vhs.append({"v": v, "z": z, "outs": outs, "into": into,+ "beta": fam["QD"] + jeq(d + 2) - 1})+ c.table(f"quo{h}", {0: one}, {0: fam["QD"]}, vhs)++ # ---------------------------------------------------------------- L5-6: the digit file+ # 8 registers of 128 digits; read port 1 at row j (a row op's digit i is its record index),+ # read port 2 at row j + off (BMAC / BMSB / BLD / BGET) or k (BPRD); every write (a BD+ # token: the digit just computed) goes to both ports' copies at port 2's row of t-1+ def nib(bits, k, inv=False):+ x = from_bits(bits[4 * k:4 * k + 4])+ return 15 - x if inv else x++ jint = sum(jt, Expr())+ jbin = c.int_bits(jint, JMAX, "jbin", 7)+ n_rows = from_bits(B[7:13])+ off = from_bits(B[13:20])+ bnq = bnrow + bget+ q2 = [c.node(c.t_eq(jint + off, r, 0, JMAX + 127, lits=[bnq])+ + c.t_eq(from_bits(k7), r, 0, 127, lits=[bprd]), f"bn.q{r}") for r in range(128)]+ lt_n = c.node(c.t_lt(jint - n_rows, 0, -63, JMAX), "bn.lt")+ eq_n = c.node(c.t_eq(jint - n_rows, 0, -63, JMAX), "bn.eq")+ mcl = [c.node(t, f"bn.m{b}") for b, t in+ enumerate(c.fn_bits(from_bits(B[20:24]), 0, 15, lambda v: min(v, 9), 4, "bn.m"))]+ sel_d = [c.node(c.t_and(lits_of(B[0:3], R) + [1 - bprd])+ + c.t_and(lits_of(spare[7:10], R) + [bprd]), f"bn.dd{R}") for R in range(8)]+ sel_s = [c.and_(lits_of(B[3:6], R), f"bn.ds{R}") for R in range(8)]+ ovw = B[6]+ # BLD's address: rs1 - 1 - j (the digit chars end at rs1 - 1); BSCAN's: rs1 + j+ bscan = uops["BSCAN"]+ notj, jj = [1 - b for b in jbin] + [one] * 9, jbin + [Expr()] * 9+ ea_sums = [[(nib(A, k) + nib(notj, k), [uops["BLD"]]), (nib(A, k) + nib(jj, k), [bscan])]+ for k in range(4)]+ eg, ep, ed = stage1_digits(c, ea_sums, "bn.ea", 30)+ eab, _ = adder_stage2(c, eg, ep, ed, 4, "bn.ea")+ dig = {1: {}, 2: {}}+ for port in (1, 2):+ for h in range(4):+ vhs = []+ for x in range(2):+ R = 2 * h + x+ dims = c.zeros(4, f"bn{port}.r{R}")+ dig[port][R] = dims+ v = {2 + b: chan(Chan(pay["BD"][b]), A_BIT) for b in range(4)}+ for rc in REF_COLS:+ v[rc] = chan(one, A_REF)+ vhs.append({"v": v, "z": {2 + b: K * one for b in range(4)},+ "outs": [(2 + b, OUT_W) for b in range(4)], "into": dims,+ "beta": fam["BD"] + sel_d[R] - 1})+ q = {r: jeq(r) for r in range(64)} if port == 1 else dict(enumerate(q2))+ c.table(f"bn{port}.{h}", q, {r: Chan(q2[r], (0, 1, 0, 0)) for r in range(128)}, vhs)+ sdig = [c.node([t for R in range(8) for t in c.t_and([sel_s[R], dig[1][R][b]])], f"bn.s{b}")+ for b in range(4)]+ ddig = [c.node([t for R in range(8) for t in c.t_and([sel_d[R], dig[2][R][b]])], f"bn.d{b}")+ for b in range(4)]+ bprod = c.node([t for v in range(1, 16) for t in c.t_and(lits_of(sdig, v), v,+ a=from_bits(mcl))], "bn.prod")++ # ---------------------------------------------------------------- L5-6: adder+ md = {"ADD": any_of("ADD"),+ "ADDI": any_of("ADDI", "LB", "LH", "LW", "LBU", "LHU", "SB", "SH", "SW", "JALR"),+ "SUB": any_of("SUB", "SLT", "SLTU", *RF.BRANCH_OPS),+ "SUBI": any_of("SLTI", "SLTIU")}+ bsrc = {"ADD": (B, False, 0), "ADDI": (imm, False, 0), "SUB": (B, True, 1),+ "SUBI": (imm, True, 1)}+ sums = [[(nib(A, k) + nib(bsrc[m][0], k, bsrc[m][1]) + (bsrc[m][2] if k == 0 else 0), [lit])+ for m, lit in md.items()] for k in range(8)]+ g, p_, d = stage1_digits(c, sums, "add", 31)+ r, c32 = adder_stage2(c, g, p_, d, 8, "add")++ # ---------------------------------------------------------------- L6-7: logic, shifter+ floor_l6 = c.at_least(12)+ floor_l6.__enter__()+ logic = []+ for i in range(32):+ terms = []+ for op, bb in (("AND", B), ("ANDI", imm)):+ terms += c.t_and([A[i], bb[i], uops[op]])+ for op, bb in (("OR", B), ("ORI", imm)):+ terms += c.t_and([uops[op]]) + c.t_and([1 - A[i], 1 - bb[i], uops[op]], -1)+ for op, bb in (("XOR", B), ("XORI", imm)):+ terms += c.t_and([A[i], 1 - bb[i], uops[op]]) + c.t_and([1 - A[i], bb[i], uops[op]])+ logic.append(c.node(terms, f"logic.b{i}"))++ sh_ops = {"L": ("SLL", "SLLI"), "R": ("SRL", "SRLI"), "A": ("SRA", "SRAI")}++ def src_bit(dirn, i):+ if 0 <= i < 32:+ return A[i]+ return A[31] if dirn == "A" and i >= 32 else None++ y = {}+ for dirn, (rop, iop) in sh_ops.items():+ y[dirn] = []+ for i in range(32):+ terms = []+ for op, sb_ in ((rop, B), (iop, imm)):+ for kk_ in range(8):+ j = i - 4 * kk_ if dirn == "L" else i + 4 * kk_+ bit = src_bit(dirn, j)+ if bit is None:+ continue+ terms += c.t_and([uops[op], bit] + lits_of(sb_[2:5], kk_))+ if dirn == "L": # Zba: A << 1..3 (stage 1 passes A)+ terms += c.t_and([any_of(*RF.ZBA), A[i]])+ y[dirn].append(c.node(terms, f"sh{dirn}1.b{i}"))+ shift = []+ for i in range(32):+ terms = []+ for dirn, (rop, iop) in sh_ops.items():+ yy = y[dirn]+ for op, sb_ in ((rop, B), (iop, imm)):+ for kk_ in range(4):+ j = i - kk_ if dirn == "L" else i + kk_+ bit = yy[j] if 0 <= j < 32 else (yy[31] if dirn == "A" and j >= 32 else None)+ if bit is None:+ continue+ terms += c.t_and([uops[op], bit] + lits_of(sb_[0:2], kk_))+ for op, k in RF.ZBA.items():+ if i >= k:+ terms += c.t_and([uops[op], y["L"][i - k]])+ shift.append(c.node(terms, f"shift.b{i}"))+ floor_l6.__exit__(None, None, None)++ # ---------------------------------------------------------------- L5-9: multiplier+ # 64-bit product of nibbles: p_ij = a_i b_j <= 225 (stage 1; a = rs1, or the quotient for+ # REM), split into nibbles (stage 2); column k sums lo_ij (i + j = k), hi_ij (i + j = k - 1)+ # and, for MULH / MULHSU, the two's-complement corrections -a31 * b and -b31 * a of the high+ # word (as ~x + 1); a column stays <= 255, so it splits into a digit and a carry (stage 3);+ # then a carry-lookahead add of digits and carries (stages 4-5). REM: R = A - sigma Q B mod+ # 2^32 (sigma = -1 when the quotient is negative) in the low 8 columns.+ mh_a = any_of("MULH", "MULHSU") # rs1 signed+ mh_b = uops["MULH"] # rs2 signed+ mul_any = any_of(*RF.MUL_OPS)+ remc = any_of("REM", "REMU")+ prod = {}+ for i in range(8):+ for j in range(8):+ terms = []+ for v in range(1, 16):+ terms += c.t_and(lits_of(A[4 * i:4 * i + 4], v) + [1 - divc], v, a=nib(B, j))+ terms += c.t_and(lits_of(qal[4 * i:4 * i + 4], v) + [divc], v, a=nib(B, j))+ prod[i, j] = c.node(terms, f"mul.p{i}{j}")+ corr_a = [c.node(c.t_and([A[31], mh_a], a=nib(B, j, inv=True)), f"mul.ca{j}") for j in range(8)]+ corr_b = [c.node(c.t_and([B[31], mh_b], a=nib(A, j, inv=True)), f"mul.cb{j}") for j in range(8)]+ plus1 = c.node(c.t_and([A[31], mh_a]) + c.t_and([B[31], mh_b]), "mul.plus1")+ lo, hi = {}, {}+ for (i, j), pij in prod.items():+ hi[i, j] = c.node([t for m in range(1, 15) for t in c.t_ge(pij, 16 * m, 0, 225)],+ f"mul.hi{i}{j}")+ lo[i, j] = pij - 16 * hi[i, j]+ # DIV signs (L5): sa / sb (signed operand negative), sigma (the quotient is negative)+ sa = c.and_([signed, A[31]], "div.sa")+ sbn = c.and_([signed, B[31]], "div.sb")+ bz = c.and_([1 - b for b in B], "div.bz")+ sigma = c.node(c.t_and([signed, A[31], 1 - B[31]])+ + c.t_and([signed, A[31]] + [1 - b for b in B], -1)+ + c.t_and([signed, 1 - A[31], B[31]]), "div.sigma")+ cols = []+ rem_p, rem_m = [], [] # REM columns, sigma = 1 (A + QB) and sigma = 0 (A - QB, offset)+ for k in range(16):+ s = sum((lo[i, k - i] for i in range(8) if 0 <= k - i < 8), Expr())+ s += sum((hi[i, k - 1 - i] for i in range(8) if 0 <= k - 1 - i < 8), Expr())+ if k < 8:+ nlo = sum(1 for i in range(8) if 0 <= k - i < 8)+ nhi = sum(1 for i in range(8) if 0 <= k - 1 - i < 8)+ rem_p.append((nib(A, k) + s, 15 + 15 * nlo + 14 * nhi))+ rem_m.append((nib(A, k) - s, (15, 15 * nlo + 14 * nhi)))+ if k >= 8:+ s += corr_a[k - 8] + corr_b[k - 8]+ if k == 8:+ s += plus1+ cols.append(s)+ rm_off = col_offsets([b for _, b in rem_m])+ # each mode's columns split into digit + carry at every position (so every stage-4 sum is in+ # range), and stage 4 picks the instruction's mode+ modes = [("mul", cols, [0] * 16, [255] * 16, [mul_any]),+ ("remp", [e for e, _ in rem_p], [0] * 8, [hp for _, hp in rem_p], [remc, sigma]),+ ("remm", [e + o for (e, _), o in zip(rem_m, rm_off)], [o - b[1] for (_, b), o in+ zip(rem_m, rm_off)],+ [o + 15 for o in rm_off], [remc, 1 - sigma])]+ tsum = [[] for _ in range(16)]+ for mname, mcols, los, his, lits in modes:+ car = [c.split(e, lo_, hi_, f"mul.{mname}.c{k}")+ for k, (e, lo_, hi_) in enumerate(zip(mcols, los, his))]+ for k, e in enumerate(mcols):+ tsum[k].append((e - 16 * car[k] + (car[k - 1] if k else 0), lits))+ mg, mp, mdg = stage1_digits(c, tsum, "mul", 30)+ mres, _ = adder_stage2(c, mg, mp, mdg, 16, "mul")+ mul_lo, mul_hi = mres[:32], mres[32:]++ # store data for SW (0 on V tokens, so an SB/SH record's last V does not leak it)+ sd = [c.and_([C[i], one - isV], f"sd.b{i}") for i in range(32)]++ # ---------------------------------------------------------------- L5-10: DIV digits+ # |B| = B' + sb with B' = B xor sb (as 8 nibbles, the +1 folded into nibble 0: 0..16)+ bp = [c.node(c.t_and([B[i]]) + c.t_and([B[i], signed, B[31]], -1)+ + c.t_and([1 - B[i], signed, B[31]]), f"div.bp{i}") for i in range(32)]+ bm = [nib(bp, 0) + sbn] + [nib(bp, j) for j in range(1, 8)]+ nzA = [c.node(c.t_const(1) + c.t_and([1 - b for b in A[4 * i:]], -1), f"div.nza{i}")+ for i in range(8)]+ # |A| >> 4s for s = 7 - k: (A xor sa) >> 4s, plus 1 when sa and A's low 4s bits are 0+ ahi = []+ for i in range(32):+ terms = []+ for k in range(8):+ j = i + 4 * (7 - k)+ if j < 32:+ terms += c.t_and([kk[k], A[j], 1 - sa]) + c.t_and([kk[k], 1 - A[j], sa])+ ahi.append(c.node(terms, f"div.ahi{i}"))+ ca = c.node([t for k in range(8)+ for t in c.t_and([kk[k], sa] + [1 - A[j] for j in range(4 * (7 - k))])],+ "div.ca")+ nzB = [c.node(c.t_const(1) + c.t_and([1 - b for b in bp[4 * i:]], -1), f"div.nzb{i}")+ for i in range(8)]+ # k0 = 7 - dA + dB clamped to 0..8 (0 when B = 0): the QS payload+ kv = 7 - sum(nzA, Expr()) + sum(nzB, Expr())+ k0b = []+ for bit in range(4):+ terms = []+ for v in range(1, 8):+ if v >> bit & 1:+ terms += c.t_eq(kv, v, -1, 15, lits=[1 - bz])+ if bit == 3:+ terms += c.t_ge(kv, 8, -1, 15, lits=[1 - bz])+ k0b.append(c.node(terms, f"div.k0.b{bit}"))+ # products Q (digits so far, at most 7 nibbles) x |B| nibbles, and m |B| for m = 1..15+ prod2, hi2, lo2 = {}, {}, {}+ for a in range(7):+ for b in range(8):+ top = 15 * (16 if b == 0 else 15)+ terms = []+ for v in range(1, 16):+ terms += c.t_and(lits_of(qal[4 * a:4 * a + 4], v), v, a=bm[b])+ prod2[a, b] = c.node(terms, f"div.p{a}{b}")+ hi2[a, b] = c.split(prod2[a, b], 0, top, f"div.hi{a}{b}")+ lo2[a, b] = prod2[a, b] - 16 * hi2[a, b]+ him, lom = {}, {}+ for m in range(1, 16):+ for b in range(8):+ top = 16 if b == 0 else 15+ terms = []+ for x in range(1, top + 1):+ dd = m * x // 16 - m * (x - 1) // 16+ if dd:+ terms += c.t_ge(bm[b], x, 0, top, dd)+ him[m, b] = c.node(terms, f"div.mh{m}.{b}")+ lom[m, b] = m * bm[b] - 16 * him[m, b]+ # the compare columns of D - m |B| (mod 16^10): |A|>>4s - 16 Q |B| - m |B|+ bounds, base = [], []+ for i in range(10):+ e = (nib(ahi, i) if i < 8 else Expr()) + (ca if i == 0 else 0)+ pos = (15 if i < 8 else 0) + (1 if i == 0 else 0)+ neg = 0+ for a in range(7):+ b = i - 1 - a+ if 0 <= b < 8:+ e -= lo2[a, b]+ neg += 15+ b = i - 2 - a+ if 0 <= b < 8:+ e -= hi2[a, b]+ neg += 15 if b == 0 else 14+ base.append(e)+ neg += (15 if i < 8 else 0) + ((15 if i == 1 else 14) if 1 <= i <= 8 else 0)+ bounds.append((pos, neg))+ coff = col_offsets(bounds)+ cmp_ge = [] # C_m = [D >= m |B|]+ for m in range(1, 16):+ dig, car = [], []+ for i in range(10):+ e = base[i] + coff[i]+ if i < 8:+ e -= lom[m, i]+ if 1 <= i <= 8:+ e -= him[m, i - 1]+ lo_, hi_ = coff[i] - bounds[i][1], coff[i] + bounds[i][0]+ cc = c.split(e, lo_, hi_, f"div.c{m}.{i}")+ car.append((cc, hi_ // 16))+ dig.append(e - 16 * cc)+ g_, p2_ = [], []+ for i in range(9):+ s = dig[i] + (car[i - 1][0] if i else 0)+ shi = 15 + (car[i - 1][1] if i else 0)+ g_.append(c.node(c.t_ge(s, 16, 0, shi), f"div.g{m}.{i}"))+ p2_.append(c.node(c.t_eq(s, 15, 0, shi), f"div.pp{m}.{i}"))+ s9 = dig[9] + car[8][0]+ shi = 15 + car[8][1]+ u = []+ for inc in (0, 1):+ terms = []+ for theta in range(1, shi + 1):+ dd = int((theta + inc) % 16 >= 8) - int((theta - 1 + inc) % 16 >= 8)+ if dd:+ terms += c.t_ge(s9, theta, 0, shi, dd)+ u.append(c.node(terms, f"div.u{inc}.{m}"))+ # bit 39 of the sum: u1 if a carry enters nibble 9, else u0+ terms = c.t_and([u[0]])+ for j in range(9):+ src = [g_[j]] + p2_[j + 1:9]+ terms += c.t_and(src + [u[1]]) + c.t_and(src + [u[0]], -1)+ cmp_ge.append(one - c.node(terms, f"div.neg{m}"))+ qbits = [sum(((m >> bit & 1) - ((m - 1) >> bit & 1)) * cmp_ge[m - 1] for m in range(1, 16))+ for bit in range(4)]+ kbits = [sum((kk[k] for k in range(8) if k >> bit & 1), Expr()) for bit in range(3)]+ # the signed quotient: -Q = ~Q + 1 when sigma+ qs = []+ for i in range(32):+ zero_below = [1 - qal[j] for j in range(i)]+ qs.append(c.node(c.t_and([1 - sigma, qal[i]]) + c.t_and([sigma, qal[i]] + zero_below)+ + c.t_and([sigma, 1 - qal[i]])+ + c.t_and([sigma, 1 - qal[i]] + zero_below, -1), f"div.qs{i}"))++ # ---------------------------------------------------------------- L7-8: Zba, branch target+ zres = add_nibbles(c, shift, B, 8, [any_of(*RF.ZBA)], "zba")+ with c.at_least(14):+ btgt = add_nibbles(c, w, boff, 4, [branches], "btgt")++ # ---------------------------------------------------------------- L7-8: RAM+ # word index wi = EA[14:2] (RAM at 0): key head = (wi[12], wi[3:0]) = (EA[14], EA[5:2]),+ # value head wi[4] = EA[6], row wi[11:5] = EA[13:7]+ EA = [r[i] + eab[i] if i < 15 else r[i] for i in range(32)] # (BLD: r = 0; else eab = 0)+ is_store = stores+ mem = c.zeros(32, "mem")+ n_ram_heads = S.RAM_WORDS >> 8+ for h in range(n_ram_heads):+ hl = lits_of(EA[2:6], h & 15) + lits_of(EA[14:14 + (n_ram_heads >> 4).bit_length() - 1],+ h >> 4)+ keys, q = {}, {}+ for row in range(128):+ rl = lits_of(EA[7:14], row)+ keys[row] = Chan(andpre(rl + hl + [is_store]), (0, 1, 0, 0))+ q[row] = andpre(rl + hl)+ vhs = []+ for x in range(2):+ v, z, outs, into = {}, {}, [], []+ for i in range(32): # block A: SW data from t-1+ v[2 + i] = chan(Chan(sd[i], (0, 1, 0, 0)), A_BIT)+ for i in range(32): # block B: the SB/SH word from the record's V tokens+ lag = 3 - i // 15 if i < 30 else 1+ taps = [0, 0, 0, 0]+ taps[0], taps[lag] = 1, -1+ v[34 + i] = chan(Chan(-1 * P[i % 15 if i < 30 else i - 30] + isSS + isV,+ tuple(taps)), A_BIT)+ for rc in REF_COLS:+ v[rc] = chan(one, A_REF)+ for col in range(2, 66):+ z[col] = K * (EA[6] if x else 1 - EA[6])+ outs.append((col, OUT_W))+ into.append(mem[(col - 2) % 32])+ vhs.append({"v": v, "z": z, "outs": outs, "into": into,+ "beta": fam[f"ST{x}"] + fam[f"SS{x}"]})+ c.table(f"ram{h}", q, keys, vhs)++ # ---------------------------------------------------------------- L9: DROM+ # word index wi = EA[16:2] (DROM at 0x20000): key head wi[14:11] = EA[16:13], value head+ # wi[10] = EA[12], row pair (p, p + 64) with p = EA[11:6], half EA[5], word EA[4:2]+ ddims = c.zeros(4, "drom.b")+ for h in range(S.DROM_WORDS >> 11):+ hl = lits_of(EA[13:17], h)+ q = {}+ for p in range(64):+ e = andpre(lits_of(EA[6:12], p) + hl)+ q[p], q[p + 64] = e, e+ vhs = []+ for x in range(2):+ z, outs, into = {}, [], []+ for half in range(2):+ for k in range(8):+ ze = K * andpre(lits_of([EA[5]], half) + lits_of(EA[2:5], k)+ + lits_of([EA[12]], x))+ for b in range(4):+ col = 2 + 32 * half + 4 * k + b+ z[col] = ze+ outs.append((col, OUT_W))+ into.append(ddims[b])+ vhs.append({"v": {}, "z": z, "outs": outs, "into": into})+ c.table(f"drom{h}", q, {}, vhs, levels="byte")+ dbits = [b for k in range(4) for b in c.int_bits(256 * ddims[k], 255, f"drom.b{k}", 8)]++ # ---------------------------------------------------------------- P3: the text buffer+ # row i = the i-th captured token's bytes (2 value heads of 8 bytes), read as memory at+ # TEXT_ADDR + 16 i: row EA[10:4], value head EA[3], word EA[2]+ if text:+ tmem = c.zeros(32, "tmem")+ tsel = [EA[15]] + [1 - EA[b] for b in range(11, 15)]+ q = {i: andpre(lits_of(EA[4:11], i) + tsel) for i in caprow}+ keys = {i: Chan(e) for i, e in caprow.items()}+ vhs = []+ for x in range(2):+ v, z, outs, into = {}, {}, [], []+ for k in range(8):+ for i in range(8):+ col = 2 + 8 * k + i+ v[col] = chan(c.tb[8 * x + k][i], A_BIT)+ z[col] = K * andpre(lits_of([EA[3]], x) + lits_of([EA[2]], k // 4))+ outs.append((col, OUT_W))+ into.append(tmem[8 * (k % 4) + i])+ for rc in REF_COLS:+ v[rc] = chan(one, A_REF)+ vhs.append({"v": v, "z": z, "outs": outs, "into": into, "beta": capw,+ "a": 238 * c.isTCO}) # cleared at <tool_call>+ c.table("tbuf", q, keys, vhs)++ # ---------------------------------------------------------------- loads, merge, compares+ lane = EA[0:2]+ in_ram = c.and_([1 - b for b in EA[15:32]], "in_ram")+ in_drom = c.and_([1 - b for b in EA[18:32]] + [EA[17]], "in_drom")+ in_text = (c.and_(tsel + [1 - b for b in EA[16:32]], "in_text") if text else None)+ ld = []+ for i in range(32):+ terms = c.t_and([uops["LW"], mem[i], in_ram] + lits_of(lane, 0))+ for k in range(4):+ if i < 8:+ terms += c.t_and([uops["LBU"], mem[8 * k + i], in_ram] + lits_of(lane, k))+ terms += c.t_and([uops["LB"], mem[8 * k + i], in_ram] + lits_of(lane, k))+ else:+ terms += c.t_and([uops["LB"], mem[8 * k + 7], in_ram] + lits_of(lane, k))+ for hh in range(2):+ if i < 16:+ terms += c.t_and([uops["LHU"], mem[16 * hh + i], in_ram] + lits_of(lane, 2 * hh))+ terms += c.t_and([uops["LH"], mem[16 * hh + i], in_ram] + lits_of(lane, 2 * hh))+ else:+ terms += c.t_and([uops["LH"], mem[16 * hh + 15], in_ram] + lits_of(lane, 2 * hh))+ ld.append(c.node(terms, f"load.b{i}"))+ bnib = [c.node([t for k in range(4) for t in c.t_and([uops["BLD"], mem[8 * k + b], in_ram]+ + lits_of(lane, k))], f"bn.nib{b}")+ for b in range(4)]+ isdig = c.node([t for k in range(4) for t in+ c.t_and([bscan] + lits_of(lane, k) + lits_of(mem[8 * k + 3:8 * k + 8], 6))+ + c.t_and([bscan] + lits_of(lane, k) + lits_of(mem[8 * k + 1:8 * k + 8], 28))],+ "bn.isdig")+ cin = from_bits(pay["BD"][4:8])+ dv, pv = from_bits(ddig), bprod+ forms = [(dv + pv + cin, 165, [uops["BMAC"], 1 - ovw], lambda v: (v % 10, v // 10)),+ (pv + cin, 150, [uops["BMAC"], ovw], lambda v: (v % 10, v // 10)),+ (150 + dv - pv - cin, 165, [uops["BMSB"]],+ lambda v: (v % 10, max(15 - v // 10, 0))),+ (from_bits(bnib), 15, [uops["BLD"]], lambda v: (min(v, 9), 0))]+ with c.at_least(18):+ dt, ct = [[] for _ in range(4)], [[] for _ in range(4)]+ for z, hi_, lits, f in forms:+ for b, t in enumerate(c.fn_bits(z, 0, hi_, lambda v: f(v)[0], 4, "bn", lits)):+ dt[b] += t+ for b, t in enumerate(c.fn_bits(z, 0, hi_, lambda v: f(v)[1], 4, "bn", lits)):+ ct[b] += t+ bd_dig = [c.node(dt[b], f"bn.dig{b}") for b in range(4)]+ bd_car = [c.node(ct[b], f"bn.car{b}") for b in range(4)]+ merged = []+ for i in range(32):+ terms = []+ for k in range(4):+ inl = 8 * k <= i < 8 * k + 8+ src = C[i - 8 * k] if inl else mem[i]+ terms += c.t_and([uops["SB"], src] + lits_of(lane, k))+ for hh in range(2):+ inl = 16 * hh <= i < 16 * hh + 16+ src = C[i - 16 * hh] if inl else mem[i]+ terms += c.t_and([uops["SH"], src] + lits_of(lane, 2 * hh))+ merged.append(c.node(terms, f"merge.b{i}"))+ b31 = {"SUB": B[31], "SUBI": imm[31]}+ slt = c.node([t for m in b31 for t in c.t_and([md[m], A[31], 1 - b31[m]])+ + c.t_and([md[m], A[31], b31[m], r[31]])+ + c.t_and([md[m], 1 - A[31], 1 - b31[m], r[31]])], "slt")+ sltu = c.node(c.t_and([md["SUB"] + md["SUBI"], 1 - c32]), "sltu")+ eq = c.and_([1 - b for b in r], "eq")++ # ---------------------------------------------------------------- faults, ecalls+ loads = any_of(*RF.LOAD_OPS)+ halfs, words = any_of("LH", "LHU"), uops["LW"]+ s_halfs, s_words = uops["SH"], uops["SW"]+ a7_is = {v: c.and_(lits_of(A, v), f"a7=={v}") for v in (1, 93)}+ ecall = uops["ECALL"]+ mis_ld = c.node(c.t_and([halfs, EA[0]]) + c.t_and([words, EA[0]])+ + c.t_and([words, 1 - EA[0], EA[1]]), "fault.misld")+ mis_st = c.node(c.t_and([s_halfs, EA[0]]) + c.t_and([s_words, EA[0]])+ + c.t_and([s_words, 1 - EA[0], EA[1]]), "fault.misst")+ oor_ld = c.node(c.t_and([loads, 1 - mis_ld]) + c.t_and([loads, 1 - mis_ld, in_ram], -1)+ + c.t_and([loads, 1 - mis_ld, in_drom], -1)+ + (c.t_and([loads, 1 - mis_ld, in_text], -1) if text else []), "fault.oorld")+ oor_st = c.node(c.t_and([is_store, 1 - mis_st])+ + c.t_and([is_store, 1 - mis_st, in_ram], -1), "fault.oorst")+ jalr = uops["JALR"]+ # a bad JALR target (EA & ~1): misaligned, or outside the ROM (EA[31:19] = 0, EA[18] = 1)+ rom_ok = [1 - b for b in EA[19:32]] + [EA[18]]+ fc6_terms = (c.t_and([jalr, EA[1]]) + c.t_and([jalr, 1 - EA[1]])+ + c.t_and([jalr, 1 - EA[1]] + rom_ok, -1) + c.t_and([uops["JAL"], jbad]))+ fc = {0: c.and_([uops["ILLEGAL"]], "fault.0"), 1: c.and_([uops["EBREAK"]], "fault.1"),+ 2: mis_ld, 3: mis_st, 4: oor_ld, 5: oor_st,+ 7: c.node(c.t_and([ecall]) + c.t_and([ecall, a7_is[1]], -1)+ + c.t_and([ecall, a7_is[93]], -1), "fault.7")}++ # ---------------------------------------------------------------- rd value, size, branch+ addish = any_of("ADD", "ADDI", "SUB")+ sltish, sltuish = any_of("SLT", "SLTI"), any_of("SLTU", "SLTIU")+ logicops = any_of("AND", "ANDI", "OR", "ORI", "XOR", "XORI")+ shiftops = any_of("SLL", "SLLI", "SRL", "SRLI", "SRA", "SRAI")+ mulhs = any_of("MULH", "MULHSU", "MULHU")+ zero = Expr()+ misa = [one if S.MISA >> i & 1 else zero for i in range(32)]+ srcs = [([addish], r), ([uops["LUI"]], [zero] * 12 + upper),+ ([uops["AUIPC"]], [zero] * 2 + w[:10] + upper), ([uops["MISA"]], misa),+ ([logicops], logic), ([shiftops], shift), ([loads, in_ram], ld),+ ([uops["MUL"]], mul_lo), ([mulhs], mul_hi), ([any_of(*RF.ZBA)], zres),+ ([uops["LW"], in_drom], dbits)]+ for k in range(4): # DROM bytes and halves+ lk = lits_of(lane, k)+ byte = dbits[8 * k:8 * k + 8]+ srcs.append(([uops["LBU"], in_drom] + lk, byte + [zero] * 24))+ srcs.append(([uops["LB"], in_drom] + lk, byte + [byte[7]] * 24))+ if k % 2 == 0:+ half = dbits[8 * k:8 * k + 16]+ srcs.append(([uops["LHU"], in_drom] + lk, half + [zero] * 16))+ srcs.append(([uops["LH"], in_drom] + lk, half + [half[15]] * 16))+ if text: # text-buffer loads: the same word / byte / half selection+ srcs.append(([uops["LW"], in_text], tmem))+ for k in range(4):+ lk = lits_of(lane, k)+ byte = tmem[8 * k:8 * k + 8]+ srcs.append(([uops["LBU"], in_text] + lk, byte + [zero] * 24))+ srcs.append(([uops["LB"], in_text] + lk, byte + [byte[7]] * 24))+ if k % 2 == 0:+ half = tmem[8 * k:8 * k + 16]+ srcs.append(([uops["LHU"], in_text] + lk, half + [zero] * 16))+ srcs.append(([uops["LH"], in_text] + lk, half + [half[15]] * 16))+ n_plain = len(srcs)+ srcs += [([any_of("DIV", "DIVU")], qs), ([remc], mul_lo)]+ val, k0t, k1t = [], [], []+ for i in range(32):+ terms = []+ for gate, bits in srcs:+ if bits[i].terms:+ terms += c.t_and([bits[i]] + gate)+ if i == 0:+ terms += c.t_and([slt, sltish]) + c.t_and([sltu, sltuish])+ val.append(c.node(terms, f"val.b{i}"))+ for gate, bits in srcs:+ k0t += c.t_and(gate + bits[14:32]) + c.t_and(gate + [1 - b for b in bits[14:32]])+ k1t += c.t_and(gate + bits[29:32]) + c.t_and(gate + [1 - b for b in bits[29:32]])+ k0t += c.t_and([sltish + sltuish])+ k1t += c.t_and([sltish + sltuish])+ k0 = c.node(k0t, "fits15")+ k1 = c.node(k1t, "fits30")+ taken = c.node(c.t_and([uops["BEQ"], eq]) + c.t_and([uops["BNE"], 1 - eq])+ + c.t_and([uops["BLT"], slt]) + c.t_and([uops["BGE"], 1 - slt])+ + c.t_and([uops["BLTU"], sltu]) + c.t_and([uops["BGEU"], 1 - sltu]),+ "taken")+ # fault 6: a bad JALR target, a bad JAL target, or a taken branch to a bad target+ fc[6] = c.node(fc6_terms + c.t_and([taken, bbad]), "fault.6")+ any_fault = sum(fc.values(), Expr())+ cause = [sum((fc[k] for k in fc if k >> b & 1), Expr()) for b in range(3)]++ # ---------------------------------------------------------------- the next token+ valc = (addish + uops["LUI"] + uops["AUIPC"] + uops["MISA"] + logicops + shiftops + loads+ + uops["MUL"] + mulhs + any_of(*RF.ZBA) + sltish + sltuish)+ ok = one - any_fault+ nr = [j0, one - at_max, ok] # the first token of a record (no refresh, no fault)+ cases = [] # (literals, family, payload bits or int)+ # P3: at a BYTE / HALT the next token is the echo (a Qwen token), at </tool_call> it is GO;+ # every other case is off there+ nb = [one - fam["BYTE"] - fam["HALT"] - c.isTCC] if text else []++ def case(lits, fam_name, payload=0):+ cases.append((lits + nb, fam_name, payload))++ case([j0, at_max], "R", w)+ case(nr + [valc, rdw_any, k0], "F0", val[0:15])+ case(nr + [valc, rdw_any, 1 - k0], "V", val[0:15])+ case([j1, ok, valc, rdw_any, k1], "F1", val[15:30])+ case([j1, ok, valc, rdw_any, 1 - k1], "V", val[15:30])+ case([j2, ok, valc, rdw_any], "F2", val[30:32])+ case(nr + [valc, 1 - rdw_any], "NT")+ # DIV-class: QS(k0), the digits QD(k << 4 | q), then the value record (v counts its tokens)+ case(nr + [divc, rdw_any], "QS", k0b)+ case(nr + [divc, 1 - rdw_any], "NT")+ case([divc, dp], "QD", qbits + kbits)+ dv = [divc, 1 - dp, 1 - j0]+ case(dv + [v0, k0], "F0", val[0:15])+ case(dv + [v0, 1 - k0], "V", val[0:15])+ case(dv + [v1, k1], "F1", val[15:30])+ case(dv + [v1, 1 - k1], "V", val[15:30])+ case(dv + [v2], "F2", val[30:32])+ case(nr + [branches, taken], "J", btgt)+ case(nr + [branches, 1 - taken], "NT")+ case(nr + [uops["JAL"]], "J", jtgt)+ case(nr + [uops["JALR"]], "J", EA[2:18]) # word = (EA - 0x40000) >> 2+ case(nr + [uops["SW"], 1 - EA[6]], "ST0")+ case(nr + [uops["SW"], EA[6]], "ST1")+ sbsh = any_of("SB", "SH")+ case(nr + [sbsh], "V", merged[0:15])+ case([j1, ok, sbsh], "V", merged[15:30])+ case([j2, ok, sbsh], "V", merged[30:32])+ case([j3, ok, sbsh, 1 - EA[6]], "SS0")+ case([j3, ok, sbsh, EA[6]], "SS1")+ case(nr + [ecall, a7_is[1]], "BYTE", B[0:9]) # a byte, or (256..) a P3 special token+ case(nr + [ecall, a7_is[93]], "HALT")+ case(nr + [uops["FENCE"]], "NT")+ # digit-file ops: BD per digit (j < n), then rd = the last carry (or NT)+ for start in ([j0, one - at_max], [one - j0]):+ case(start + [bnrow, lt_n], "BD", bd_dig + bd_car)+ case(start + [bnrow, eq_n, rdw_any], "F0", pay["BD"][4:8])+ case(start + [bnrow, eq_n, 1 - rdw_any], "NT")+ lt63 = one - jt[62]+ for start in ([j0, one - at_max], [one - j0]):+ case(start + [bscan, isdig, lt63], "SC")+ for stop in ([1 - isdig], [isdig, 1 - lt63]):+ case(start + [bscan, rdw_any] + stop, "F0", jbin)+ case(start + [bscan, 1 - rdw_any] + stop, "NT")+ case(nr + [bget, rdw_any], "F0", ddig)+ case(nr + [bget, 1 - rdw_any], "NT")+ case(nr + [bprd], "BYTE", ddig + [one, one]) # '0' + digit+ # a fault: FLT(cause), the PC's 8 hex digits (most significant first), HALT+ case([j0, one - at_max, any_fault], "FLT", cause)+ for k in range(8):+ case([jeq(1 + k), any_fault], "HEX", pc_bits[4 * (7 - k):4 * (8 - k)])+ case([jeq(9), any_fault], "HALT")+ owner = None+ if text:+ cases.append(([fam["BYTE"]], "Q", c.pay["BYTE"]))+ nl = text.printable_index(text.NL) # a run ends in a newline+ cases.append(([fam["HALT"]], "Q", [one if nl >> i & 1 else Expr() for i in range(9)]))+ cases.append(([c.isTCC], "GO", 0))+ # who predicts the next token: the circuit from </tool_call> through a run's last echo+ owner = c.node(c.t_and([running]) + c.t_and([fam["HALT"]])+ + c.t_and([c.isTCC, one - running]), "owner")+ c.owner = owner+ own = [owner] if text else []+ fam_cases = [(l_, FAM_ID[fn]) for l_, fn, _ in cases]+ pay_cases = [(l_, pl) for l_, _, pl in cases if not isinstance(pl, int)]+ if text:+ # merged: no separate select dims (they would outlive the exit and reach Qwen's norms);+ # predictions read the exit's code dims+ c.out_fam = c.out_pay = None+ exit_min = max(n.level for n in c.nodes)+ else:+ fam_bits = c.select_bits(fam_cases, NFAM, "exit.fam")+ pay_bits = c.select_bits(pay_cases, NPAY, "exit.pay")+ c.out_fam, c.out_pay = fam_bits, pay_bits+ exit_min = max(c.level(b) for b in fam_bits) - 1+ # the exit: the input code is cleared, and the same cases write KAPPA * (2 * bit - 1) into+ # the code dims in the output stage itself (no extra layer); a stock output head tied to the+ # embedding then scores every token by how many code bits agree+ for e in c.code:+ c.ffn([(-1, one, e), (1, one, -1 * e)], "code.clear", into=e)+ for i, e in enumerate(c.code):+ cs, bit = (fam_cases, i) if i < NFAM else (pay_cases, i - NFAM)+ terms = c.t_and(own, -KAPPA) if text else c.t_const(-KAPPA)+ for lits, val_ in cs:+ if isinstance(val_, int):+ if val_ >> bit & 1:+ terms += c.t_and(lits + own, 2 * KAPPA)+ elif bit < len(val_) and val_[bit].terms:+ terms += c.t_and([*lits, val_[bit]] + own, 2 * KAPPA)+ c.ffn(terms, f"exit.code{i}", into=e, min_level=exit_min)+ c.keep = [next(iter(e.terms)) for e in c.code]+ if text:+ # tau: 0 wherever the circuit predicts (chip tokens: -1 + 1; owned text: TAU - TAU), the+ # text value where Qwen does, which keeps chip tokens out of Qwen's text (q35.merge)+ c.ffn([(1, c.mu, one)] + c.t_and([owner, c.is_text], -TAU_TEXT), "exit.tau",+ into=c.tau, min_level=exit_min)+ c.keep += [next(iter(c.tau.terms)), next(iter(c.mu.terms))]+ c.probe = {"w": w, "anchor": anchor, "cnt": cnt, "jt": jt, "vt": vt, "A": A, "B": B, "C": C,+ "imm": imm, "r": r, "val": val, "mem": mem, "ld": ld, "k0": k0, "k1": k1,+ "taken": taken, "logic": logic, "shift": shift, "merged": merged,+ "mul": mres, "fault": fc, "qal": qal, "qs": qs, "cmp": cmp_ge, "ahi": ahi,+ "k0b": k0b, "dbits": dbits, "btgt": btgt, "zres": zres, "bn_s": sdig,+ "bn_d": ddig, "bn_prod": bprod, "bn_dig": bd_dig, "bn_car": bd_car, "q2": q2}+ c.n_plain = n_plain+ return c+++# --- snapshots (the state the host injects) ---------------------------------------------------+++def reg_row(c: RvCore, value: int) -> np.ndarray:+ row = np.zeros(HD)+ row[list(REF_COLS)] = A_REF+ for i in range(32):+ if value >> i & 1:+ row[c.reg_cols[i][0]] = A_BIT+ return row+++def counter_rows(k: int) -> dict:+ r0 = np.zeros(HD)+ r0[2] = A_BIT * 2.0**-k+ r1 = np.zeros(HD)+ r1[list(REF_COLS)] = A_REF+ return {0: r0, 1: r1}+++def byte_rows(words: np.ndarray, n_heads: int) -> list:+ """Per head, the two value heads' rows of a byte table of 32-bit words (word index: head,+ value head, row pair, half, word in row); every row carries the ballast."""+ per = len(words) // n_heads+ out = []+ for h in range(n_heads):+ vh = []+ for x in range(2):+ blk = words[h * per + x * (per // 2):h * per + (x + 1) * (per // 2)]+ rows = np.zeros((128, HD))+ rows[:, list(REF_COLS)] = A_REF+ wl = np.arange(len(blk))+ pp, half, k = wl >> 4, wl >> 3 & 1, wl & 7+ rr = pp + 64 * half+ for b in range(4):+ rows[rr, 2 + 32 * half + 4 * k + b] = blk >> (8 * b) & 0xFF+ vh.append({i: rows[i] for i in range(128)})+ out.append(vh)+ return out+++_ROM_CACHE: dict = {}+++def snapshot(c: RvCore, m: "RF.Machine") -> None:+ """Load a machine's state into the circuit's tables (the heads' initial states), as at the+ start of a record (j = 0)."""+ pc = np.zeros(HD)+ pc[list(REF_COLS)] = A_REF+ for i in range(16):+ pc[2 + i] = A_BIT * (m.anchor >> i & 1)+ c.tables["pc"].init = [{0: pc}]+ c.tables["count"].init = [counter_rows(m.n)]+ c.tables["j"].init = [counter_rows(0)]+ c.tables["vcnt"].init = [counter_rows(0)]+ regs = {r: reg_row(c, m.x[r]) for r in range(1, 32)}+ for p in ("rs1", "rs2", "rd"):+ c.tables[f"regs.{p}"].init = [regs]+ quo = np.zeros(HD)+ quo[list(REF_COLS)] = A_REF+ for h in range(4):+ c.tables[f"quo{h}"].init = [{0: quo.copy()} for _ in range(2)]+ key = (hash(tuple(m.rom)), hash(m.drom)) # (rows of one firmware share ROM and DROM)+ if _ROM_CACHE.get("key") != key:+ rom = np.array([RF.rom_word(wd, S.ROM_BASE + 4 * i) if wd else 0+ for i, wd in enumerate(m.rom)], dtype=np.int64)+ drom = np.frombuffer(m.drom.ljust(4 * S.DROM_WORDS, bytes(1)), dtype="<u4").astype(np.int64)+ _ROM_CACHE.update(key=key, rom=byte_rows(rom, S.ROM_WORDS >> 11),+ drom=byte_rows(drom, S.DROM_WORDS >> 11))+ for h, vh in enumerate(_ROM_CACHE["rom"]):+ c.tables[f"rom{h}"].init = vh+ for h, vh in enumerate(_ROM_CACHE["drom"]):+ c.tables[f"drom{h}"].init = vh+ for port in (1, 2):+ for h in range(4):+ vh = []+ for x in range(2):+ rows = {}+ for r, dg in enumerate(m.bn[2 * h + x]):+ row = np.zeros(HD)+ row[list(REF_COLS)] = A_REF+ for b in range(4):+ row[2 + b] = A_BIT * (dg >> b & 1)+ rows[r] = row+ vh.append(rows)+ c.tables[f"bn{port}.{h}"].init = vh+ for h in range(S.RAM_WORDS >> 8):+ r0, r1 = {}, {}+ for wi in range(S.RAM_WORDS):+ if (wi & 15 | (wi >> 12) << 4) != h or not m.ram[wi]:+ continue+ row = np.zeros(HD)+ row[list(REF_COLS)] = A_REF+ for i in range(32):+ row[2 + i] = A_BIT * (m.ram[wi] >> i & 1)+ (r1 if wi >> 4 & 1 else r0)[wi >> 5 & 127] = row+ c.tables[f"ram{h}"].init = [r0, r1]+++def decode_code(fid: int, p: int, text=None) -> int:+ """The token of an output code (family id, payload), -1 if it names none; a Q code is the+ text token of ``text``'s printable token p."""+ if not 0 <= fid < len(OUT_NAMES):+ return -1+ nm = OUT_NAMES[fid]+ if nm == "Q":+ q = text.printable[p] if text and p < len(text.printable) else None+ return -1 if q is None else S.TEXT_TOKEN + q+ if nm in S.CONTROLS:+ return S.tok(nm)+ return S.tok(nm, p) if p < S.FAMILIES[nm][1] else -1+++def predictions(c: RvCore, mat: np.ndarray) -> list[int]:+ """The predicted next token at every position (from the output code dims)."""+ def bits(es):+ return [np.rint(sum(mat[:, d] * float(w_) for d, w_ in e.terms.items()))+ if e.terms else np.zeros(mat.shape[0]) for e in es]++ if c.out_fam is None: # merged: the exit's code dims (KAPPA * (2 bit - 1) where it owns)+ cb = [(sum(mat[:, d] * float(w_) for d, w_ in e.terms.items()) > 0).astype(int)+ for e in c.code]+ fb, pb = cb[:NFAM], cb[NFAM:]+ else:+ fb, pb = bits(c.out_fam), bits(c.out_pay)+ out = []+ for t in range(mat.shape[0]):+ fid = int(sum(int(fb[i][t]) << i for i in range(NFAM)))+ p = int(sum(int(pb[i][t]) << i for i in range(NPAY)))+ out.append(decode_code(fid, p, c.text))+ return outdiff --git a/experimental/riscv/chipformer/rv/elf.py b/experimental/riscv/chipformer/rv/elf.pynew file mode 100644index 0000000..ccd3c57--- /dev/null+++ b/experimental/riscv/chipformer/rv/elf.py@@ -0,0 +1,53 @@+"""Load a static RV32 ELF (gcc, env/rom.ld or env/qemu.ld) into an asm.Program or an Rv32."""++import struct+from pathlib import Path++from chipformer.rv import spec as S+from chipformer.rv.asm import Program+++def segments(path) -> tuple[int, list[tuple[int, bytes]]]:+ """(entry, [(vaddr, bytes incl. zero-filled bss)]) of the PT_LOAD segments."""+ b = Path(path).read_bytes()+ assert b[:4] == b"\x7fELF" and b[4] == 1, "ELF32 only"+ entry, phoff = struct.unpack_from("<II", b, 0x18)+ phentsize, phnum = struct.unpack_from("<HH", b, 0x2A)+ segs = []+ for i in range(phnum):+ p_type, off, vaddr, _, filesz, memsz, _, _ = struct.unpack_from("<8I", b, phoff + i * phentsize)+ if p_type == 1 and memsz:+ segs.append((vaddr, b[off:off + filesz] + bytes(memsz - filesz)))+ return entry, segs+++def program(path) -> Program:+ """A ROM-map ELF as a Program: code words from ROM_BASE, read-only data from DROM_BASE,+ data bytes from RAM_BASE."""+ entry, segs = segments(path)+ regions = {"rom": (S.ROM_BASE, 4 * S.ROM_WORDS), "drom": (S.DROM_BASE, 4 * S.DROM_WORDS),+ "ram": (S.RAM_BASE, 4 * S.RAM_WORDS)}+ bufs = {k: bytearray() for k in regions}+ for vaddr, blob in segs:+ for k, (base, size) in regions.items():+ if base <= vaddr and vaddr + len(blob) <= base + size:+ b = bufs[k]+ end = vaddr - base + len(blob)+ b += bytes(max(0, end - len(b)))+ b[vaddr - base:end] = blob+ break+ else:+ raise AssertionError(f"segment at {vaddr:#x} (+{len(blob)}) outside ROM/DROM/RAM")+ text = bufs["rom"] + bytes(-len(bufs["rom"]) % 4)+ words = [int.from_bytes(text[i:i + 4], "little") for i in range(0, len(text), 4)]+ return Program(words, bytes(bufs["ram"]), {"_start": entry}, data_base=S.RAM_BASE,+ rodata=bytes(bufs["drom"]))+++def load_into(cpu, path) -> None:+ """Load any static ELF into a ref.Rv32 (flat memory) and set its PC."""+ entry, segs = segments(path)+ for vaddr, blob in segs:+ for i, x in enumerate(blob):+ cpu.mem[vaddr + i] = x+ cpu.pc = entrydiff --git a/experimental/riscv/chipformer/rv/env/build_mpy.sh b/experimental/riscv/chipformer/rv/env/build_mpy.shnew file mode 100644index 0000000..647edd7--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/build_mpy.sh@@ -0,0 +1,46 @@+#!/bin/bash+# Build MicroPython for the chipformer RV32 machine, one firmware per script (run in WSL):+# build_mpy.sh OUT script.py [script.py ...]+# The configuration is dist/rv_tool's best build under 256 KB: the tree TREE carries its 3 source+# patches (FIRST-set parse prediction, fast lexer, qstr bucket index), the FIRST-set table sits in+# rodata (FS: the table dumped by rv_tool's build_fsrom.sh), 5 hot files at -O3, word-wise libc+# (fastlib.c), computed goto, the map lookup cache, no conservative GC clear, parse chunk 512;+# -march=rv32im_zba. OUT/mpy-<script>.elf is the ROM/DROM/RAM map, OUT/mpy-<script>.qemu.elf+# the same code for qemu-riscv32; OUT/qemu.json gets their exit codes and output.+set -euo pipefail+HERE=$(cd "$(dirname "$0")" && pwd)+TREE=${TREE:-/root/build/mp-all3}+FS=${FS:-$TREE/ports/rv32sim/build-v-combo-hot5O3/fs_table.h}+OUT=$(realpath "$1"); shift+P=$TREE/ports/chipformer+mkdir -p "$P" "$OUT"+cp "$HERE"/mpy/*.c "$HERE"/mpy/*.h "$HERE"/mpy/Makefile "$P"/+COPT="-DRV_FIRST_SETS -DRV_FS_ROM -DMICROPY_OPT_COMPUTED_GOTO=1 -DMICROPY_OPT_MAP_LOOKUP_CACHE=1"+COPT="$COPT -DMICROPY_GC_CONSERVATIVE_CLEAR=0 -DMICROPY_ALLOC_PARSE_CHUNK_INIT=512"+HOT="py/parse py/lexer py/qstr py/map py/vm"+for s in "$@"; do+ n=$(basename "$s" .py)+ for map in rom qemu; do+ B=build-$n-$map+ mkdir -p "$P/$B"+ cp "$FS" "$P/$B/fs_table.h"+ make -C "$P" -j16 CONFIG=basic MAP=$map SCRIPT="$(realpath "$s")" ENV="$HERE" BUILD=$B \+ HOT="$HOT" COPT="$COPT" > "/tmp/mpy-$n-$map.log" 2>&1 || { tail -40 "/tmp/mpy-$n-$map.log"; exit 1; }+ sfx=$([ $map = rom ] && echo "" || echo ".qemu")+ cp "$P/$B/firmware.elf" "$OUT/mpy-$n$sfx.elf"+ grep -E '^\.(text|rodata|data|bss)' "/tmp/mpy-$n-$map.log" | tr '\n' ' '; echo " ($n $map)"+ done+done+python3 - "$OUT" "$@" <<'PYEOF'+import json, os, subprocess, sys+out, scripts = sys.argv[1], sys.argv[2:]+path = os.path.join(out, "qemu.json")+res = json.load(open(path)) if os.path.exists(path) else {}+for s in scripts:+ n = "mpy-" + os.path.basename(s)[:-3]+ p = subprocess.run(["qemu-riscv32", "-cpu", "rv32,zba=true", os.path.join(out, n + ".qemu.elf")],+ capture_output=True, timeout=600)+ res[n] = {"exit": p.returncode, "stdout": p.stdout.decode("latin-1")}+ print(n, p.returncode, repr(res[n]["stdout"][:200]))+json.dump(res, open(path, "w"), indent=1)+PYEOFdiff --git a/experimental/riscv/chipformer/rv/env/build_tests.sh b/experimental/riscv/chipformer/rv/env/build_tests.shnew file mode 100644index 0000000..e566707--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/build_tests.sh@@ -0,0 +1,63 @@+#!/bin/bash+# Build riscv-tests (rv32ui, rv32um, rv32uzba) and the C programs for the chipformer RV32 machine+# (run in WSL):+# build_tests.sh RISCV_TESTS_SRC MINI_GMP_DIR OUT+# For each program: OUT/<name>.elf (ROM/DROM/RAM map, for the reference and the circuit) and+# OUT/<name>.qemu.elf (the same code at qemu-mappable addresses, putchar as write(2));+# OUT/qemu.json holds qemu's exit code and stdout for every .qemu.elf.+# C: gcc -march=rv32im_zba, picolibc (rv32im).+set -euo pipefail+SRC=$1+GMP=$2+OUT=$3+HERE=$(cd "$(dirname "$0")" && pwd)+PICO=/usr/lib/picolibc/riscv64-unknown-elf+GCC=riscv64-unknown-elf-gcc+MARCH=rv32im_zba+LIBGCC=$($GCC -march=rv32im -mabi=ilp32 -print-libgcc-file-name)+LIBS="-Wl,--start-group $PICO/lib/release/rv32im/ilp32/libc.a $LIBGCC -Wl,--end-group"+QEMU="qemu-riscv32 -cpu rv32,zba=true"+mkdir -p "$OUT"+rm -f "$OUT"/*.elf++# compliance tests: rv32ui (but fence_i: self-modifying code; ma_data: misaligned), rv32um,+# rv32uzba+for s in "$SRC"/isa/rv32ui/*.S "$SRC"/isa/rv32um/*.S "$SRC"/isa/rv32uzba/*.S; do+ n=$(basename "$s" .S)+ suite=$(basename "$(dirname "$s")")+ case " fence_i ma_data " in *" $n "*) continue;; esac+ for map in rom qemu; do+ sfx=$([ $map = rom ] && echo "" || echo ".qemu")+ $GCC -march=$MARCH -mabi=ilp32 -nostdlib -nostartfiles -static -I"$HERE" \+ -I"$SRC/isa/macros/scalar" -T"$HERE/$map.ld" -o "$OUT/$suite-$n$sfx.elf" "$s"+ done+done++# C programs (gmpmul links mini-gmp)+for c in "$HERE"/progs/*.c; do+ n=$(basename "$c" .c)+ extra=""+ [ "$n" = gmpmul ] && extra="-I$GMP $GMP/mini-gmp.c"+ for map in rom qemu; do+ sfx=$([ $map = rom ] && echo "" || echo ".qemu")+ def=$([ $map = rom ] && echo "" || echo "-DQEMU")+ $GCC -march=$MARCH -mabi=ilp32 -O2 $def -isystem "$PICO/include" -I"$HERE" \+ -nostdlib -nostartfiles -static -T"$HERE/$map.ld" -o "$OUT/c-$n$sfx.elf" \+ "$HERE/crt0.S" "$c" "$HERE/sys.c" $extra $LIBS+ done+done++python3 - "$OUT" "$QEMU" <<'EOF'+import json, subprocess, sys, glob, os+out, qemu = sys.argv[1], sys.argv[2].split()+res = {}+for f in sorted(glob.glob(os.path.join(out, "*.qemu.elf"))):+ p = subprocess.run(qemu + [f], capture_output=True, timeout=600)+ res[os.path.basename(f)[:-len(".qemu.elf")]] = {"exit": p.returncode,+ "stdout": p.stdout.decode("latin-1")}+json.dump(res, open(os.path.join(out, "qemu.json"), "w"), indent=1)+print(len(res), "programs;", sum(r["exit"] == 0 for r in res.values()), "exit 0")+for k, r in res.items():+ if k.startswith("c-"):+ print(k, r["exit"], r["stdout"].strip()[:80])+EOFdiff --git a/experimental/riscv/chipformer/rv/env/calc/bn.h b/experimental/riscv/chipformer/rv/env/calc/bn.hnew file mode 100644index 0000000..60d67b1--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/calc/bn.h@@ -0,0 +1,54 @@+// The chipformer digit file (custom-0 / custom-1, see chipformer/rv/ref.py): 8 registers of 128+// decimal digits, index 0 least significant. Row ops take every operand from one word:+// BN_OPS(n, off, m, d, s, ovw) = d | s << 3 | ovw << 6 | n << 7 | off << 13 | m << 20+// bn_mac: for i < n: v = (ovw ? 0 : D[i+off]) + S[i] * m + c; D[i+off] = v % 10; returns c+// bn_msb: for i < n: v = D[i+off] - S[i] * m - b; D[i+off] = v % 10; returns b+// bn_ld: for i < n: D[i+off] = min(end[-1-i] & 15, 9) (ASCII digits ending at end - 1)+// bn_get: returns D[off]+// bn_scan: the length of the ASCII digit run at s+// Printing uses a sled of BPRD instructions per register (sled.S): one token per digit.+#pragma once+#include <stdint.h>++#define BN_OPS(n, off, m, d, s, ovw) \+ ((uint32_t)(d) | (uint32_t)(s) << 3 | (uint32_t)(ovw) << 6 | (uint32_t)(n) << 7 | \+ (uint32_t)(off) << 13 | (uint32_t)(m) << 20)+#define BN_OFF(off) ((uint32_t)(off) << 13)+#define BN_M(m) ((uint32_t)(m) << 20)++static inline uint32_t bn_mac(uint32_t ops) {+ uint32_t r;+ __asm__ volatile(".insn r 0x0B, 0, 0, %0, x0, %1" : "=r"(r) : "r"(ops) : "memory");+ return r;+}++static inline uint32_t bn_msb(uint32_t ops) {+ uint32_t r;+ __asm__ volatile(".insn r 0x0B, 1, 0, %0, x0, %1" : "=r"(r) : "r"(ops) : "memory");+ return r;+}++static inline void bn_ld(const char *end, uint32_t ops) {+ __asm__ volatile(".insn r 0x0B, 2, 0, x0, %0, %1" : : "r"(end), "r"(ops) : "memory");+}++static inline uint32_t bn_get(uint32_t ops) {+ uint32_t r;+ __asm__ volatile(".insn r 0x0B, 3, 0, %0, x0, %1" : "=r"(r) : "r"(ops) : "memory");+ return r;+}++// the number of ASCII digits at s, s + 1, .. (at most 63)+static inline uint32_t bn_scan(const char *s) {+ uint32_t r;+ __asm__ volatile(".insn r 0x0B, 4, 0, %0, %1, x0" : "=r"(r) : "r"(s) : "memory");+ return r;+}++extern const uint32_t bn_sleds[8]; // sled.S: the first BPRD of each register's sled++// print the top `len` digits of register r (len <= 128), most significant first+static inline void bn_print(int r, int len) {+ void (*f)(void) = (void (*)(void))(bn_sleds[r] + 4 * (128 - len));+ f();+}diff --git a/experimental/riscv/chipformer/rv/env/calc/build_calc.sh b/experimental/riscv/chipformer/rv/env/calc/build_calc.shnew file mode 100644index 0000000..eca5232--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/calc/build_calc.sh@@ -0,0 +1,22 @@+#!/bin/bash+# Build the calc firmware (run in WSL): build_calc.sh OUT+# OUT/calc.elf: the ROM / DROM / RAM map, the expression buffer calc_in at RAM 0x100.+set -euo pipefail+HERE=$(cd "$(dirname "$0")" && pwd)+ENV=$HERE/..+OUT=$(realpath "$1"); NAME=${2:-calc}+PICO=/usr/lib/picolibc/riscv64-unknown-elf+GCC=riscv64-unknown-elf-gcc+MARCH=rv32im_zba+LIBGCC=$($GCC -march=rv32im -mabi=ilp32 -print-libgcc-file-name)+B=$(mktemp -d)+python3 "$HERE/gen_sled.py" "$B/sled.S"+# calc_in first in RAM (0x100)+sed 's/ .data : {/ .calcin : { *(.calcin) } > RAM\n .data : {/' "$ENV/rom.ld" > "$B/calc.ld"+$GCC -march=$MARCH -mabi=ilp32 ${CFLAGS:--O2} -isystem "$PICO/include" -I"$ENV" -I"$HERE" \+ -nostdlib -nostartfiles -static -T"$B/calc.ld" -o "$OUT/$NAME.elf" \+ "$ENV/crt0.S" "$HERE/calc.c" "$ENV/sys.c" "$B/sled.S" \+ -Wl,--start-group "$PICO/lib/release/rv32im/ilp32/libc.a" "$LIBGCC" -Wl,--end-group+riscv64-unknown-elf-size -A "$OUT/$NAME.elf" | grep -E '^\.(text|rodata|calcin|data|bss)'+riscv64-unknown-elf-nm "$OUT/$NAME.elf" | grep -E ' calc_in$'+rm -rf "$B"diff --git a/experimental/riscv/chipformer/rv/env/calc/calc.c b/experimental/riscv/chipformer/rv/env/calc/calc.cnew file mode 100644index 0000000..bdf2d61--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/calc/calc.c@@ -0,0 +1,518 @@+// calc: exact decimal integer expressions on the chipformer digit file (bn.h).+//+// Input: the NUL-terminated expression in calc_in (RAM 0x100; the host writes it into the+// snapshot). Grammar (Python's precedence and semantics):+// expr = term {(+|-) term} term = factor {(*|/|//|%) factor}+// factor = (-|+) factor | power power = atom [(**|^) factor]+// atom = digits | ( expr ) | gcd ( expr , expr )+// / and // are floor division, % the matching modulo (the divisor's sign), ** / ^ a power with+// a small non-negative exponent. Output: the value in decimal and "\n", or "error\n" (division by+// zero, a value over MAXD digits, a syntax error).+//+// A value is one word: register | negative << 8 | length << 16; a register's digits at and above+// the value's length are 0. Register 7 holds the constant 1 (quotient digits, +1).+#include <stdint.h>+#include "bn.h"+#include "rt.h"++char calc_in[256] __attribute__((section(".calcin")));++#define MAXD 62+#define ONE 7++typedef uint32_t num;+#define REG(a) ((a) & 7)+#define NEG(a) ((a) >> 8 & 1)+#define LEN(a) ((a) >> 16)+#define NUM(r, neg, len) ((uint32_t)(r) | (uint32_t)(neg) << 8 | (uint32_t)(len) << 16)++static uint32_t dirty[8]; // per register: digits that may be nonzero lie below this+static uint32_t used; // allocated registers+static int err;+static char one_ch = '1';++static int dig(num a, int i) { return bn_get(BN_OPS(0, i, 0, REG(a), 0, 0)); }++static void wrote(num a, int n) {+ if (n > dirty[REG(a)]) dirty[REG(a)] = n;+}++static num alloc(int len) {+ for (int r = 0; r < ONE; r++)+ if (!(used >> r & 1)) {+ used |= 1u << r;+ for (int o = 0; o < dirty[r]; o += 63) // zero what an earlier value left+ bn_mac(BN_OPS(dirty[r] - o < 63 ? dirty[r] - o : 63, o, 0, r, r, 1));+ dirty[r] = 0;+ return NUM(r, 0, len);+ }+ err = 1;+ return NUM(0, 0, len);+}++static void release(num a) { used &= ~(1u << REG(a)); }++static int is_zero(num a) { return LEN(a) == 1 && !dig(a, 0); }++static num trim(num a) { // drop leading zero digits; zero is never negative+ int n = LEN(a);+ while (n > 1 && !dig(a, n - 1)) n--;+ a = (a & 0xFFFF) | (uint32_t)n << 16;+ return n == 1 && !dig(a, 0) ? a & ~0x100u : a;+}++static int cmp_mag(num a, num b) {+ if (LEN(a) != LEN(b)) return LEN(a) < LEN(b) ? -1 : 1;+ for (int i = LEN(a) - 1; i >= 0; i--) {+ int x = dig(a, i), y = dig(b, i);+ if (x != y) return x < y ? -1 : 1;+ }+ return 0;+}++static num add_mag(num a, num b) { // |a| + |b| into a's register (a's sign)+ int n = (LEN(a) > LEN(b) ? LEN(a) : LEN(b)) + 1;+ if (n > MAXD) err = 1;+ bn_mac(BN_OPS(n, 0, 1, REG(a), REG(b), 0));+ wrote(a, n);+ if (!dig(a, n - 1)) n--;+ return (a & 0xFFFF) | (uint32_t)n << 16;+}++static num sub_mag(num a, num b) { // |a| - |b| into a's register (|a| >= |b|)+ bn_msb(BN_OPS(LEN(a), 0, 1, REG(a), REG(b), 0));+ return trim(a);+}++__attribute__((noinline)) static num add(num a, num b) { // a + b; consumes both+ num s;+ if (NEG(a) == NEG(b)) {+ s = add_mag(a, b);+ release(b);+ } else if (cmp_mag(a, b) >= 0) {+ s = sub_mag(a, b);+ release(b);+ } else {+ s = sub_mag(b, a);+ release(a);+ }+ return s;+}++__attribute__((noinline)) static num mul(num a, num b) { // a * b in a new register; keeps both+ if (LEN(a) < LEN(b)) {+ num t = a;+ a = b;+ b = t;+ }+ int la = LEN(a), lb = LEN(b);+ num c = alloc(la + lb);+ if (la + lb > MAXD) {+ err = 1;+ return c;+ }+ uint32_t base = BN_OPS(la + 1, 0, 0, REG(c), REG(a), 0);+ for (int j = 0; j < lb; j++) {+ uint32_t d = dig(b, j);+ if (d) bn_mac(base | BN_OFF(j) | BN_M(d));+ }+ wrote(c, la + lb);+ return trim(c | (NEG(a) ^ NEG(b)) << 8);+}++typedef struct {+ num q, r;+} qr;++__attribute__((noinline)) static qr divmod(num a, num b) { // |a| = q |b| + r: r in a's register, q new, b kept+ if (is_zero(b)) err = 1;+ if (err || cmp_mag(a, b) < 0) return (qr){alloc(1), a & ~0x100u};+ int la = LEN(a), lb = LEN(b), ra = REG(a);+ num q = alloc(la - lb + 1);+ int b1 = dig(b, lb - 1), den = lb >= 2 ? b1 * 10 + dig(b, lb - 2) : b1;+ uint32_t msb = BN_OPS(lb + 1, 0, 0, ra, REG(b), 0), mac = BN_OPS(lb + 1, 0, 1, ra, REG(b), 0);+ uint32_t setq = BN_OPS(1, 0, 0, REG(q), ONE, 1);+ for (int t = la - lb; t >= 0; t--) {+ int top = t + lb; // the window a[t .. top]: its value / |b| is the digit (<= 9)+ int w = dig(a, top) * 10 + dig(a, top - 1);+ if (lb >= 2) w = w * 10 + dig(a, top - 2);+ int qd = (lb >= 3 ? w + 1 : w) / den; // never below the digit, at most 2 above+ if (qd > 9) qd = 9;+ if (qd) {+ uint32_t o = BN_OFF(t);+ if (bn_msb(msb | o | BN_M(qd))) // negative: add b back+ do qd--;+ while (!bn_mac(mac | o));+ if (qd) bn_mac(setq | o | BN_M(qd));+ }+ }+ wrote(q, la - lb + 1);+ return (qr){trim(q), trim(NUM(ra, 0, lb))};+}++__attribute__((noinline)) static num divide(num a, num b, int want_mod) { // floor division / modulo; consumes both+ int sa = NEG(a), sb = NEG(b);+ qr d = divmod(a, b);+ if (sa != sb && !is_zero(d.r)) {+ d.q = add_mag(d.q, NUM(ONE, 0, 1));+ num r2 = sub_mag(b & ~0x100u, d.r); // |b| - r, in b's register+ release(d.r);+ d.r = r2;+ } else {+ release(b);+ }+ release(want_mod ? d.q : d.r);+ num out = want_mod ? d.r : d.q;+ out = (out & ~0x100u) | (uint32_t)(want_mod ? sb : sa ^ sb) << 8;+ return is_zero(out) ? out & ~0x100u : out;+}++__attribute__((noinline)) static num gcd(num a, num b) {+ a &= ~0x100u;+ b &= ~0x100u;+ while (!err && !is_zero(b)) {+ qr d = divmod(a, b);+ release(d.q);+ a = b;+ b = d.r;+ }+ release(b);+ return a;+}++__attribute__((noinline)) static num power(num a, num e) {+ int x = 0;+ if (NEG(e) || LEN(e) > 3) err = 1;+ for (int i = LEN(e) - 1; i >= 0 && !err; i--) x = x * 10 + dig(e, i);+ release(e);+ num res = alloc(1);+ bn_mac(BN_OPS(1, 0, 1, REG(res), ONE, 1));+ wrote(res, 1);+ while (x && !err) {+ if (x & 1) {+ num t = mul(res, a);+ release(res);+ res = t;+ }+ x >>= 1;+ if (x) {+ num t = mul(a, a);+ release(a);+ a = t;+ }+ }+ release(a);+ return res;+}+++// --- the expression: an operator-precedence (shunting-yard) loop, no recursion ---------------+// ops: + - (1), * / // % (2: 'd', 'f'), unary minus 'n' (3), ^ (4, right associative); '(' and 'g' (gcd's+// open bracket) mark groups++static num vals[16];+static char ops[16];+void _exit(int code) __attribute__((noreturn));++static int prec(int op) { // 'f': // (floor), 'd': / (exact)+ return op == '^' ? 4 : op == 'n' ? 3 : op == '*' || op == 'f' || op == 'd' || op == '%' ? 2 : 1;+}++// a / b exact: an integer, or (only as the expression's final value) a reduced fraction whose+// denominator is vden[i]+static num vden[16];+static int hasden[16];++static num copy_of(num a) {+ num c = alloc(LEN(a));+ bn_mac(BN_OPS(LEN(a), 0, 1, REG(c), REG(a), 1));+ wrote(c, LEN(a));+ return NUM(REG(c), NEG(a), LEN(a));+}++__attribute__((noinline)) static num exact_div(num a, num b, int i) { // consumes both+ int neg = NEG(a) ^ NEG(b);+ if (is_zero(b)) {+ err = 1;+ return a;+ }+ num g = gcd(copy_of(a), copy_of(b));+ qr x = divmod(a & ~0x100u, g);+ release(x.r);+ qr y = divmod(b & ~0x100u, g);+ release(y.r);+ release(g);+ if (LEN(y.q) == 1 && dig(y.q, 0) == 1) {+ release(y.q);+ } else {+ vden[i] = y.q;+ hasden[i] = 1;+ }+ return is_zero(x.q) ? x.q : x.q | (uint32_t)neg << 8;+}++// decimals: a value is the integer vals[i] scaled by 10^-vsc[i] (+ - * keep it exact; the other+// operators need integers)+static int vsc[16];++static num shift(num a, int k) { // a * 10^k in a new register; consumes a+ int n = LEN(a) + k;+ if (n > MAXD) {+ err = 1;+ return a;+ }+ num c = alloc(n);+ bn_mac(BN_OPS(LEN(a), k, 1, REG(c), REG(a), 1));+ wrote(c, n);+ release(a);+ return NUM(REG(c), NEG(a), n);+}++static int apply(int op, int nv) { // returns the new value count+ if (op == 'n') {+ num v = vals[nv - 1];+ vals[nv - 1] = is_zero(v) ? v : v ^ 0x100u;+ return nv;+ }+ if (nv < 2) {+ err = 1;+ return nv;+ }+ int sb = vsc[nv - 1], sa = vsc[nv - 2];+ if (hasden[nv - 1] || hasden[nv - 2]) err = 1; // a fraction is only ever the final value+ num b = vals[--nv], a = vals[nv - 1], r;+ if (err) {+ r = a;+ } else if (op == '+' || op == '-') {+ if (sa < sb) a = shift(a, sb - sa), sa = sb;+ if (sb < sa) b = shift(b, sa - sb);+ r = add(a, op == '-' && !is_zero(b) ? b ^ 0x100u : b);+ } else if (op == '*') {+ r = mul(a, b);+ release(a);+ release(b);+ sa += sb;+ } else if (sa || sb) {+ err = 1;+ r = a;+ } else if (op == '^') {+ r = power(a, b);+ } else if (op == 'g') {+ r = gcd(a, b);+ } else if (op == 'd') {+ hasden[nv - 1] = 0;+ r = exact_div(a, b, nv - 1);+ } else {+ r = divide(a, b, op == '%');+ }+ if (op != 'd') hasden[nv - 1] = 0;+ vals[nv - 1] = r;+ vsc[nv - 1] = sa;+ return nv;+}++static int calc_eval(void) { // prints the value (or "error") and a newline; 0 on success+ used = 0;+ err = 0;+ bn_ld(&one_ch + 1, BN_OPS(1, 0, 0, ONE, 0, 0));+ const char *s = calc_in;+ int operand = 1, nv = 0, no = 0; // operand: an operand (or a prefix) comes next+ // the fast path: "A op B" (two plain numbers, one operator)+ int na = bn_scan(s);+ if (na && na <= MAXD) {+ const char *o = s + na;+ int c = o[0], two = (c == '*' || c == '/') && o[1] == c;+ int op = c == '*' && two ? '^' : c == '/' ? (two ? 'f' : 'd') : c;+ const char *t = o + 1 + two;+ int nb = bn_scan(t);+ if (nb && nb <= MAXD && !t[nb] &&+ (op == '+' || op == '-' || op == '*' || op == 'f' || op == 'd' || op == '%' ||+ op == '^')) {+ bn_ld(o, BN_OPS(na, 0, 0, 0, 0, 0));+ bn_ld(t + nb, BN_OPS(nb, 0, 0, 1, 0, 0));+ used = 3;+ dirty[0] = na;+ dirty[1] = nb;+ vals[0] = dig(NUM(0, 0, na), na - 1) ? NUM(0, 0, na) : trim(NUM(0, 0, na));+ vals[1] = dig(NUM(1, 0, nb), nb - 1) ? NUM(1, 0, nb) : trim(NUM(1, 0, nb));+ vsc[0] = vsc[1] = 0;+ hasden[0] = hasden[1] = 0;+ nv = apply(op, 2);+ if (err) goto fail;+ goto print;+ }+ }+ for (;;) {+ int c = *s;+ if (c == ' ') {+ s++;+ continue;+ }+ if (nv >= 15 || no >= 15) err = 1;+ if (err) break;+ if (operand) {+ if ((unsigned)(c - '0') < 10) {+ int n = bn_scan(s); // (63: a longer run, over MAXD)+ const char *end = s + n;+ int frac = 0;+ if (*end == '.' && (unsigned)(end[1] - '0') < 10) { // a decimal: digits . digits+ static char dbuf[MAXD + 2];+ frac = bn_scan(end + 1);+ if (n + frac > MAXD) err = 1;+ if (err) break;+ for (int i = 0; i < n; i++) dbuf[i] = s[i];+ for (int i = 0; i < frac; i++) dbuf[n + i] = end[1 + i];+ s = end + 1 + frac;+ n += frac;+ end = dbuf + n;+ } else {+ s = end;+ }+ num a = alloc(n);+ if (n > MAXD) err = 1;+ bn_ld(end, BN_OPS(n, 0, 0, REG(a), 0, 0));+ wrote(a, n);+ vsc[nv] = frac;+ hasden[nv] = 0;+ vals[nv++] = dig(a, n - 1) ? a : trim(a);+ operand = 0;+ } else if (c == '(') {+ ops[no++] = '(';+ s++;+ } else if (c == '-' || c == '+') {+ if (c == '-') ops[no++] = 'n';+ s++;+ } else if (c == 'g' && s[1] == 'c' && s[2] == 'd' && s[3] == '(') {+ ops[no++] = 'g';+ s += 4;+ } else {+ err = 1;+ }+ continue;+ }+ if (c == ')' || c == ',') {+ while (no && ops[no - 1] != '(' && ops[no - 1] != 'g') nv = apply(ops[--no], nv);+ if (!no) {+ err = 1;+ } else if (c == ')') {+ if (ops[--no] == 'g') nv = apply('g', nv);+ } else if (ops[no - 1] != 'g') {+ err = 1;+ } else {+ operand = 1;+ }+ s++;+ continue;+ }+ if (!c) break;+ int op = c, len = 1;+ if (c == '*' && s[1] == '*') {+ op = '^';+ len = 2;+ } else if (c == '/') {+ op = s[1] == '/' ? 'f' : 'd';+ len = s[1] == '/' ? 2 : 1;+ } else if (c != '+' && c != '-' && c != '*' && c != '/' && c != '%' && c != '^') {+ err = 1;+ continue;+ }+ int pr = prec(op);+ while (no && ops[no - 1] != '(' && ops[no - 1] != 'g' &&+ (prec(ops[no - 1]) > pr || (prec(ops[no - 1]) == pr && op != '^')))+ nv = apply(ops[--no], nv);+ ops[no++] = op;+ s += len;+ operand = 1;+ }+ while (!err && no) {+ int op = ops[--no];+ if (op == '(' || op == 'g')+ err = 1;+ else+ nv = apply(op, nv);+ }+ if (err || nv != 1 || operand) {+ fail:+ for (const char *m = "error\n"; *m; m++) rv_putchar(*m);+ return 1;+ }+print:;+ num v = vals[0];+ int sc = vsc[0], n = LEN(v);+ if (NEG(v)) rv_putchar('-');+ if (!sc) {+ bn_print(REG(v), n);+ } else { // digits with the point, no trailing zeros after it (0.5, 12, -0.004)+ int z = 0;+ while (z < sc && !dig(v, z)) z++;+ if (n <= sc) rv_putchar('0');+ for (int i = n > sc ? n - 1 : sc - 1; i >= z; i--) {+ if (i == sc - 1) rv_putchar('.');+ rv_putchar('0' + (i < n ? dig(v, i) : 0));+ }+ }+ if (hasden[0]) { // a reduced fraction: num/den+ rv_putchar('/');+ bn_print(REG(vden[0]), LEN(vden[0]));+ }+ rv_putchar('\n');+ return 0;+}++#ifdef BRIDGE+// The P3 bridge: each run (GO, after </tool_call>) reads the tool call's text from the text+// buffer (the call's i-th token at 0x8000 + 16 i), takes its first parameter's value (what+// follows the second '>' up to the next '<', newlines dropped) into calc_in, prints the tool+// response and parks (exit ecall) until the next call:+// \n<tool_response>\nVALUE\n</tool_response>\n<think>\n\n</think>\n (+ the HALT echo \n)+#define TEXT ((const volatile unsigned char *)0x8000)+#define ROWS 80+enum { TOOL_RESPONSE = 256, TOOL_RESPONSE_END, THINK, THINK_END };++static void put(int c) { rv_ecall2(1, c, 0, 0); }++static int gather(void) {+ int n = 0, gt = 0;+ for (int i = 1; i <= ROWS; i++) {+ const volatile unsigned char *r = TEXT + 16 * i;+ int c = r[0];+ if (!c) break;+ for (int k = 0; k < 16 && c; c = ++k < 16 ? r[k] : 0) {+ if (gt < 2) {+ gt += c == '>';+ } else if (c == '<') {+ calc_in[n] = 0;+ return n;+ } else if (c != '\n' && n < (int)sizeof calc_in - 1) {+ calc_in[n++] = (char)c;+ }+ }+ }+ calc_in[n] = 0;+ return gt < 2 ? -1 : n;+}++int main(void) {+ for (;;) {+ rv_ecall2(93, 0, 0, 0); // park: HALT; the next GO resumes here+ int n = gather();+ put('\n');+ put(TOOL_RESPONSE);+ put('\n');+ if (n < 0) {+ for (const char *m = "error\n"; *m; m++) put(*m);+ } else {+ calc_eval();+ }+ put(TOOL_RESPONSE_END);+ put('\n');+ put(THINK);+ put('\n');+ put('\n');+ put(THINK_END);+ put('\n');+ }+}+#else+int main(void) { return calc_eval(); }+#endifdiff --git a/experimental/riscv/chipformer/rv/env/calc/gen_sled.py b/experimental/riscv/chipformer/rv/env/calc/gen_sled.pynew file mode 100644index 0000000..c365e6b--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/calc/gen_sled.py@@ -0,0 +1,12 @@+"""Write sled.S: per digit-file register r, BPRD r, 127 .. BPRD r, 0 then ret (bn.h bn_print+jumps in at the top digit to print, one token per digit)."""++import sys++out = [" .section .text", " .globl bn_sleds"]+for r in range(8):+ out += [f"bn_sled{r}:"] + [f" .insn i 0x2B, 0, x{r}, x0, {k}" for k in range(127, -1, -1)]+ out += [" ret"]+out += [" .section .rodata", " .align 2", "bn_sleds:"]+out += [f" .word bn_sled{r}" for r in range(8)]+open(sys.argv[1], "w").write("\n".join(out) + "\n")diff --git a/experimental/riscv/chipformer/rv/env/crt0.S b/experimental/riscv/chipformer/rv/env/crt0.Snew file mode 100644index 0000000..dbec6eb--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/crt0.S@@ -0,0 +1,11 @@+# Minimal C runtime: the host (or qemu) sets sp, and the snapshot holds .data and a zeroed .bss+# (qemu loads them from the ELF); tp points at the TLS block; main's return value goes to exit+# (ecall a7 = 93).+ .section .text.init+ .globl _start+_start:+ .weak __tls_base+ la tp, __tls_base+ call main+ call exit+1: j 1bdiff --git a/experimental/riscv/chipformer/rv/env/mpy/Makefile b/experimental/riscv/chipformer/rv/env/mpy/Makefilenew file mode 100644index 0000000..4b61ad7--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/mpy/Makefile@@ -0,0 +1,60 @@+# MicroPython for the chipformer RV32 machine (RV32IM + Zba, ROM / DROM / RAM map, putchar and+# exit ecalls), based on ports/minimal. Built inside a MicroPython tree by build_mpy.sh:+# make CONFIG=basic SCRIPT=/abs/script.py MAP=rom|qemu BUILD=... [HOT=...] [COPT=...]+CONFIG ?= basic+MAP ?= rom+STACK_KB ?= 6+BUILD ?= build-$(CONFIG)-$(MAP)+include ../../py/mkenv.mk+QSTR_DEFS = qstrdefsport.h+MICROPY_ROM_TEXT_COMPRESSION ?= 1+include $(TOP)/py/py.mk+CROSS_COMPILE = riscv64-unknown-elf-+ENV ?= $(abspath ../../../env)+PICO = /usr/lib/picolibc/riscv64-unknown-elf+MARCH ?= rv32im_zba+ARCH = -march=$(MARCH) -mabi=ilp32+INC += -I. -I$(TOP) -I$(BUILD) -I$(ENV) -isystem $(PICO)/include+CFLAGS += $(INC) -Wall -std=gnu99 $(ARCH) $(COPT) -DRV_CFG_$(CONFIG)=1 -DHEAP_KB=0+CFLAGS += -DSTACK_BYTES=$(shell echo $$(( $(STACK_KB) * 1024 - 512 ))) $(if $(filter qemu,$(MAP)),-DQEMU)+CFLAGS += $(OPTFLAGS) -DNDEBUG -fdata-sections -ffunction-sections+OPTFLAGS ?= -Os+CSUPEROPT = $(OPTFLAGS)+AFLAGS = $(ARCH)+LIBGCC = $(shell $(CC) $(ARCH) -print-libgcc-file-name)+LIBS = -Wl,--start-group $(PICO)/lib/release/rv32im/ilp32/libc.a \+ $(PICO)/lib/release/rv32im/ilp32/libm.a $(LIBGCC) -Wl,--end-group+LD = $(CC)+LDFLAGS = $(ARCH) -nostdlib -nostartfiles -static -Wl,--gc-sections -Wl,-Map=$(BUILD)/firmware.map \+ -T$(BUILD)/mpy.ld+SRC_C = main.c shared/runtime/gchelper_native.c shared/runtime/stdout_helpers.c+SRC_S = shared/runtime/gchelper_rv32i.s+HOT ?=+HOTFLAGS ?= -O3+OBJ += $(PY_CORE_O)+OBJ += $(addprefix $(BUILD)/, $(SRC_C:.c=.o))+OBJ += $(addprefix $(BUILD)/, $(SRC_S:.s=.o))+OBJ += $(BUILD)/crt0.o $(BUILD)/fastlib.o $(BUILD)/sys.o+$(foreach f,$(HOT),$(eval $(BUILD)/$(f).o: CFLAGS += $(HOTFLAGS)))+$(BUILD)/main.o: $(BUILD)/script.h+$(BUILD)/script.h: $(SCRIPT)+ $(Q)mkdir -p $(BUILD)+ $(Q)python3 -c "import sys; s = open(sys.argv[1], 'rb').read(); print('static const char script[] = {' + ','.join(str(b) for b in s) + ',0};')" $< > $@+$(BUILD)/crt0.o: $(ENV)/crt0.S+ $(Q)mkdir -p $(BUILD)+ $(Q)$(CC) $(ARCH) -c -o $@ $<+$(BUILD)/sys.o: $(ENV)/sys.c+ $(Q)mkdir -p $(BUILD)+ $(Q)$(CC) $(ARCH) -O2 -I$(ENV) -isystem $(PICO)/include $(if $(filter qemu,$(MAP)),-DQEMU) -c -o $@ $<+$(BUILD)/fastlib.o: fastlib.c+ $(Q)mkdir -p $(BUILD)+ $(Q)$(CC) $(ARCH) -O2 -fno-builtin -fno-tree-loop-distribute-patterns -c -o $@ $<+$(BUILD)/mpy.ld: $(ENV)/$(MAP).ld+ $(Q)mkdir -p $(BUILD)+ $(Q)sed 's/__heap_end = 0x7000;.*/__heap_end = $(shell printf 0x%x $$(( 0x8000 - $(STACK_KB) * 1024 )));/' $< > $@+all: $(BUILD)/firmware.elf+$(BUILD)/firmware.elf: $(OBJ) $(BUILD)/mpy.ld+ $(ECHO) "LINK $@"+ $(Q)$(LD) $(LDFLAGS) -o $@ $(OBJ) $(LIBS)+ $(Q)$(SIZE) -A $@ | egrep '^\.(text|rodata|data|bss|tdata|tbss)|Total'+include $(TOP)/py/mkrules.mkdiff --git a/experimental/riscv/chipformer/rv/env/mpy/fastlib.c b/experimental/riscv/chipformer/rv/env/mpy/fastlib.cnew file mode 100644index 0000000..a922783--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/mpy/fastlib.c@@ -0,0 +1,70 @@+// Word-at-a-time memcpy/memmove/memset/memcmp/strlen/strcmp/strchr for RV32 (overrides picolibc's byte loops).+// Build with -O2 -fno-builtin -fno-tree-loop-distribute-patterns so gcc does not turn loops back into calls.+#include <stddef.h>+#include <stdint.h>++typedef uint32_t __attribute__((may_alias)) w32;++void *memset(void *d, int c, size_t n) {+ uint8_t *p = d; uint32_t v = (uint8_t)c; v |= v << 8; v |= v << 16;+ while (n && ((uintptr_t)p & 3)) { *p++ = (uint8_t)c; n--; }+ while (n >= 16) { ((w32 *)p)[0] = v; ((w32 *)p)[1] = v; ((w32 *)p)[2] = v; ((w32 *)p)[3] = v; p += 16; n -= 16; }+ while (n >= 4) { *(w32 *)p = v; p += 4; n -= 4; }+ while (n) { *p++ = (uint8_t)c; n--; }+ return d;+}++void *memcpy(void *restrict d, const void *restrict s, size_t n) {+ uint8_t *dp = d; const uint8_t *sp = s;+ if ((((uintptr_t)dp ^ (uintptr_t)sp) & 3) == 0) {+ while (n && ((uintptr_t)dp & 3)) { *dp++ = *sp++; n--; }+ while (n >= 16) { w32 a = ((const w32 *)sp)[0], b = ((const w32 *)sp)[1], c = ((const w32 *)sp)[2], e = ((const w32 *)sp)[3];+ ((w32 *)dp)[0] = a; ((w32 *)dp)[1] = b; ((w32 *)dp)[2] = c; ((w32 *)dp)[3] = e; dp += 16; sp += 16; n -= 16; }+ while (n >= 4) { *(w32 *)dp = *(const w32 *)sp; dp += 4; sp += 4; n -= 4; }+ }+ while (n) { *dp++ = *sp++; n--; }+ return d;+}++void *memmove(void *d, const void *s, size_t n) {+ uint8_t *dp = d; const uint8_t *sp = s;+ if (dp == sp || n == 0) return d;+ if (dp < sp || dp >= sp + n) return memcpy(d, s, n);+ dp += n; sp += n;+ if ((((uintptr_t)dp ^ (uintptr_t)sp) & 3) == 0) {+ while (n && ((uintptr_t)dp & 3)) { *--dp = *--sp; n--; }+ while (n >= 4) { dp -= 4; sp -= 4; *(w32 *)dp = *(const w32 *)sp; n -= 4; }+ }+ while (n) { *--dp = *--sp; n--; }+ return d;+}++int memcmp(const void *a, const void *b, size_t n) {+ const uint8_t *p = a, *q = b;+ if ((((uintptr_t)p | (uintptr_t)q) & 3) == 0) {+ while (n >= 4 && *(const w32 *)p == *(const w32 *)q) { p += 4; q += 4; n -= 4; }+ }+ while (n) { if (*p != *q) return *p - *q; p++; q++; n--; }+ return 0;+}++#define HASZERO(v) (((v) - 0x01010101u) & ~(v) & 0x80808080u)+size_t strlen(const char *s) {+ const char *p = s;+ while ((uintptr_t)p & 3) { if (!*p) return p - s; p++; }+ while (!HASZERO(*(const w32 *)p)) p += 4;+ while (*p) p++;+ return p - s;+}++int strcmp(const char *a, const char *b) {+ if ((((uintptr_t)a | (uintptr_t)b) & 3) == 0) {+ while (1) { uint32_t x = *(const w32 *)a, y = *(const w32 *)b; if (x != y || HASZERO(x)) break; a += 4; b += 4; }+ }+ while (*a && *a == *b) { a++; b++; }+ return (uint8_t)*a - (uint8_t)*b;+}++char *strchr(const char *s, int c) {+ for (;; s++) { if (*s == (char)c) return (char *)s; if (!*s) return NULL; }+}diff --git a/experimental/riscv/chipformer/rv/env/mpy/main.c b/experimental/riscv/chipformer/rv/env/mpy/main.cnew file mode 100644index 0000000..7e07f62--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/mpy/main.c@@ -0,0 +1,57 @@+// MicroPython on the chipformer RV32 machine: boot, run the script compiled into the firmware+// (script.h, in the DROM), print through putchar ecalls, exit with 0 (or 1 on an exception).+#include <stdint.h>+#include <string.h>+#include "py/builtin.h"+#include "py/compile.h"+#include "py/runtime.h"+#include "py/gc.h"+#include "py/cstack.h"+#include "py/mperrno.h"+#include "shared/runtime/gchelper.h"+#include "rt.h"+#include "script.h" // static const char script[]++extern char __heap_start[], __heap_end[];++static int run_script(const char *src, size_t len) {+ nlr_buf_t nlr;+ if (nlr_push(&nlr) == 0) {+ mp_lexer_t *lex = mp_lexer_new_from_str_len(MP_QSTR__lt_stdin_gt_, src, len, 0);+ qstr source_name = lex->source_name;+ mp_parse_tree_t parse_tree = mp_parse(lex, MP_PARSE_FILE_INPUT);+ mp_obj_t module_fun = mp_compile(&parse_tree, source_name, false);+ mp_call_function_0(module_fun);+ nlr_pop();+ return 0;+ }+ mp_obj_print_exception(&mp_plat_print, (mp_obj_t)nlr.ret_val);+ return 1;+}++int main(void) {+ mp_cstack_init_with_sp_here(STACK_BYTES);+ gc_init(__heap_start, __heap_end);+ mp_init();+ int rc = run_script(script, sizeof(script) - 1);+ mp_deinit();+ return rc;+}++void gc_collect(void) {+ gc_collect_start();+ gc_helper_collect_regs_and_stack();+ gc_collect_end();+}++int mp_hal_stdin_rx_chr(void) { return 4; }++mp_uint_t mp_hal_stdout_tx_strn(const char *str, mp_uint_t len) {+ for (mp_uint_t i = 0; i < len; i++) rv_putchar(str[i]);+ return len;+}++mp_lexer_t *mp_lexer_new_from_file(qstr filename) { mp_raise_OSError(MP_ENOENT); }+mp_import_stat_t mp_import_stat(const char *path) { return MP_IMPORT_STAT_NO_EXIST; }+void nlr_jump_fail(void *val) { for (;;) rv_ecall2(93, 3, 0, 0); }+void MP_NORETURN __fatal_error(const char *msg) { for (;;) rv_ecall2(93, 4, 0, 0); }diff --git a/experimental/riscv/chipformer/rv/env/mpy/mpconfigport.h b/experimental/riscv/chipformer/rv/env/mpy/mpconfigport.hnew file mode 100644index 0000000..fafd7e8--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/mpy/mpconfigport.h@@ -0,0 +1,93 @@+#include <stdint.h>+#if defined(RV_CFG_min)+#define MICROPY_CONFIG_ROM_LEVEL (MICROPY_CONFIG_ROM_LEVEL_MINIMUM)+#define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_NONE)+#elif defined(RV_CFG_basic)+#define MICROPY_CONFIG_ROM_LEVEL (MICROPY_CONFIG_ROM_LEVEL_BASIC_FEATURES)+#define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_DOUBLE)+#elif defined(RV_CFG_extra)+#ifndef MICROPY_PY_UCTYPES+#define MICROPY_PY_UCTYPES (0)+#endif+#define MICROPY_CONFIG_ROM_LEVEL (MICROPY_CONFIG_ROM_LEVEL_EXTRA_FEATURES)+#define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_DOUBLE)+#endif+#ifdef RV_NOCOMP+#ifndef MICROPY_ENABLE_COMPILER+#define MICROPY_ENABLE_COMPILER (0)+#endif+#else+#ifndef MICROPY_ENABLE_COMPILER+#define MICROPY_ENABLE_COMPILER (1)+#endif+#endif+#ifndef MICROPY_ENABLE_GC+#define MICROPY_ENABLE_GC (1)+#endif+#ifndef MICROPY_HELPER_REPL+#define MICROPY_HELPER_REPL (1)+#endif+#ifndef MICROPY_LONGINT_IMPL+#define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_MPZ)+#endif+#ifndef MICROPY_ENABLE_EXTERNAL_IMPORT+#define MICROPY_ENABLE_EXTERNAL_IMPORT (0)+#endif+#ifndef MICROPY_ALLOC_PATH_MAX+#define MICROPY_ALLOC_PATH_MAX (256)+#endif+#ifndef MICROPY_ALLOC_PARSE_CHUNK_INIT+#define MICROPY_ALLOC_PARSE_CHUNK_INIT (16)+#endif+#ifndef MICROPY_PY_SYS_MODULES+#define MICROPY_PY_SYS_MODULES (0)+#endif+#ifndef MICROPY_PY_SYS_EXIT+#define MICROPY_PY_SYS_EXIT (0)+#endif+#ifndef MICROPY_PY_SYS_PATH+#define MICROPY_PY_SYS_PATH (0)+#endif+#ifndef MICROPY_PY_SYS_ARGV+#define MICROPY_PY_SYS_ARGV (0)+#endif+#ifndef MICROPY_PY_SYS_STDFILES+#define MICROPY_PY_SYS_STDFILES (0)+#endif+#ifndef MICROPY_PY_OS+#define MICROPY_PY_OS (0)+#endif+#ifndef MICROPY_PY_TIME+#define MICROPY_PY_TIME (0)+#endif+#ifndef MICROPY_VFS+#define MICROPY_VFS (0)+#endif+#ifndef MICROPY_READER_VFS+#define MICROPY_READER_VFS (0)+#endif+#ifndef MICROPY_PY_IO+#define MICROPY_PY_IO (0)+#endif+#ifndef MICROPY_PY_SELECT+#define MICROPY_PY_SELECT (0)+#endif+#ifndef MICROPY_PY_MACHINE+#define MICROPY_PY_MACHINE (0)+#endif+#ifndef MICROPY_PY_RANDOM+#define MICROPY_PY_RANDOM (0)+#endif+#ifndef MICROPY_GCREGS_SETJMP+#define MICROPY_GCREGS_SETJMP (0)+#endif+#define MP_SSIZE_MAX (0x7fffffff)+#ifndef MICROPY_PERSISTENT_CODE_LOAD+#define MICROPY_PERSISTENT_CODE_LOAD (1)+#endif+typedef long mp_off_t;+#include <alloca.h>+#define MICROPY_HW_BOARD_NAME "chipformer"+#define MICROPY_HW_MCU_NAME "rv32im_zba"+#define MICROPY_HEAP_SIZE (HEAP_KB * 1024)+#define MP_STATE_PORT MP_STATE_VMdiff --git a/experimental/riscv/chipformer/rv/env/mpy/mphalport.h b/experimental/riscv/chipformer/rv/env/mpy/mphalport.hnew file mode 100644index 0000000..a909c22--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/mpy/mphalport.h@@ -0,0 +1,2 @@+static inline mp_uint_t mp_hal_ticks_ms(void) { return 0; }+static inline void mp_hal_set_interrupt_char(char c) { (void)c; }diff --git a/experimental/riscv/chipformer/rv/env/mpy/qstrdefsport.h b/experimental/riscv/chipformer/rv/env/mpy/qstrdefsport.hnew file mode 100644index 0000000..00d3e2a--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/mpy/qstrdefsport.h@@ -0,0 +1,2 @@+// qstrs specific to this port+// *FORMAT-OFF*diff --git a/experimental/riscv/chipformer/rv/env/mpy/scripts/s1.py b/experimental/riscv/chipformer/rv/env/mpy/scripts/s1.pynew file mode 100644index 0000000..b789000--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/mpy/scripts/s1.py@@ -0,0 +1 @@+print(2**100)diff --git a/experimental/riscv/chipformer/rv/env/mpy/scripts/s5.py b/experimental/riscv/chipformer/rv/env/mpy/scripts/s5.pynew file mode 100644index 0000000..5a0d970--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/mpy/scripts/s5.py@@ -0,0 +1,18 @@+def is_prime(n):+ if n < 2:+ return False+ i = 2+ while i * i <= n:+ if n % i == 0:+ return False+ i += 1+ return True++words = "the quick brown fox jumps over the lazy dog".split()+counts = {}+for w in words:+ counts[w] = counts.get(w, 0) + 1+primes = [n for n in range(50) if is_prime(n)]+print(primes)+print(sorted(counts.items()))+print(max(words, key=len))diff --git a/experimental/riscv/chipformer/rv/env/progs/crc32.c b/experimental/riscv/chipformer/rv/env/progs/crc32.cnew file mode 100644index 0000000..69c9032--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/crc32.c@@ -0,0 +1,10 @@+static const char msg[] = "The quick brown fox jumps over the lazy dog";++int main(void) {+ unsigned crc = 0xFFFFFFFFu;+ for (const char *p = msg; *p; p++) {+ crc ^= (unsigned char)*p;+ for (int k = 0; k < 8; k++) crc = (crc >> 1) ^ (0xEDB88320u & -(crc & 1u));+ }+ return (int)~crc; /* 0x414FA339 */+}diff --git a/experimental/riscv/chipformer/rv/env/progs/crc32p.c b/experimental/riscv/chipformer/rv/env/progs/crc32p.cnew file mode 100644index 0000000..e14f5eb--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/crc32p.c@@ -0,0 +1,12 @@+#include <stdio.h>+static const char msg[] = "The quick brown fox jumps over the lazy dog";++int main(void) {+ unsigned crc = 0xFFFFFFFFu;+ for (const char *p = msg; *p; p++) {+ crc ^= (unsigned char)*p;+ for (int k = 0; k < 8; k++) crc = (crc >> 1) ^ (0xEDB88320u & -(crc & 1u));+ }+ printf("crc32=%08x\n", ~crc); /* 414fa339 */+ return 0;+}diff --git a/experimental/riscv/chipformer/rv/env/progs/fib.c b/experimental/riscv/chipformer/rv/env/progs/fib.cnew file mode 100644index 0000000..96df207--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/fib.c@@ -0,0 +1,9 @@+#include <stdio.h>++static int fib(int n) { return n < 2 ? n : fib(n - 1) + fib(n - 2); }++int main(void) {+ volatile int n = 25;+ printf("fib(25)=%d\n", fib(n));+ return 0;+}diff --git a/experimental/riscv/chipformer/rv/env/progs/gmpmul.c b/experimental/riscv/chipformer/rv/env/progs/gmpmul.cnew file mode 100644index 0000000..a2401f2--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/gmpmul.c@@ -0,0 +1,17 @@+/* mini-gmp: parse two 20-digit numbers, multiply them, print the 40-digit product. */+#include <stdio.h>+#include "mini-gmp.h"+static char buf[64];++int main(void) {+ mpz_t a, b, c;+ mpz_init(a);+ mpz_init(b);+ mpz_init(c);+ mpz_set_str(a, "12345678901234567890", 10);+ mpz_set_str(b, "98765432109876543211", 10);+ mpz_mul(c, a, b);+ mpz_get_str(buf, 10, c);+ printf("mul=%s\n", buf);+ return 0;+}diff --git a/experimental/riscv/chipformer/rv/env/progs/qsort.c b/experimental/riscv/chipformer/rv/env/progs/qsort.cnew file mode 100644index 0000000..078115a--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/qsort.c@@ -0,0 +1,28 @@+#include <stdio.h>+#include <stdlib.h>+#define N 1000+static int a[N];+static unsigned s = 12345;++static int rnd(void) {+ s = s * 1103515245u + 12345u;+ return (s >> 8) & 0xffffff;+}++static int cmp(const void *x, const void *y) {+ int p = *(const int *)x, q = *(const int *)y;+ return (p > q) - (p < q);+}++int main(void) {+ for (int i = 0; i < N; i++) a[i] = rnd();+ qsort(a, N, sizeof a[0], cmp);+ long err = 0;+ unsigned h = 0;+ for (int i = 0; i < N; i++) {+ if (i && a[i - 1] > a[i]) err++;+ h = h * 31u + (unsigned)a[i];+ }+ printf("qsort 1000: err=%ld min=%d max=%d hash=%08x\n", err, a[0], a[N - 1], h);+ return 0;+}diff --git a/experimental/riscv/chipformer/rv/env/progs/sieve.c b/experimental/riscv/chipformer/rv/env/progs/sieve.cnew file mode 100644index 0000000..8e8a613--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/sieve.c@@ -0,0 +1,11 @@+static unsigned char comp[2000];++int main(void) {+ int count = 0;+ for (int i = 2; i < 2000; i++) {+ if (comp[i]) continue;+ count++;+ for (int j = i + i; j < 2000; j += i) comp[j] = 1;+ }+ return count; /* 303 */+}diff --git a/experimental/riscv/chipformer/rv/env/progs/sieve4.c b/experimental/riscv/chipformer/rv/env/progs/sieve4.cnew file mode 100644index 0000000..7679c14--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/sieve4.c@@ -0,0 +1,15 @@+#include <stdio.h>+#define N 10000+static unsigned char comp[N];++int main(void) {+ int count = 0;+ for (int i = 2; i < N; i++) {+ if (!comp[i]) {+ count++;+ for (int j = i * i; j < N; j += i) comp[j] = 1;+ }+ }+ printf("primes below %d: %d\n", N, count);+ return 0;+}diff --git a/experimental/riscv/chipformer/rv/env/progs/sort.c b/experimental/riscv/chipformer/rv/env/progs/sort.cnew file mode 100644index 0000000..f04bec9--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/sort.c@@ -0,0 +1,23 @@+static int a[64];++int main(void) {+ unsigned x = 12345;+ for (int i = 0; i < 64; i++) {+ x = x * 1103515245u + 12345u; /* libgcc __mulsi3 on rv32i */+ a[i] = (int)(x >> 8) - (1 << 22);+ }+ for (int i = 1; i < 64; i++) {+ int v = a[i], j = i - 1;+ while (j >= 0 && a[j] > v) {+ a[j + 1] = a[j];+ j--;+ }+ a[j + 1] = v;+ }+ unsigned h = 0;+ for (int i = 0; i < 64; i++) {+ if (i && a[i - 1] > a[i]) return 255;+ h = (h << 5) - h + (unsigned)a[i];+ }+ return (int)h;+}diff --git a/experimental/riscv/chipformer/rv/env/progs/strops.c b/experimental/riscv/chipformer/rv/env/progs/strops.cnew file mode 100644index 0000000..81f83e3--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/strops.c@@ -0,0 +1,19 @@+static char buf[64];+static const char src[] = "risc-v inside qwen, exactly";++static int len(const char *s) {+ int n = 0;+ while (s[n]) n++;+ return n;+}++int main(void) {+ int n = len(src);+ for (int i = 0; i < n; i++) buf[i] = src[n - 1 - i];+ buf[n] = 0;+ unsigned short h[8] = {0};+ for (int i = 0; i < n; i++) h[i & 7] = (unsigned short)((h[i & 7] << 3) - h[i & 7] + (unsigned char)buf[i]);+ int s = 0;+ for (int i = 0; i < 8; i++) s += (short)h[i];+ return s + n;+}diff --git a/experimental/riscv/chipformer/rv/env/progs/sudoku.c b/experimental/riscv/chipformer/rv/env/progs/sudoku.cnew file mode 100644index 0000000..075c6e6--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/progs/sudoku.c@@ -0,0 +1,39 @@+#include <stdio.h>+static int g[81];+static long steps;++static int ok(int p, int v) {+ int r = p / 9, c = p % 9, br = r / 3 * 3, bc = c / 3 * 3;+ for (int i = 0; i < 9; i++) {+ if (g[r * 9 + i] == v || g[i * 9 + c] == v) return 0;+ if (g[(br + i / 3) * 9 + bc + i % 3] == v) return 0;+ }+ return 1;+}++static int solve(int p) {+ while (p < 81 && g[p]) p++;+ if (p == 81) return 1;+ for (int v = 1; v <= 9; v++) {+ steps++;+ if (ok(p, v)) {+ g[p] = v;+ if (solve(p + 1)) return 1;+ g[p] = 0;+ }+ }+ return 0;+}++/* the Wikipedia example puzzle */+static const char puz[] =+ "53..7....6..195....98....6.8...6...34..8.3..17...2...6.6....28....419..5....8..79";++int main(void) {+ for (int i = 0; i < 81; i++) g[i] = puz[i] == '.' ? 0 : puz[i] - '0';+ int s = solve(0);+ printf("solved=%d steps=%ld row0=", s, steps);+ for (int i = 0; i < 9; i++) putchar('0' + g[i]);+ putchar('\n');+ return 0;+}diff --git a/experimental/riscv/chipformer/rv/env/qemu.ld b/experimental/riscv/chipformer/rv/env/qemu.ldnew file mode 100644index 0000000..f45e37f--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/qemu.ld@@ -0,0 +1,26 @@+/* The same layout at addresses qemu-riscv32 (user mode) can map (the heap inside .bss, so it is+ mapped too). */+ENTRY(_start)+SECTIONS+{+ . = 0x100000;+ .text : { *(.text.init) *(.text .text.*) }+ . = 0x180000;+ .rodata : {+ *(.rodata .rodata.* .srodata .srodata.*)+ *(.init_array .init_array.* .fini_array .fini_array.* .eh_frame .eh_frame_hdr)+ }+ . = 0x200000;+ .data : { *(.data .data.* .sdata .sdata.*) }+ .tdata : { __tls_base = .; *(.tdata .tdata.*) }+ .tbss : { *(.tbss .tbss.*) }+ .bss : {+ . = . + SIZEOF(.tbss);+ *(.bss .bss.* .sbss .sbss.*) *(COMMON)+ . = ALIGN(16);+ __heap_start = .;+ . = . + 0x10000;+ __heap_end = .;+ }+ _end = .;+}diff --git a/experimental/riscv/chipformer/rv/env/riscv_test.h b/experimental/riscv/chipformer/rv/env/riscv_test.hnew file mode 100644index 0000000..d52103b--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/riscv_test.h@@ -0,0 +1,18 @@+// riscv-tests environment for the chipformer RV32 machine: no CSRs, no traps; a test ends with+// the Linux-style exit ecall (a7 = 93, a0 = 0 on pass, the failing TESTNUM otherwise), which+// qemu-riscv32 (user mode) and the circuit both understand.+#ifndef _ENV_CHIPFORMER_RV_H+#define _ENV_CHIPFORMER_RV_H++#define RVTEST_RV64U .macro init; .endm+#define RVTEST_RV32U .macro init; .endm+#define TESTNUM gp+#define RVTEST_CODE_BEGIN .text; .globl _start; _start: init;+#define RVTEST_CODE_END unimp+#define RVTEST_PASS li a0, 0; li a7, 93; ecall+#define RVTEST_FAIL mv a0, TESTNUM; li a7, 93; ecall+#define RVTEST_DATA_BEGIN .data; .align 4; .global begin_signature; begin_signature:+#define RVTEST_DATA_END .align 4; .global end_signature; end_signature:+#define EXTRA_DATA++#endifdiff --git a/experimental/riscv/chipformer/rv/env/rom.ld b/experimental/riscv/chipformer/rv/env/rom.ldnew file mode 100644index 0000000..3ff589f--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/rom.ld@@ -0,0 +1,29 @@+/* The chipformer RV32 address map (P2): RAM at 0 (data / TLS / bss / heap from 0x100, the stack+ below 0x8000), read-only data in the DROM at 0x20000 (128 KB), code in the ROM at 0x40000+ (256 KB). crt0 points tp at __tls_base (the single thread's TLS block: .tdata then .tbss). */+ENTRY(_start)+MEMORY+{+ RAM (rw) : ORIGIN = 0x100, LENGTH = 0x7F00+ DROM (r) : ORIGIN = 0x20000, LENGTH = 128K+ ROM (rx) : ORIGIN = 0x40000, LENGTH = 256K+}+SECTIONS+{+ .text : { *(.text.init) *(.text .text.*) } > ROM+ .rodata : {+ *(.rodata .rodata.* .srodata .srodata.*)+ *(.init_array .init_array.* .fini_array .fini_array.* .eh_frame .eh_frame_hdr)+ } > DROM+ .data : { *(.data .data.* .sdata .sdata.*) } > RAM+ .tdata : { __tls_base = .; *(.tdata .tdata.*) } > RAM+ .tbss : { *(.tbss .tbss.*) } > RAM+ .bss : {+ . = . + SIZEOF(.tbss); /* (.tbss takes no address space of its own) */+ *(.bss .bss.* .sbss .sbss.*) *(COMMON)+ . = ALIGN(16);+ __heap_start = .;+ } > RAM+ __heap_end = 0x7000; /* 4 KB of stack above the heap */+ _end = __heap_start;+}diff --git a/experimental/riscv/chipformer/rv/env/rt.h b/experimental/riscv/chipformer/rv/env/rt.hnew file mode 100644index 0000000..22d14bd--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/rt.h@@ -0,0 +1,20 @@+// System calls of the chipformer RV32 machine: putchar is ecall a7 = 1, exit is a7 = 93.+// Built with -DQEMU, putchar becomes write(1, &c, 1) (a7 = 64) for qemu-riscv32 user mode.+#pragma once++static inline long rv_ecall2(long n, long a0, long a1, long a2) {+ register long r_a0 __asm__("a0") = a0, r_a1 __asm__("a1") = a1, r_a2 __asm__("a2") = a2;+ register long r_a7 __asm__("a7") = n;+ __asm__ volatile("ecall" : "+r"(r_a0) : "r"(r_a1), "r"(r_a2), "r"(r_a7) : "memory");+ return r_a0;+}++static inline void rv_putchar(char c) {+#ifdef QEMU+ rv_ecall2(64, 1, (long)&c, 1);+#else+ rv_ecall2(1, (unsigned char)c, 0, 0);+#endif+}++static inline void rv_mark(int id) { (void)id; }diff --git a/experimental/riscv/chipformer/rv/env/sys.c b/experimental/riscv/chipformer/rv/env/sys.cnew file mode 100644index 0000000..b2624b5--- /dev/null+++ b/experimental/riscv/chipformer/rv/env/sys.c@@ -0,0 +1,19 @@+// picolibc stdio and exit on the chipformer RV32 machine.+#include <stdio.h>+#include <stdlib.h>+#include "rt.h"++static int rv_putc(char c, FILE *f) {+ (void)f;+ rv_putchar(c);+ return (unsigned char)c;+}++static FILE __stdio = FDEV_SETUP_STREAM(rv_putc, NULL, NULL, _FDEV_SETUP_WRITE);+FILE *const stdin = &__stdio;+FILE *const stdout = &__stdio;+FILE *const stderr = &__stdio;++void _exit(int code) {+ for (;;) rv_ecall2(93, code, 0, 0);+}diff --git a/experimental/riscv/chipformer/rv/evalarith.py b/experimental/riscv/chipformer/rv/evalarith.pynew file mode 100644index 0000000..34f2190--- /dev/null+++ b/experimental/riscv/chipformer/rv/evalarith.py@@ -0,0 +1,149 @@+"""Arithmetic evals of the rv1 checkpoint: Qwen with the RV32 calc tool inside its weights.++ python -m chipformer.rv.evalarith MODEL --data arith --n 40 --out DIR+ python -m chipformer.rv.evalarith BASE --cond pytool --data dmx_mul_big --n 100 --out DIR+ python -m chipformer.rv.evalarith BASE --cond base_nudge --data dmx_mul_big --out DIR++Problems: ``arith`` is q35.evals' seeded synthetic set (the CHIP-8 runs' ids); any other name is+a q35.arith_bench dataset (DeepMind modules, NUPA, ...), same seeds. The prompt is a system nudge+(the calc tool, or exact arithmetic by hand without a tool), the question and evals' boxed-answer+suffix; scoring is arith_bench's exact rational match. Conditions:++ rv1 the merged checkpoint; the machine answers calc calls (lockstep-checked against the+ reference machine; the token ids are kept for rv/replay.py)+ pytool base Qwen3.5 with the same tool, its responses computed by the host in the+ firmware's exact format (calc.expected: // floor, %, **, gcd)+ base_nudge base Qwen3.5 without tools++Qwen writes at most ``--budget`` tokens (4096). Records are appended per problem (resumable):+DIR/<data>.<cond>.jsonl.+"""++import argparse+import json+import re+import time+from pathlib import Path++import torch++from chipformer.q35 import arith_bench as AB+from chipformer.q35 import evals as E+from chipformer.rv import bridge as BR+from chipformer.rv import calc, elf, ref+from chipformer.rv.chat import CALC_TOOL, RvChat++NUDGE = ("You have a calc tool for exact arithmetic. Use it for every arithmetic step instead "+ "of calculating by hand.")+NUDGE_NONE = "Do every arithmetic step exactly."+++def calc_response(expr: str) -> str:+ """The firmware's tool response for an expression (computed by the host)."""+ e = expr.replace("\n", "")+ v = calc.expected(e) if re.fullmatch(r"[0-9.+\-*/%^(), gcd]*", e) and e.strip() else "error\n"+ return f"\n<tool_response>\n{v}</tool_response>\n<think>\n\n</think>\n\n"+++def problems(data: str, n: int, seed: int) -> list[dict]:+ return E.load_data("arith", n, seed) if data == "arith" else AB.load(data, n, seed)++[email protected]_grad()+def run_base(model, tok, ids, tool: bool, budget: int):+ """Greedy decoding of base Qwen; with ``tool`` the host answers calc calls."""+ dev = next(model.parameters()).device+ tcc = tok.convert_tokens_to_ids("</tool_call>")+ eos = {tok.convert_tokens_to_ids("<|im_end|>"), tok.convert_tokens_to_ids("<|endoftext|>")}+ seq, gen, own, calls = list(ids), [], 0, 0+ while own < budget:+ out = model.generate(torch.tensor([seq], device=dev), max_new_tokens=budget - own,+ do_sample=False, eos_token_id=sorted(eos | ({tcc} if tool else set())))+ new = out[0, len(seq):].tolist()+ gen += new+ seq += new+ own += len(new)+ if not new or new[-1] in eos or not tool or new[-1] != tcc:+ break+ txt = tok.decode(gen, skip_special_tokens=False)+ m = re.findall(r"<parameter=[^>]*>(.*?)</parameter>", txt.split("<tool_call>")[-1], re.S)+ resp = tok(calc_response(m[0] if m else ""), add_special_tokens=False).input_ids+ gen += resp+ seq += resp+ calls += 1+ return gen, calls, own+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("model")+ ap.add_argument("--cond", default="rv1", choices=("rv1", "pytool", "base_nudge"))+ ap.add_argument("--data", default="arith")+ ap.add_argument("--n", type=int, default=None)+ ap.add_argument("--seed", type=int, default=0)+ ap.add_argument("--out", required=True)+ ap.add_argument("--firmware", default="tests/data/rv/calc-bridge.elf")+ ap.add_argument("--budget", type=int, default=4096, help="Qwen tokens per problem")+ ap.add_argument("--max-chip", type=int, default=200000)+ args = ap.parse_args()+ n = args.n or (40 if args.data == "arith" else None)+ probs = problems(args.data, n, args.seed)+ out = Path(args.out) / f"{args.data}.{args.cond}.jsonl"+ out.parent.mkdir(parents=True, exist_ok=True)+ done = {json.loads(x)["id"] for x in out.read_text().splitlines()} if out.exists() else set()+ tool = args.cond != "base_nudge"+ if args.cond == "rv1":+ chat = RvChat(args.model)+ tok = chat.tok+ else:+ from transformers import AutoModelForImageTextToText, AutoTokenizer++ tok = AutoTokenizer.from_pretrained(args.model)+ model = AutoModelForImageTextToText.from_pretrained(args.model, dtype=torch.bfloat16)+ model = model.to("cuda").eval()+ for p in probs:+ if p["id"] in done:+ continue+ msgs = [{"role": "system", "content": NUDGE if tool else NUDGE_NONE},+ {"role": "user", "content": p["question"] + E.SUFFIX}]+ t0 = time.time()+ rec = {"id": p["id"], "gold": p["gold"], "question": p["question"], "cond": args.cond}+ if args.cond == "rv1":+ ids = chat.prompt_ids(msgs)+ m = ref.Machine.from_program(elf.program(args.firmware))+ BR.park(m)+ m2 = ref.Machine.from_program(elf.program(args.firmware))+ BR.park(m2)+ lock = BR.Session(chat.T, m2) # every circuit token checked against the reference+ lock.history(ids)+ r = chat.generate(ids, m, args.budget, max_chip=args.max_chip, check_head=True,+ lock=lock)+ gen = r["tokens"]+ text = chat.text_of(gen)+ outs = [chat.T.decode_bytes(e).decode("utf-8", "replace") for e in lock.runs]+ rec.update(prompt_ids=ids, gen_ids=gen, calls=len(lock.runs),+ n_qwen=r["text_tokens"] - sum(len(e) for e in lock.runs),+ n_chip=r["chip_tokens"], lock_error=r["lock_error"],+ head_mismatch=r["head_mismatch"], gen_seconds=round(r["seconds"], 1),+ results=[o.split("\n")[2] if o.count("\n") > 2 else o for o in outs])+ else:+ ids = tok.apply_chat_template(msgs, tools=[CALC_TOOL] if tool else None,+ add_generation_prompt=True, enable_thinking=False,+ tokenize=True)+ ids = ids if isinstance(ids, list) else ids["input_ids"]+ gen, calls, own = run_base(model, tok, ids, tool, args.budget)+ text = tok.decode(gen, skip_special_tokens=False)+ rec.update(calls=calls, n_qwen=own, n_chip=0)+ rec.update(AB.exact_fields(p, text))+ rec.update(seconds=round(time.time() - t0, 1), text=text)+ with out.open("a", encoding="utf-8") as f:+ f.write(json.dumps(rec, ensure_ascii=False) + "\n")+ keys = ("id", "gold", "pred_exact", "exact", "calls", "n_qwen", "n_chip", "seconds",+ "lock_error", "head_mismatch")+ print(json.dumps({k: rec[k] for k in keys if k in rec}), flush=True)+ rows = [json.loads(x) for x in out.read_text().splitlines()]+ print(f"{out.name}: {sum(r['exact'] for r in rows)}/{len(rows)} exact")+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/fastexec.py b/experimental/riscv/chipformer/rv/fastexec.pynew file mode 100644index 0000000..26c82f1--- /dev/null+++ b/experimental/riscv/chipformer/rv/fastexec.py@@ -0,0 +1,447 @@+"""A compact executor for a q35 circuit graph: the circuit's arithmetic on its live structure+only, batch rows, one decode step per token, captured as a CUDA graph.++Per sublayer, all of a layer's heads run as batched tensors:++ q, k, v (lagged inputs) @ W, silu == relu, with every matrix cut to the input dims it reads+ and the output columns it writes; one matmul per sublayer+ decay the state times 1 / 0.5 / 0, only for the value heads that ever decay+ write keys are one-hot: one row scatter per value head (where beta and a key are set)+ read queries are at most k-hot (2, or a head's ``qhot``): a top-k row gather, normalized+ by the ballast columns+ MLP relu(x[R] @ Wb) * (x[R] @ Wa) @ Wd over the layer's units, written to its dims++This is the IR's semantics in fp32 (every value the circuit produces is exact there), not the+HF modules; ``check`` compares it with ir.evaluate on every dim. ``FreeRun`` lets it generate:+each step decodes the output code into the next token and builds that token's embedding on the+GPU, so a whole run is replays of one graph.+"""++import time++import numpy as np+import torch++from chipformer.dsl.graph import ONE+from chipformer.q35.ir import FFN, HD, REF_COLS, Head+++class Lin:+ """y = x @ W for a W [L, D, N] over lagged inputs xs [B, L, D], reading only W's nonzero+ block: the input dims each lag uses and the output columns any lag writes."""++ def __init__(self, w: np.ndarray, device):+ w = w if w.ndim == 3 else w[None]+ self.n = w.shape[2]+ cols = np.nonzero(np.abs(w).sum((0, 1)))[0]+ rows = [np.nonzero(np.abs(w[lag]).sum(1))[0] for lag in range(w.shape[0])]+ self.lags = [lag for lag in range(w.shape[0]) if len(rows[lag])]+ self.rows = [torch.tensor(rows[lag], device=device) for lag in self.lags]+ blocks = [w[lag][rows[lag]][:, cols] for lag in self.lags]+ self.w = torch.tensor(np.concatenate(blocks) if blocks else np.zeros((0, len(cols))),+ dtype=torch.float32, device=device)+ self.cols = torch.tensor(cols, device=device)+ self.empty = not len(cols) or not self.lags++ def __call__(self, xs):+ """xs: [B, L, D] (lag 0 first); returns [B, N]."""+ y = torch.zeros(xs.shape[0], self.n, device=xs.device)+ if self.empty:+ return y+ xin = torch.cat([xs[:, lag][:, r] for lag, r in zip(self.lags, self.rows)], 1)+ y[:, self.cols] = xin @ self.w+ return y+++class SparseLin:+ """Lin from its nonzeros only: y[:, n] = sum of w * xs[:, lag, d] (a gather, a product and a+ scatter-add; the sums are of exactly representable terms, so any order is exact)."""++ def __init__(self, w: np.ndarray, device):+ w = w if w.ndim == 3 else w[None]+ self.n, self.D = w.shape[2], w.shape[1]+ lag, d, n = np.nonzero(w)+ self.lags = sorted(set(lag.tolist()))+ self.empty = not len(n)+ self.src = torch.tensor(lag * self.D + d, device=device)+ self.dst = torch.tensor(n, device=device)+ self.w = torch.tensor(w[lag, d, n], dtype=torch.float32, device=device)+ self.nl = (max(self.lags) + 1) if self.lags else 1++ def __call__(self, xs):+ """xs: [B, L, D] (lag 0 first); returns [B, N]."""+ y = torch.zeros(xs.shape[0], self.n, device=xs.device)+ if self.empty:+ return y+ flat = xs[:, :self.nl].reshape(xs.shape[0], -1)+ return y.index_add_(1, self.dst, flat[:, self.src] * self.w)+++def _mat(exprs, n_dims):+ w = np.zeros((n_dims, len(exprs)))+ for i, e in enumerate(exprs):+ for d, cf in e.terms.items():+ w[d, i] += float(cf)+ return w+++class Executor:+ def __init__(self, c, device="cuda", batch: int = 1, sparse: bool = False):+ torch.backends.cuda.matmul.allow_tf32 = False # fp32 products must stay exact+ self.c, self.dev, self.B = c, device, batch+ Lin_ = SparseLin if sparse else Lin+ D = len(c.dims)+ self.D = D+ self.steps = []+ for lv in sorted({n.level for n in c.nodes}):+ nodes = [n for n in c.nodes if n.level == lv]+ if isinstance(nodes[0], FFN):+ terms = [(cf, a, b, n.out) for n in nodes for cf, a, b in n.terms]+ wd = np.zeros((len(terms), D))+ for u, (cf, _, _, out) in enumerate(terms):+ wd[u, out] = float(cf)+ wab = np.concatenate([_mat([a for _, a, _, _ in terms], D),+ _mat([b for _, _, b, _ in terms], D)], 1)+ self.steps.append(("ffn", Lin_(wab, device), len(terms), Lin_(wd, device)))+ continue+ heads = [n for n in nodes if isinstance(n, Head)]+ H = len(heads)+ vh = [(h, vd) for h, n in enumerate(heads) for vd in n.vheads()]+ V = len(vh)+ wq, wk = np.zeros((4, D, H * HD)), np.zeros((4, D, H * HD))+ wv = np.zeros((4, D, V * HD))+ wa_, wbeta, wz = np.zeros((D, V)), np.zeros((D, V)), np.zeros((D, V * HD))+ bconst = np.zeros(V)+ wo = np.zeros((V * HD, D))+ for h, n in enumerate(heads):+ for chans, w_ in ((n.q, wq), (n.k, wk)):+ for s, ch in chans.items():+ for lag, tw in enumerate(ch.taps):+ for d, cf in ch.expr.terms.items():+ w_[lag, d, h * HD + s] += float(cf) * tw+ for j, (h, vd) in enumerate(vh):+ for col, ch in vd.v.items():+ for lag, tw in enumerate(ch.taps):+ for d, cf in ch.expr.terms.items():+ wv[lag, d, j * HD + col] += float(cf) * tw+ for d, cf in vd.a.terms.items():+ wa_[d, j] += float(cf)+ if vd.beta is None:+ bconst[j] = 1.0+ else:+ for d, cf in vd.beta.terms.items():+ wbeta[d, j] += float(cf)+ for col, e in vd.z.items():+ for d, cf in e.terms.items():+ wz[d, j * HD + col] += float(cf)+ for col, d, w in vd.out:+ wo[j * HD + col, d] += w+ decays = [j for j in range(V) if np.abs(wa_[:, j]).sum()]+ lag0 = np.zeros((4, D, len(decays) + V + V * HD)) # a, beta, z read lag 0 only+ lag0[0] = np.concatenate([wa_[:, decays], wbeta, wz], 1)+ proj = np.concatenate([wq, wk, wv, lag0], 2) # one matmul for the sublayer+ nq, nv, nd = H * HD, V * HD, len(decays)+ self.steps.append({+ "proj": Lin_(proj, device), "split": [nq, nq, nv, nd, V, nv],+ "decays": torch.tensor(decays, device=device, dtype=torch.long),+ "bconst": torch.tensor(bconst, device=device, dtype=torch.float32),+ "o": Lin_(wo, device), "kh": torch.tensor([h for h, _ in vh], device=device),+ "H": H, "V": V, "heads": heads,+ "qhot": max(getattr(n, "qhot", 2) for n in heads),+ "S": torch.zeros(batch, V, HD, HD, device=device)})+ self.feat_ids = sorted(c.token_fns)+ self.bi = torch.arange(batch, device=device)+ self.hist = [None if isinstance(st, tuple) else torch.zeros(batch, 3, D, device=device)+ for st in self.steps]++ def load_init(self, row: int = 0):+ """The heads' initial states (the snapshot currently in the graph) into batch row ``row``."""+ for st in self.steps:+ if isinstance(st, tuple):+ continue+ buf = np.zeros(st["S"].shape[1:], np.float32)+ j = 0+ for n in st["heads"]:+ init = n.init or []+ for x in range(len(n.vheads())):+ for r, vec in (init[x] if x < len(init) else {}).items():+ buf[j, r] = vec+ j += 1+ st["S"][row].copy_(torch.from_numpy(buf))++ def embed(self, toks):+ out = np.zeros((len(toks), self.D))+ out[:, ONE] = 1.0+ for d in self.feat_ids:+ out[:, d] = [float(self.c.feature_value(d, t)) for t in toks]+ return torch.tensor(out, dtype=torch.float32, device=self.dev)++ def reset(self, rows=None):+ """Fresh lagged inputs for the rows (in place: a captured graph keeps these buffers)."""+ for h in self.hist:+ if h is not None:+ (h.zero_() if rows is None else h[rows].zero_())++ def step(self, x):+ """One position: x [B, D] (the embedding rows) -> the residual after every node."""+ B, bi = x.shape[0], self.bi[:x.shape[0]]+ for i, st in enumerate(self.steps):+ if isinstance(st, tuple):+ _, wab, nu, wd = st+ ab = wab(x[:, None])+ x = x + wd((torch.relu(ab[:, nu:]) * ab[:, :nu])[:, None])+ continue+ V, S, kh = st["V"], st["S"], st["kh"]+ hist = self.hist[i]+ xs = torch.cat([x[:, None], hist], 1) # lags 0..3+ q, k, v, a, bt, z = torch.split(st["proj"](xs), st["split"], 1)+ q, k, v = (torch.relu(e).view(B, -1, HD) for e in (q, k, v))+ if len(st["decays"]):+ fac = torch.where(a == 0, 1.0, torch.where(a == 110, 0.5, 0.0))+ S[:, st["decays"]] *= fac[:, :, None, None]+ beta = (bt + st["bconst"]) == 1+ kv = k[:, kh]+ kmax, idx = kv.max(2)+ write = (kmax > 0) & beta+ vi = torch.arange(V, device=x.device)[None, :]+ old = S[bi[:, None], vi, idx]+ S[bi[:, None], vi, idx] = torch.where(write[:, :, None], v, old)+ qv, qi = torch.topk(q[:, kh], st["qhot"], dim=2) # queries are at most k-hot+ rows = S[bi[:, None, None], vi[:, :, None], qi] * (qv > 0).float()[..., None]+ o = rows.sum(2)+ ref = (o[..., REF_COLS[0]:REF_COLS[-1] + 1] ** 2).sum(2)+ norm = torch.where(ref > 0, torch.sqrt(ref / HD), torch.ones_like(ref))+ o = torch.where((ref > 0)[..., None], o / norm[..., None], torch.zeros_like(o))+ z = torch.relu(z).view(B, V, HD)+ x = x + st["o"](((o * z).reshape(B, -1))[:, None])+ hist.copy_(xs[:, :3])+ return x+++def check(c, toks, device="cpu", sparse=False) -> int:+ """Executor == ir.evaluate on every dim at every position (exact)."""+ from chipformer.q35 import ir++ ex = Executor(c, device, sparse=sparse)+ ex.load_init()+ emb = ex.embed(toks)+ want = ir.evaluate(c, toks, check=False)+ for t in range(len(toks)):+ got = ex.step(emb[t:t + 1])[0].double().cpu().numpy()+ bad = np.nonzero(got != want[t])[0]+ assert not len(bad), (t, [(c.dims[d].name, got[d], want[t, d]) for d in bad[:5]])+ return len(toks)+++class FreeRun:+ """Generation on the GPU: each step decodes the circuit's output code (core.OUT_NAMES) into+ the next token and writes that token's embedding as the next input, inside one graph."""++ def __init__(self, c, batch: int = 1, device="cuda", compile_=True, sparse=False, v2=False):+ from chipformer.rv import core as C++ self.c, self.B, self.dev = c, batch, device+ if v2: # the Triton executor (tritonexec.py)+ from chipformer.rv.tritonexec import Executor2++ self.ex = Executor2(c, batch, device)+ compile_ = False+ else:+ self.ex = Executor(c, device, batch, sparse)+ if compile_:+ self.ex.step = torch.compile(self.ex.step, mode="max-autotune-no-cudagraphs",+ dynamic=False)+ lt = lambda a: torch.tensor(a, dtype=torch.long, device=device)+ self.fam_idx = lt([self._dim(e) for e in c.out_fam])+ self.pay_idx = lt([self._dim(e) for e in c.out_pay])+ self.fam_w = 2.0 ** torch.arange(len(c.out_fam), device=device)+ self.pay_w = 2.0 ** torch.arange(len(c.out_pay), device=device)+ # a big family's token embeddings are linear in the payload bits; the small families+ # (whose features may be any function of the payload, e.g. QS / QD's digit index) come+ # from a table+ from chipformer.rv import spec as S++ nf, nb = len(C.OUT_NAMES), C.NPAY+ base, bits = np.zeros((nf, self.ex.D)), np.zeros((nf, nb, self.ex.D))+ ids = np.zeros(nf, dtype=np.int64)+ small, row0, table = np.zeros(nf, dtype=bool), np.zeros(nf, dtype=np.int64), []+ for f, nm in enumerate(C.OUT_NAMES):+ t0 = S.tok(nm) if nm in S.CONTROLS else S.tok(nm, 0)+ size = 1 if nm in S.CONTROLS else S.FAMILIES[nm][1]+ ids[f] = t0+ base[f] = self.ex.embed([t0])[0].cpu().numpy()+ if size <= 4096:+ small[f], row0[f] = True, len(table)+ table += list(self.ex.embed(list(range(t0, t0 + size))).cpu().numpy())+ else:+ for i in range(size.bit_length() - 1):+ bits[f, i] = self.ex.embed([t0 + (1 << i)])[0].cpu().numpy() - base[f]+ t = lambda a, dt=torch.float32: torch.tensor(np.asarray(a), dtype=dt, device=device)+ self.base, self.bits, self.ids = t(base), t(bits), t(ids, torch.long)+ self.small, self.row0, self.table = t(small, torch.bool), t(row0, torch.long), t(table)+ self.x = torch.zeros(batch, self.ex.D, device=device)+ self.tok = torch.zeros(batch, dtype=torch.long, device=device)+ self.graph = None++ @staticmethod+ def _dim(e):+ ((d, w),) = e.terms.items()+ assert w == 1+ return d++ def load(self, row: int, first_token: int):+ """Row ``row`` starts from the snapshot now in the graph's tables, then first_token."""+ self.ex.load_init(row)+ self.ex.reset([row])+ self.x[row] = self.ex.embed([first_token])[0]++ def _step(self):+ out = self.ex.step(self.x)+ fam = (torch.round(out[:, self.fam_idx]) * self.fam_w).sum(1)+ pay = torch.round(out[:, self.pay_idx])+ fid = fam.long().clamp(0, len(self.ids) - 1)+ p = (pay * self.pay_w).sum(1).long()+ self.tok.copy_(self.ids[fid] + p)+ row = (self.row0[fid] + p).clamp(0, len(self.table) - 1)+ big = self.base[fid] + torch.einsum("bi,bid->bd", pay, self.bits[fid])+ self.x.copy_(torch.where(self.small[fid][:, None], self.table[row], big))++ def capture(self):+ x0 = self.x.clone()+ heads = [st for st in self.ex.steps if isinstance(st, dict) and "S" in st]+ states = [st["S"].clone() for st in heads]+ s = torch.cuda.Stream()+ s.wait_stream(torch.cuda.current_stream())+ with torch.cuda.stream(s):+ for _ in range(3):+ self._step()+ torch.cuda.current_stream().wait_stream(s)+ g = torch.cuda.CUDAGraph()+ with torch.cuda.graph(g):+ self._step()+ # undo the warm-up steps+ self.x.copy_(x0)+ for st, s0 in zip(heads, states):+ st["S"].copy_(s0)+ self.ex.reset()+ self.graph = g++ def run(self, n_steps: int) -> np.ndarray:+ """[B, n_steps] generated tokens."""+ cuda = self.dev != "cpu"+ if cuda and self.graph is None:+ self.capture()+ out = torch.zeros(self.B, n_steps, dtype=torch.long, device=self.dev)+ for t in range(n_steps):+ self.graph.replay() if cuda else self._step()+ out[:, t].copy_(self.tok)+ if cuda:+ torch.cuda.synchronize()+ return out.cpu().numpy()+++def timed(c, toks, steps=500, graph=True, compile_=False, batch=1, sparse=False) -> float:+ """Seconds per step of B rows (teacher-forced; the graph's replays are checked against the+ IR first)."""+ ex = Executor(c, "cuda", batch, sparse)+ for b in range(batch):+ ex.load_init(b)+ if compile_:+ ex.step = torch.compile(ex.step, mode="max-autotune-no-cudagraphs", dynamic=False)+ emb = ex.embed(toks[:steps + 20])+ xin = torch.zeros(batch, ex.D, device="cuda")+ states = [st["S"].clone() for st in ex.steps if not isinstance(st, tuple)]++ def restore():+ for st, s0 in zip([st for st in ex.steps if not isinstance(st, tuple)], states):+ st["S"].copy_(s0)+ ex.reset()++ if graph:+ s = torch.cuda.Stream()+ s.wait_stream(torch.cuda.current_stream())+ with torch.cuda.stream(s):+ for _ in range(3):+ xin.copy_(emb[0].expand(batch, -1))+ ex.step(xin)+ torch.cuda.current_stream().wait_stream(s)+ restore()+ g = torch.cuda.CUDAGraph()+ with torch.cuda.graph(g):+ out = ex.step(xin)+ from chipformer.q35 import ir++ n = min(300, len(emb))+ want = ir.evaluate(c, toks[:n], check=False)+ restore()+ for t in range(n):+ xin.copy_(emb[t].expand(batch, -1))+ g.replay()+ got = out.double().cpu().numpy()+ assert (got == want[t][None]).all(), f"graph replay differs at {t}"+ restore()+ run = g.replay+ else:+ def run():+ ex.step(xin)+ for t in range(20):+ xin.copy_(emb[t].expand(batch, -1))+ run()+ torch.cuda.synchronize()+ t0 = time.time()+ for t in range(20, 20 + steps):+ xin.copy_(emb[t].expand(batch, -1))+ run()+ torch.cuda.synchronize()+ return (time.time() - t0) / steps+++def timed2(c, toks, steps=500, batch=1) -> float:+ """Seconds per step of B rows with the Triton executor (tritonexec), under a CUDA graph;+ replays are checked against the IR first."""+ from chipformer.q35 import ir+ from chipformer.rv.tritonexec import Executor2++ ex = Executor2(c, batch, "cuda")+ for b in range(batch):+ ex.load_init(b)+ emb = ex.embed(toks[:steps + 20])+ heads = [st for st in ex.steps if st["kind"] == "head"]+ states = [st["S"].clone() for st in heads]++ def restore():+ for st, s0 in zip(heads, states):+ st["S"].copy_(s0)+ ex.reset()++ xin = torch.zeros(batch, ex.D, device="cuda")+ s = torch.cuda.Stream()+ s.wait_stream(torch.cuda.current_stream())+ with torch.cuda.stream(s):+ for _ in range(3):+ xin.copy_(emb[0].expand(batch, -1))+ ex.step(xin)+ torch.cuda.current_stream().wait_stream(s)+ restore()+ g = torch.cuda.CUDAGraph()+ with torch.cuda.graph(g):+ ex.step(xin)+ n = min(300, len(emb))+ want = ir.evaluate(c, toks[:n], check=False)+ restore()+ for t in range(n):+ xin.copy_(emb[t].expand(batch, -1))+ g.replay()+ got = xin.double().cpu().numpy()+ assert (got == want[t][None]).all(), f"graph replay differs at {t}"+ restore()+ for t in range(20):+ xin.copy_(emb[t].expand(batch, -1))+ g.replay()+ torch.cuda.synchronize()+ t0 = time.time()+ for t in range(20, 20 + steps):+ xin.copy_(emb[t].expand(batch, -1))+ g.replay()+ torch.cuda.synchronize()+ return (time.time() - t0) / stepsdiff --git a/experimental/riscv/chipformer/rv/firmware.py b/experimental/riscv/chipformer/rv/firmware.pynew file mode 100644index 0000000..f52600a--- /dev/null+++ b/experimental/riscv/chipformer/rv/firmware.py@@ -0,0 +1,64 @@+"""Firmware for rv-tiny: an ELF becomes a state snapshot in the model's HF cache, then the model+runs the program by plain greedy decoding.++ python -m chipformer.rv.firmware dist/rv-tiny program.elf [--max-new 4000]++The snapshot is the circuit's head states (ROM rows, RAM words, registers, PC, counters; see+core.snapshot), written into the linear-attention layers' recurrent states after a one-token+PAD prefix (a zero embedding: it leaves every state and conv window at 0). The run then starts+at GO. Everything after that is the model's own greedy decoding (``generate``).+"""++import argparse++import torch++from chipformer.rv import core, elf, hf, ref+from chipformer.rv import spec as S+++def load_cache(model, c, machine):+ """A DynamicCache holding the PAD prefix and the machine's snapshot."""+ dev = next(model.parameters()).device+ core.snapshot(c, machine)+ out = model(input_ids=torch.tensor([[S.VOCAB]], device=dev), use_cache=True)+ hf.inject(c, out.past_key_values)+ return out.past_key_values+++def run(model, c, machine, max_new: int = 4000) -> list[int]:+ """The generated tokens after GO (stopping at HALT)."""+ dev = next(model.parameters()).device+ cache = load_cache(model, c, machine)+ ids = torch.tensor([[S.VOCAB, S.tok("GO")]], device=dev)+ with torch.no_grad():+ out = model.generate(input_ids=ids, past_key_values=cache, max_new_tokens=max_new,+ do_sample=False, eos_token_id=S.tok("HALT"), pad_token_id=S.VOCAB)+ return out[0, 2:].tolist()+++def output_text(tokens: list[int]) -> str:+ """The program's putchar output in a generated trace."""+ return bytes(p for t in tokens if t < S.VOCAB and (f := S.family(t))[0] == "BYTE"+ for p in [f[1]]).decode("latin-1")+++def main():+ from transformers import Qwen3_5ForCausalLM++ ap = argparse.ArgumentParser()+ ap.add_argument("model")+ ap.add_argument("elf")+ ap.add_argument("--max-new", type=int, default=4000)+ args = ap.parse_args()+ model = Qwen3_5ForCausalLM.from_pretrained(args.model, dtype=torch.bfloat16).eval()+ if torch.cuda.is_available():+ model = model.cuda()+ c = core.build()+ toks = run(model, c, ref.Machine.from_program(elf.program(args.elf)), args.max_new)+ print(output_text(toks), end="")+ print(f"[{len(toks)} tokens, last {S.name(toks[-1]) if toks[-1] < S.VOCAB else toks[-1]}]")+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/fuzz.py b/experimental/riscv/chipformer/rv/fuzz.pynew file mode 100644index 0000000..d01b654--- /dev/null+++ b/experimental/riscv/chipformer/rv/fuzz.py@@ -0,0 +1,138 @@+"""Random RV32IM + Zba programs, and free-running checks of the circuit executor against ref.py.++ python -m chipformer.rv.fuzz --programs 128 --batch 32 [--elf c-fib c-qsort ...]++A program is a loop over a random block: every ALU, shift, compare, MUL, DIV and Zba op on random+registers, loads and stores (all widths) to a 1 KB RAM buffer and loads from a 256-byte DROM+table through masked addresses, forward branches, calls to a leaf function, putchar of a+register's low byte, and LUI/AUIPC. The+executor generates every program's trace by itself (fastexec.FreeRun: each predicted token is+the next input), batch rows in parallel, and every row must equal the reference trace up to its+HALT.+"""++import argparse+import random+import time++from chipformer.rv import core, elf, ref+from chipformer.rv import spec as S+from chipformer.rv.asm import assemble++REGS = ["a0", "a1", "a2", "a3", "a4", "a5", "t0", "t1", "t2", "t3", "t4", "s1", "s2", "s3", "s4"]+OPS3 = ["add", "sub", "and", "or", "xor", "sll", "srl", "sra", "slt", "sltu", "mul", "mulh",+ "mulhsu", "mulhu", "div", "divu", "rem", "remu", "sh1add", "sh2add", "sh3add"]+OPSI = ["addi", "andi", "ori", "xori", "slti", "sltiu"]+OPSS = ["slli", "srli", "srai"]+LOADS = [("lw", 0x3FC), ("lh", 0x3FE), ("lhu", 0x3FE), ("lb", 0x3FF), ("lbu", 0x3FF)]+STORES = [("sw", 0x3FC), ("sh", 0x3FE), ("sb", 0x3FF)]+++def gen_program(seed: int, block: int = 40, iters: int = 30) -> str:+ """A random fault-free RV32IM + Zba program (s0: loop counter, s11: the RAM buffer, s10: the+ DROM table, t5: addresses)."""+ rng = random.Random(seed)+ val = lambda: rng.choice([rng.randint(-2**31, 2**31 - 1), rng.randint(-40, 40),+ rng.randint(0, 0x3FF)])+ lines = ["_start: la s11, buf", "la s10, rtab", f"li s0, {iters}"]+ lines += [f"li {r}, {val()}" for r in REGS]+ lines.append("loop:")+ skip = 0+ for _ in range(block):+ k = rng.random()+ d, a, b = rng.choice(REGS), rng.choice(REGS), rng.choice(REGS)+ if k < 0.35:+ lines.append(f"{rng.choice(OPS3)} {d}, {a}, {b}")+ elif k < 0.50:+ lines.append(f"{rng.choice(OPSI)} {d}, {a}, {rng.randint(-2048, 2047)}")+ elif k < 0.56:+ lines.append(f"{rng.choice(OPSS)} {d}, {a}, {rng.randint(0, 31)}")+ elif k < 0.64:+ op, mask = rng.choice(LOADS)+ lines += [f"andi t5, {a}, {mask}", "add t5, t5, s11", f"{op} {d}, 0(t5)"]+ elif k < 0.70:+ op, mask = rng.choice(LOADS)+ lines += [f"andi t5, {a}, {mask & 0xFF}", "add t5, t5, s10", f"{op} {d}, 0(t5)"]+ elif k < 0.82:+ op, mask = rng.choice(STORES)+ lines += [f"andi t5, {a}, {mask}", "add t5, t5, s11", f"{op} {b}, 0(t5)"]+ elif k < 0.90:+ br = rng.choice(["beq", "bne", "blt", "bge", "bltu", "bgeu"])+ lines += [f"{br} {a}, {b}, skip{skip}", f"{rng.choice(OPS3)} {d}, {a}, {b}",+ f"skip{skip}:"]+ skip += 1+ elif k < 0.94:+ lines += [f"mv a6, {a}", "call leaf", f"mv {d}, a6"]+ elif k < 0.97:+ lines += [f"andi a0, {a}, 127", "li a7, 1", "ecall"]+ else:+ lines.append(f"lui {d}, {rng.randint(0, 2**20 - 1)}" if rng.random() < 0.5+ else f"auipc {d}, {rng.randint(0, 2**20 - 1)}")+ lines += ["addi s0, s0, -1", "bnez s0, loop", "li a0, 0", "li a7, 93", "ecall",+ "leaf: slli a6, a6, 3", "xori a6, a6, 0x5a5", "add a6, a6, s0", "ret",+ ".data", "buf: .space 1024",+ ".rodata", "rtab: .word " + ", ".join(str(rng.getrandbits(32)) for _ in range(64))]+ return "\n".join(lines)+++def free_run(c, progs: list, batch: int, device="cuda", compile_=True, v2=False) -> dict:+ """Free-run every program (batch rows at a time); returns totals, raises on any mismatch.+ A program is an asm / ELF Program or a function returning a fresh ref.Machine."""+ from chipformer.rv import fastexec++ def fresh(prog):+ return prog() if callable(prog) else ref.Machine.from_program(prog)++ fr = fastexec.FreeRun(c, batch, device, compile_, v2=v2)+ total_instr = total_tok = 0+ t0 = time.time()+ for i in range(0, len(progs), batch):+ chunk = progs[i:i + batch]+ refs = []+ for row in range(batch):+ prog = chunk[row % len(chunk)]+ core.snapshot(c, fresh(prog))+ fr.load(row, S.tok("GO"))+ refs.append(fresh(prog).run(10**8))+ n = max(len(r) for r in refs) - 1+ got = fr.run(n)+ for row, r in enumerate(refs[:len(chunk)]):+ want = r[1:]+ bad = [t for t in range(len(want)) if got[row, t] != want[t]]+ if bad:+ t = bad[0]+ raise AssertionError(f"program {i + row}: token {t + 1} is {got[row, t]}, "+ f"expected {S.name(want[t])}")+ total_tok += len(r)+ total_instr += ref.trace_stats(r)["instructions"]+ if (i // batch) % 20 == 0:+ print(f" .. {i + len(chunk)} / {len(progs)} rows, {total_instr} instructions, "+ f"{time.time() - t0:.0f} s", flush=True)+ return {"programs": len(progs), "instructions": total_instr, "tokens": total_tok,+ "seconds": time.time() - t0}+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--programs", type=int, default=128)+ ap.add_argument("--batch", type=int, default=32)+ ap.add_argument("--iters", type=int, default=30)+ ap.add_argument("--elf", nargs="*", default=[])+ ap.add_argument("--device", default="cuda")+ ap.add_argument("--v2", action="store_true", help="the Triton executor (tritonexec)")+ args = ap.parse_args()+ c = core.build()+ progs = [assemble(gen_program(s, iters=args.iters)) for s in range(args.programs)]+ if progs:+ res = free_run(c, progs, args.batch, args.device, args.device != "cpu", args.v2)+ print(f"fuzz: {res}", flush=True)+ if args.elf:+ from chipformer.rv.measure import DATA++ eprogs = [elf.program(DATA / f"{n}.elf") for n in args.elf]+ res = free_run(c, eprogs, len(eprogs), args.device, args.device != "cpu", args.v2)+ print(f"elf {args.elf}: {res}", flush=True)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/gpucheck.py b/experimental/riscv/chipformer/rv/gpucheck.pynew file mode 100644index 0000000..f23c66a--- /dev/null+++ b/experimental/riscv/chipformer/rv/gpucheck.py@@ -0,0 +1,67 @@+"""P0.2 on a GPU: the RV32 core in stock HF modules with the fla kernels (or the torch fallbacks),+bf16 and fp32, chunked prefill and token-by-token decode, against the IR evaluator.++ python -m chipformer.rv.gpucheck [--dtype bf16|fp32] [--progs signext,call,...] [--decode]++Each program's ROM / RAM / registers / PC are injected into the HF cache (hf.inject), so this+also covers the byte-column 2-hot ROM reads, the sibling value heads' write gates, and the lag-1+key commits through prefill-then-decode. Prints one line per (program, mode) and exits 1 on any+mismatch.+"""++import argparse+import sys+import time++import torch++from chipformer.rv import core, elf, hf, ref+from chipformer.rv.measure import DATA+from chipformer.rv.programs import ALL, program+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--dtype", default="bf16", choices=["bf16", "fp32"])+ ap.add_argument("--progs", default="signext,call,memcpy,rv32ui-lb,rv32ui-sh,rv32ui-jalr,"+ "rv32ui-sra,c-crc32,c-strops")+ ap.add_argument("--decode", action="store_true", help="also decode token by token")+ args = ap.parse_args()+ dtype = torch.bfloat16 if args.dtype == "bf16" else torch.float32+ try:+ import fla # noqa: F401++ kernels = "fla"+ except ImportError:+ kernels = "torch fallbacks"+ print(f"device {torch.cuda.get_device_name() if torch.cuda.is_available() else 'cpu'}, "+ f"{args.dtype}, kernels: {kernels}", flush=True)+ c = core.build()+ bad = 0+ for name in args.progs.split(","):+ prog = program(name) if name in ALL else elf.program(DATA / f"{name}.elf")+ core.snapshot(c, ref.Machine.from_program(prog))+ toks = ref.Machine.from_program(prog).run(10**6)+ model, lay, ids = hf.build_model(c, toks, dtype)+ model = model.to("cuda" if torch.cuda.is_available() else "cpu")+ for mode in ["prefill"] + (["decode"] if args.decode else []):+ t = time.time()+ if args.dtype == "fp32":+ n, nd, wrong = hf.check(c, model, lay, ids, toks, mode, tokens=True)+ bad += wrong > 0+ print(f"{name:14s} {mode:7s} {n} positions: {nd} dims not bit-exact, "+ f"{wrong} wrong tokens ({time.time() - t:.1f} s)", flush=True)+ continue+ try:+ n = hf.check(c, model, lay, ids, toks, mode)+ print(f"{name:14s} {mode:7s} exact on {n} positions ({time.time() - t:.1f} s)",+ flush=True)+ except AssertionError as e:+ bad += 1+ print(f"{name:14s} {mode:7s} MISMATCH {str(e)[:300]}", flush=True)+ del model+ sys.exit(1 if bad else 0)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/hf.py b/experimental/riscv/chipformer/rv/hf.pynew file mode 100644index 0000000..0c80289--- /dev/null+++ b/experimental/riscv/chipformer/rv/hf.py@@ -0,0 +1,112 @@+"""The RV32 core in a stock Qwen3_5ForCausalLM, with the snapshot injected into the HF cache.++ model, lay, ids = build_model(c, tokens) # chip layers only (+ one zero attention layer)+ check(c, model, lay, ids, toks, mode="prefill") # or "decode": the residual equals the IR's++The state snapshot (ROM, RAM, registers, PC: the heads' ``init``) goes into the linear-attention+layers' ``recurrent_states`` (state row = key dim, columns = value dims) after a one-token prefix+whose embedding is 0 (a text token: it leaves every conv window and state at 0). The trace then+runs from GO either as one chunked prefill (``torch_chunk_gated_delta_rule`` with that initial+state) or token by token (``torch_recurrent_gated_delta_rule``).+"""++import numpy as np+import torch++from chipformer.q35 import ir+from chipformer.q35.emit import check_bf16, embedding_rows, emit_layers+from chipformer.q35.ir import Head+from chipformer.q35.tiny import BASE, pick_layout+++def build_model(c, toks: list[int], dtype=torch.bfloat16):+ """A model whose vocabulary is the distinct tokens of ``toks`` (ids remapped)."""+ from transformers import Qwen3_5ForCausalLM, Qwen3_5TextConfig++ vocab = sorted(set(toks))+ ids = {t: i for i, t in enumerate(vocab)}+ ids[None] = len(vocab) # the zero prefix token+ lay = pick_layout(len(c.dims))+ n_chip = c.n_layers()+ n_vocab = -(-(len(vocab) + 1) // 64) * 64+ tc = dict(BASE, hidden_size=lay.hidden, vocab_size=n_vocab, num_hidden_layers=n_chip + 1,+ layer_types=["linear_attention"] * n_chip + ["full_attention"],+ linear_num_key_heads=c.target.key_heads, linear_num_value_heads=c.target.value_heads,+ intermediate_size=c.target.intermediate)+ sd = emit_layers(c, lay, tc, 0, "model.")+ emb = torch.zeros(n_vocab, lay.hidden)+ emb[: len(vocab)] = embedding_rows(c, lay, vocab, lay.hidden)+ sd["model.embed_tokens.weight"] = emb+ norm = torch.full((lay.hidden,), -1.0)+ check_bf16("final norm", norm)+ sd["model.norm.weight"] = norm+ model = Qwen3_5ForCausalLM(Qwen3_5TextConfig(**tc))+ for p in model.parameters():+ p.data.zero_()+ model.load_state_dict(sd, strict=False)+ model.tie_weights()+ return model.to(dtype).eval(), lay, ids+++def head_slots(c):+ """IR node index -> (layer, key head) in the emitted model."""+ out = {}+ for L in range(c.n_layers()):+ heads = [i for i, n in enumerate(c.nodes) if isinstance(n, Head) and n.level == 2 * L + 1]+ for j, i in enumerate(heads):+ out[i] = (L, j)+ return out+++def inject(c, cache) -> None:+ """Write the heads' initial states (the snapshot) into the HF cache's recurrent states."""+ r = c.target.value_heads // c.target.key_heads+ for ni, (L, j) in head_slots(c).items():+ st = cache.layers[L].recurrent_states[0]+ init = c.nodes[ni].init or []+ for x in range(len(c.nodes[ni].vheads())):+ S = st[0, j * r + x]+ S.zero_()+ for row, vec in (init[x] if x < len(init) else {}).items():+ S[row] = torch.tensor(vec, dtype=S.dtype, device=S.device)+++def check(c, model, lay, ids, toks: list[int], mode: str = "prefill", tokens: bool = False):+ """Run toks (from GO) through the model with the snapshot injected first; returns the+ number of positions compared. Asserts the residual equals the IR evaluator's on every IR dim+ at every position. With ``tokens``, returns (positions, inexact dims, wrong next-token+ predictions) instead of asserting (fp32 is not bit-exact: see docs/rv32.md)."""+ want = ir.evaluate(c, toks, check=False)+ dims = list(range(1, len(c.dims)))+ cols = [lay.dim(d) for d in dims]+ dev = next(model.parameters()).device+ x = torch.tensor([[ids[t] for t in toks]], device=dev)+ with torch.no_grad():+ out = model(input_ids=torch.tensor([[ids[None]]], device=dev), use_cache=True)+ cache = out.past_key_values+ inject(c, cache)+ hs = []+ if mode == "prefill":+ o = model(input_ids=x, past_key_values=cache, use_cache=True,+ output_hidden_states=True)+ hs.append(o.hidden_states[c.n_layers()][0])+ else:+ for t in range(len(toks)):+ o = model(input_ids=x[:, t:t + 1], past_key_values=cache, use_cache=True,+ output_hidden_states=True)+ hs.append(o.hidden_states[c.n_layers()][0])+ h = torch.cat(hs).double().cpu().numpy()[:, cols]+ w = want[:, dims]+ bad = np.nonzero((h != w).any(0))[0]+ if tokens:+ from chipformer.rv import core++ mat = np.zeros_like(want)+ mat[:, dims] = h+ pred = core.predictions(c, mat)+ wrong = sum(pred[t] != toks[t + 1] for t in range(len(toks) - 1))+ return h.shape[0], len(bad), wrong+ assert not len(bad), [(c.dims[dims[i]].name, int(np.argmax(h[:, i] != w[:, i])),+ float(h[np.argmax(h[:, i] != w[:, i]), i]),+ float(w[np.argmax(h[:, i] != w[:, i]), i])) for i in bad[:5]]+ return h.shape[0]diff --git a/experimental/riscv/chipformer/rv/measure.py b/experimental/riscv/chipformer/rv/measure.pynew file mode 100644index 0000000..cacac08--- /dev/null+++ b/experimental/riscv/chipformer/rv/measure.py@@ -0,0 +1,161 @@+"""P0 measurements of the RV32 core: depth, capacity per sublayer, nonzeros, tokens per+instruction (15- vs 16-bit payloads), vocabulary, and speed estimates.++ python -m chipformer.rv.measure [--nonzeros] > dist/rv/measure.json+"""++import argparse+import json+from collections import Counter+from pathlib import Path++from chipformer.q35.ir import FFN, HD, Head+from chipformer.rv import core, elf, ref+from chipformer.rv import spec as S++DATA = Path(__file__).resolve().parents[2] / "tests" / "data" / "rv"+C_PROGS = ["c-crc32", "c-sieve", "c-sort", "c-strops"]+# dynamic instruction mix of small C programs (dist/rv_tool/REPORT.md, P0.6), percent+MIX_SMALL_C = {"ALU": 47.0, "LW": 13.7, "LBH": 1.4, "SW": 8.9, "SBH": 2.3, "BRT": 11.3,+ "BRNT": 10.1, "JAL": 1.4, "JALR": 2.2, "U": 0.1, "MUL": 0.2, "DIV": 1.5}+DENSE_4B = {"hidden": 3072, "key_heads": 16, "value_heads": 32, "intermediate": 9216}+++def per_sublayer(c):+ units, heads = Counter(), Counter()+ for n in c.nodes:+ if isinstance(n, FFN):+ units[n.level] += len(n.terms)+ else:+ heads[n.level] += 1+ names = Counter()+ for n in c.nodes:+ if isinstance(n, Head):+ names[(n.level, n.name.rstrip("0123456789").rstrip("."))] += 1+ return ({f"L{(lv + 1) // 2}.{'attn' if lv % 2 else 'mlp'}": u for lv, u in sorted(units.items())},+ {f"L{(lv + 1) // 2}.attn": h for lv, h in sorted(heads.items())},+ {f"L{(lv + 1) // 2}.{nm}": k for (lv, nm), k in sorted(names.items())})+++def value_lengths(vals, payload_bits):+ """Tokens of each value record with chunks of payload_bits (sign-extended final chunk)."""+ out = Counter()+ for v in vals:+ s = v - (1 << 32) if v >> 31 else v+ k = 1+ while not -(1 << (payload_bits * k - 1)) <= s < 1 << (payload_bits * k - 1):+ k += 1+ out[min(k, -(-32 // payload_bits))] += 1+ return out+++def trace_classes():+ """Per-class record counts and rd values over the C programs' traces."""+ cls, vals = Counter(), []+ for name in C_PROGS:+ m = ref.Machine.from_program(elf.program(DATA / f"{name}.elf"))+ while not m.halted:+ if m.n == S.N_MAX:+ m.record()+ cls["R"] += 1+ continue+ ins = ref.decode(m.rom[(m.pc - S.ROM_BASE) >> 2])+ before = list(m.x)+ m.record()+ op = ins.op+ if op in ref.BRANCH_OPS:+ cls["BR"] += 1+ elif op in ("JAL", "JALR", "SW", "SB", "SH", "ECALL", "EBREAK", "FENCE"):+ cls[op] += 1+ elif ins.rd:+ cls["value"] += 1+ vals.append(m.x[ins.rd])+ else:+ cls["NT"] += 1+ del before+ return cls, vals+++def tokens_per_instruction(vbar: float, refresh: float = 0.0) -> float:+ """Expected tokens per instruction for the small-C mix (value records cost vbar)."""+ t = {"ALU": vbar, "LW": vbar, "LBH": vbar, "U": vbar, "MUL": vbar, "DIV": vbar,+ "SW": 1, "SBH": 4, "BRT": 1, "BRNT": 1, "JAL": 1, "JALR": 1}+ return sum(MIX_SMALL_C[k] * t[k] for k in MIX_SMALL_C) / sum(MIX_SMALL_C.values()) + refresh+++def payload_options(vals, hidden: int = 3072) -> dict:+ """Value-chunk widths: tokens per instruction (small-C mix) against vocabulary rows.++ 12-bit chunks need a 2-token jump (the ROM word index is wider than 12 bits); 16-bit chunks+ let an SB/SH write its merged word as V + final (2 tokens instead of 4) but need 16-bit SS+ families too."""+ out = {}+ for pb in (12, 15, 16):+ lens = value_lengths(vals, pb)+ vbar = sum(k * n for k, n in lens.items()) / max(sum(lens.values()), 1)+ jump = 2 if pb < 14 else 1+ t = {"ALU": vbar, "LW": vbar, "LBH": vbar, "U": vbar, "MUL": vbar, "DIV": vbar,+ "SW": 1, "SBH": 2 if pb == 16 else 4, "BRT": jump, "BRNT": 1, "JAL": jump,+ "JALR": jump}+ tpi = sum(MIX_SMALL_C[k] * t[k] for k in MIX_SMALL_C) / sum(MIX_SMALL_C.values())+ rows = {12: 5 * 4096 + 256 + 7, 15: 5 * (1 << 15) + 4 + 7, 16: 7 * (1 << 16) + 5}[pb]+ out[pb] = {"value_record_lengths": dict(sorted(lens.items())), "vbar": round(vbar, 3),+ "tokens_per_instruction_small_c_mix": round(tpi, 3), "vocab_rows": rows,+ "embedding_gb_bf16": round(rows * hidden * 2 / 1e9, 2)}+ return out+++def nonzeros(c):+ """Nonzero weights of the emitted chip layers (standalone layout; independent of width)."""+ from chipformer.q35.emit import emit_layers+ from chipformer.q35.tiny import BASE, pick_layout++ lay = pick_layout(len(c.dims))+ tc = dict(BASE, hidden_size=lay.hidden, linear_num_key_heads=c.target.key_heads,+ linear_num_value_heads=c.target.value_heads, intermediate_size=c.target.intermediate)+ sd = emit_layers(c, lay, tc, 0, "model.")+ per = Counter()+ for k, t in sd.items():+ per[int(k.split(".")[2])] += int((t != 0).sum())+ return dict(sorted(per.items())), lay.hidden+++def dense_params_per_layer(h, kh, vh, inter):+ qkv = (2 * kh * HD + vh * HD) * h+ return qkv + vh * HD * h + 2 * vh * h + h * vh * HD + 3 * inter * h + 4 * (2 * kh * HD + vh * HD)+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--nonzeros", action="store_true")+ args = ap.parse_args()+ c = core.build()+ units, heads, names = per_sublayer(c)+ cls, vals = trace_classes()+ n_instr = sum(v for k, v in cls.items() if k != "R")+ res = {"stats": c.stats(), "units_per_sublayer": units, "heads_per_sublayer": heads,+ "heads_by_table": names, "trace_classes": dict(cls)}+ res["payload_options"] = payload_options(vals)+ per_prog = {}+ toks = 0+ for n in C_PROGS:+ st = ref.trace_stats(ref.Machine.from_program(elf.program(DATA / f"{n}.elf")).run(10**7))+ per_prog[n] = {k: st[k] for k in ("instructions", "tokens", "tokens_per_instruction")}+ toks += st["tokens"]+ res["c_programs"] = per_prog+ res["measured_tokens_per_instruction_c"] = toks / n_instr+ dense = dense_params_per_layer(DENSE_4B["hidden"], 16, 32, 9216)+ res["dense_params_per_chip_layer_4b"] = dense+ res["dense_chip_params_4b"] = dense * c.n_layers()+ cap = {"rom_bytes_per_sublayer": 16 * 1024 * 4, "ram_bytes_per_sublayer": 16 * 2 * 128 * 4}+ res["capacity"] = cap+ if args.nonzeros:+ nz, h = nonzeros(c)+ res["nonzeros_per_layer"] = nz+ res["nonzeros_total"] = sum(nz.values())+ res["nonzeros_hidden"] = h+ print(json.dumps(res, indent=1))+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/merge.py b/experimental/riscv/chipformer/rv/merge.pynew file mode 100644index 0000000..f3bda0d--- /dev/null+++ b/experimental/riscv/chipformer/rv/merge.py@@ -0,0 +1,248 @@+"""Merge the RV32 machine and its text bridge into Qwen3.5-4B (layout "rv1").++ python -m chipformer.rv.merge SRC --out dist/q4-rv1 [--no-mute]++The result is a text-only stock ``Qwen3_5ForCausalLM``:++- the chip layers come first (the circuit of ``core.build(text=...)``), then Qwen's layers,+ zero-padded to the new hidden size (q35.merge.widen's rules: Qwen never reads or writes chip+ dims; the full-residual norms are rescaled so its normalized inputs stay the same);+- every Qwen vocabulary row carries the ballast and its text features in the chip dims (is_text,+ tau, its first 16 UTF-8 bytes, the ``<tool_call>`` / ``</tool_call>`` flags, and for the 260+ printable tokens the output code of their echo); the RV32 tokens are appended after Qwen's+ vocabulary as special tokens;+- the tied output head reads only the output code and tau among the chip dims (q35.merge's+ final-norm gains), and Qwen's layers are muted on chip tokens (q35.mute, on mu).++``chipformer.json`` records the layout for the decoder (rv/chat.py).+"""++import json+import re+from pathlib import Path++import numpy as np+import torch+from safetensors import safe_open++from chipformer.q35.merge import (IM_START, IN_COLS, OUT_ROWS, TAU_GAIN, WIDE_NORMS,+ pick_hidden)+from chipformer.rv import bridge as BR+from chipformer.rv import core+from chipformer.rv import spec as S++SRC_TEXT = "model.language_model."+DST = "model."+++def chip_token_names() -> list[str]:+ names = []+ for t in range(S.VOCAB):+ fam, p = S.family(t)+ names.append(f"<|rv:{fam}|>" if fam in S.CONTROLS else f"<|rv:{fam}:{p}|>")+ return names+++def extend_tokenizer(src: Path, dst: Path, first_id: int) -> None:+ import shutil++ from tokenizers import AddedToken, Tokenizer++ tok = Tokenizer.from_file(str(src / "tokenizer.json"))+ n = tok.get_vocab_size(with_added_tokens=True)+ pad = [AddedToken(f"<|reserved_{i}|>", special=True) for i in range(first_id - n)]+ chip = [AddedToken(s, special=True) for s in chip_token_names()]+ tok.add_special_tokens(pad + chip)+ assert tok.token_to_id(chip_token_names()[0]) == first_id+ dst.mkdir(parents=True, exist_ok=True)+ tok.save(str(dst / "tokenizer.json"))+ for f in ("chat_template.jinja", "LICENSE"):+ if (src / f).exists():+ shutil.copy(src / f, dst / f)+ tc = json.loads((src / "tokenizer_config.json").read_text(encoding="utf-8"))+ dec = tc.setdefault("added_tokens_decoder", {})+ for i, t in enumerate(pad + chip):+ dec[str(n + i)] = {"content": t.content, "lstrip": False, "normalized": False,+ "rstrip": False, "single_word": False, "special": True}+ (dst / "tokenizer_config.json").write_text(json.dumps(tc, indent=2, ensure_ascii=False),+ encoding="utf-8")+++def text_rows(c, lay, T: BR.Text, base_emb: torch.Tensor, v0: int) -> torch.Tensor:+ """Qwen's rows: the Qwen part, plus the chip features of each text token (a few special+ rows exactly, every other row = the plain text row plus its byte bits)."""+ from chipformer.q35.emit import embedding_rows++ h = lay.hidden+ d0 = base_emb.shape[1]+ full = torch.zeros(v0, h, dtype=torch.bfloat16)+ full[:, :d0] = base_emb[:v0, :d0]+ special = sorted({T.TCO, T.TCC, *T.printable})+ q_plain = next(q for q in range(T.n) if q not in set(special))+ plain = embedding_rows(c, lay, [BR.text_token(q_plain)], h)[0] # (bytes zeroed below)+ byte_cols = [[lay.dim(next(iter(c.tb[k][i].terms))) for i in range(8)]+ for k in range(S.TEXT_ROW)]+ for k in range(S.TEXT_ROW):+ for i in range(8):+ plain[byte_cols[k][i]] = 0.0+ full[:, d0:] = plain[d0:].to(torch.bfloat16)+ by = np.zeros((v0, S.TEXT_ROW), dtype=np.uint8)+ for q in range(min(v0, T.n)):+ b = T.row_bytes(q)+ by[q] = np.frombuffer(b, dtype=np.uint8)+ for k in range(S.TEXT_ROW):+ for i in range(8):+ full[:, byte_cols[k][i]] = torch.tensor((by[:, k] >> i) & 1, dtype=torch.bfloat16)+ rows = embedding_rows(c, lay, [BR.text_token(q) for q in special], h)+ for q, r in zip(special, rows):+ full[q, d0:] = r[d0:].to(torch.bfloat16)+ return full+++def build_merged(src, dst, mute_qwen: bool = True, hidden: int | None = None,+ m: int | None = None) -> dict:+ from safetensors.torch import save_file++ from chipformer.q35.emit import Layout, allocate, check_bf16, emit_layers+ from chipformer.q35.ir import Target+ from chipformer.q35.tiny import code_gain+ from chipformer.rv.tiny import embedding_matrix++ src, dst = Path(src), Path(dst)+ cfg = json.loads((src / "config.json").read_text())+ tc0 = cfg["text_config"]+ d0, v0, inter = tc0["hidden_size"], tc0["vocab_size"], tc0["intermediate_size"]+ T = BR.Text(src / "tokenizer.json")+ c = core.build(Target(tc0["linear_num_key_heads"], tc0["linear_num_value_heads"], inter), T)+ keep = set(c.keep)+ alloc = allocate(c, None, keep, inter)+ if hidden is None:+ hidden, m = pick_hidden(d0, 1 + alloc.n_slots)+ lay = Layout(hidden=hidden, off=d0, m=m, slots=alloc.slots)+ assert d0 + m + 1 + alloc.n_slots <= hidden, (alloc.n_slots, hidden)+ lay.scale # noqa: B018 - asserts m * B^2 / hidden is a power of 4+ n_chip = c.n_layers()+ n0 = tc0["num_hidden_layers"]+ print(f"circuit {c.stats()}, slots {alloc.n_slots}, hidden {hidden} (m={m}), "+ f"{n_chip} chip layers", flush=True)++ tc = {k: v for k, v in tc0.items() if k != "full_attention_interval"}+ tc.update(hidden_size=hidden, num_hidden_layers=n_chip + n0, mtp_num_hidden_layers=0,+ layer_types=["linear_attention"] * n_chip + tc0["layer_types"],+ vocab_size=-(-(v0 + S.VOCAB) // 64) * 64, tie_word_embeddings=True,+ architectures=["Qwen3_5ForCausalLM"], model_type="qwen3_5_text")+ vocab = tc["vocab_size"]++ sinks = None+ if mute_qwen:+ from chipformer.q35.mute import fit_anti_sink++ dev = "cuda" if torch.cuda.is_available() else "cpu"+ sinks = fit_anti_sink(src, dev)+ torch.cuda.empty_cache() if torch.cuda.is_available() else None++ # Qwen's weights, widened and renamed one tensor at a time (keeps peak memory low)+ scale = (d0 / hidden) ** 0.5+ sd = {}+ base_emb = None+ for f in sorted(src.glob("*.safetensors")):+ with safe_open(f, "pt") as fh:+ for k in fh.keys(): # noqa: SIM118+ if not k.startswith(SRC_TEXT):+ continue # vision tower, mtp+ w = fh.get_tensor(k)+ if k == SRC_TEXT + "embed_tokens.weight":+ base_emb = w+ continue+ if IN_COLS.search(k):+ w = torch.cat([w, w.new_zeros(*w.shape[:-1], hidden - d0)], -1)+ elif OUT_ROWS.search(k):+ w = torch.cat([w, w.new_zeros(hidden - d0, *w.shape[1:])], 0)+ elif WIDE_NORMS.search(k):+ g = (1.0 + w.float()) * scale - 1.0+ w = torch.cat([g.to(w.dtype), w.new_zeros(hidden - d0)])+ k2 = DST + k[len(SRC_TEXT):]+ k2 = re.sub(r"^(model\.layers\.)(\d+)\.", lambda mt: f"{mt[1]}{int(mt[2]) + n_chip}.",+ k2)+ sd[k2] = w+ assert base_emb is not None+ if mute_qwen:+ from chipformer.q35.mute import mute++ mute(sd, tc, n0, lay.dim(next(iter(c.mu.terms))), hidden**0.5, prefix=DST, sinks=sinks,+ first=n_chip)++ # the chip layers (fp32 while emitted, bf16 stored)+ layers = emit_layers(c, lay, tc, 0, DST, alloc, keep)+ for k in list(layers):+ sd[k] = layers.pop(k).to(torch.bfloat16)++ # embeddings: Qwen's rows, then the RV32 tokens+ emb = torch.zeros(vocab, hidden, dtype=torch.bfloat16)+ emb[:v0] = text_rows(c, lay, T, base_emb, v0)+ del base_emb+ chip = embedding_matrix(c, lay, S.VOCAB)+ emb[v0:v0 + S.VOCAB] = chip[:S.VOCAB].to(torch.bfloat16)+ del chip+ check_bf16("embedding", emb[:, d0:].float()[: min(vocab, 300000)])+ sd[DST + "embed_tokens.weight"] = emb++ code = [next(iter(e.terms)) for e in c.code]+ tau = next(iter(c.tau.terms))+ norm = sd[DST + "norm.weight"].float()+ norm[d0:] = -1.0 # chip dims are invisible to the lm_head ...+ for d in code: # ... except the output code and tau+ norm[lay.dim(d)] = code_gain(lay)+ norm[lay.dim(tau)] = TAU_GAIN+ check_bf16("final norm", norm[d0:])+ sd[DST + "norm.weight"] = norm.to(torch.bfloat16)++ dst.mkdir(parents=True, exist_ok=True)+ shard, size, shards = {}, 0, []+ for k in sorted(sd):+ shard[k] = sd[k].contiguous()+ size += sd[k].numel() * 2+ if size > 2**31:+ shards.append(shard)+ shard, size = {}, 0+ shards.append(shard)+ index = {"metadata": {"total_size": sum(t.numel() * 2 for t in sd.values())},+ "weight_map": {}}+ for i, sh in enumerate(shards):+ name = f"model-{i + 1:05d}-of-{len(shards):05d}.safetensors"+ save_file(sh, str(dst / name), metadata={"format": "pt"})+ index["weight_map"].update({k: name for k in sh})+ (dst / "model.safetensors.index.json").write_text(json.dumps(index, indent=2))+ (dst / "config.json").write_text(json.dumps(tc, indent=2))+ extend_tokenizer(src, dst, v0)+ gen = {"eos_token_id": [248046, tc0.get("eos_token_id", 248044)],+ "pad_token_id": tc0.get("eos_token_id", 248044), "bad_words_ids": [[IM_START]],+ "do_sample": False}+ (dst / "generation_config.json").write_text(json.dumps(gen, indent=2))+ meta = {"layout": "rv1", "chip_token_offset": v0, "chip_vocab": S.VOCAB,+ "chip_layers": n_chip, "qwen_layers_from": n_chip, "hidden": hidden,+ "chip_dims_offset": d0, "ballast_dims": m, "circuit": c.stats(),+ "slots": alloc.n_slots, "muted": mute_qwen,+ "code_dims": [lay.dim(d) for d in code], "tau_dim": lay.dim(tau),+ "mu_dim": lay.dim(next(iter(c.mu.terms))), "nfam": core.NFAM, "npay": core.NPAY,+ "tool_call": [T.TCO, T.TCC], "printable": T.printable}+ (dst / "chipformer.json").write_text(json.dumps(meta, indent=1))+ return meta+++def main() -> None:+ import argparse++ ap = argparse.ArgumentParser(description="Merge the RV32 machine into Qwen3.5 (layout rv1)")+ ap.add_argument("src")+ ap.add_argument("--out", default="dist/q4-rv1")+ ap.add_argument("--no-mute", action="store_true")+ ap.add_argument("--hidden", type=int)+ ap.add_argument("--ballast", type=int)+ args = ap.parse_args()+ meta = build_merged(args.src, args.out, not args.no_mute, args.hidden, args.ballast)+ print(json.dumps({k: meta[k] for k in ("hidden", "chip_layers", "slots", "circuit")}))+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/programs.py b/experimental/riscv/chipformer/rv/programs.pynew file mode 100644index 0000000..a38a89a--- /dev/null+++ b/experimental/riscv/chipformer/rv/programs.py@@ -0,0 +1,136 @@+"""Small RV32I test programs (assembly) for the reference and the circuit."""++from chipformer.rv.asm import assemble++LOOP = """+_start: li a0, 0+ li t0, 10+loop: addi a0, a0, 3+ addi t0, t0, -1+ bnez t0, loop+ li a7, 93+ ecall+"""++MEMCPY = """+_start: la a0, dst+ la a1, src+ li a2, 13+bytes: lbu t0, 0(a1)+ sb t0, 0(a0)+ addi a0, a0, 1+ addi a1, a1, 1+ addi a2, a2, -1+ bnez a2, bytes+ la a0, wdst+ la a1, src+ li a2, 3+words: lw t0, 0(a1)+ sw t0, 0(a0)+ addi a0, a0, 4+ addi a1, a1, 4+ addi a2, a2, -1+ bnez a2, words+ la t1, dst+ lw a0, 4(t1)+ lw a1, 8(t1)+ li a7, 93+ ecall+.data+src: .word 0x64636261, 0x68676665, 0xF00DBEEF, 0x80+dst: .space 16+wdst: .space 12+"""++FIB = """+_start: li a0, 25+ li t0, 0+ li t1, 1+fib: beqz a0, done+ add t2, t0, t1+ mv t0, t1+ mv t1, t2+ addi a0, a0, -1+ j fib+done: mv a0, t0+ li a7, 93+ ecall+"""++CALL = """+_start: li a0, 7+ call fact+ mv s1, a0+ li a0, 12+ call sum+ li a7, 93+ ecall+fact: addi sp, sp, -16+ sw ra, 12(sp)+ sw s0, 8(sp)+ mv s0, a0+ li t0, 2+ blt a0, t0, fact1+ addi a0, a0, -1+ call fact+ mv t1, a0+ li a0, 0+mloop: beqz s0, mdone # a0 = t1 * s0 by repeated addition (no M in P0)+ add a0, a0, t1+ addi s0, s0, -1+ j mloop+mdone: j fact2+fact1: li a0, 1+fact2: lw ra, 12(sp)+ lw s0, 8(sp)+ addi sp, sp, 16+ ret+sum: addi sp, sp, -16 # recursive sum 1..a0+ sw ra, 12(sp)+ sw a0, 8(sp)+ beqz a0, sum0+ addi a0, a0, -1+ call sum+ lw t0, 8(sp)+ add a0, a0, t0+ j sum1+sum0: li a0, 0+sum1: lw ra, 12(sp)+ addi sp, sp, 16+ ret+"""++SIGNEXT = """+_start: la t0, vals+ lb a0, 0(t0)+ lbu a1, 0(t0)+ lh a2, 2(t0)+ lhu a3, 2(t0)+ lb a4, 5(t0)+ lh a5, 6(t0)+ li t1, -2+ sh t1, 8(t0)+ sb t1, 11(t0)+ lw a6, 8(t0)+ lui a7, 0xABCDE+ srai s2, a7, 7+ srli s3, a7, 7+ slli s4, a7, 3+ slt s5, a7, zero+ sltu s6, a7, zero+ sltiu s7, a0, -1+ xori s8, a1, 0x7FF+ ori s9, a0, 0x123+ andi s10, a2, -256+ sub s11, a2, a7+ li a7, 93+ ecall+.data+vals: .word 0x8F7F80FF, 0x7F7F0080, 0x11223344+"""++ALL = {"loop": LOOP, "memcpy": MEMCPY, "fib": FIB, "call": CALL, "signext": SIGNEXT}+++def program(name: str):+ return assemble(ALL[name])diff --git a/experimental/riscv/chipformer/rv/ref.py b/experimental/riscv/chipformer/rv/ref.pynew file mode 100644index 0000000..5ffb64b--- /dev/null+++ b/experimental/riscv/chipformer/rv/ref.py@@ -0,0 +1,614 @@+"""Reference RV32IM + Zba: an ISA emulator, the pre-decoded ROM, and the trace (records) the+circuit must produce.++``Rv32`` plain RV32IM + Zba semantics (cross-checked against qemu-riscv32 on gcc-built ELFs).+``rom_word`` the circuit's view of an instruction: one pre-decoded 32-bit ROM word (see below).+``Machine`` the traced machine: ``record()`` returns the next record's tokens and commits it.++The ROM word is four bytes, each a field the circuit decodes in one MLP stage:++ b0 = uop | S0 << 7 b1 = RD << 3 | S1..S3 b2 = RS1 << 3 | S4..S6 b3 = RS2 << 3 | S7..S9++RD / RS1 / RS2 are 5-bit register slots (the regfile's three read ports query them) and S0..S9+are spare bits. Static facts are folded in, so every format fits:++ R-type (ALU, MUL, DIV, Zba) RD = rd, RS1 = rs1, RS2 = rs2+ I-type (ALU imm, loads, JALR) RD = rd, RS1 = rs1, imm12 = RS2 (bits 0..4) + S0..S6 (bits 5..11)+ stores RD = the data register, RS1 = base, imm12 as I-type+ branches RS1, RS2; the target as a signed word offset: RD (bits 0..4) ++ S0..S5 (bits 5..10); S6 = the target is bad (misaligned / not ROM)+ JAL RD = rd; the target's ROM word index: RS1 (0..4) + RS2 (5..9) ++ S0..S5 (10..15); S6 = bad+ LUI / AUIPC RD = rd; the value's upper 20 bits: RS1 (0..4) + RS2 (5..9) ++ S0..S9 (10..19); AUIPC's are (pc + imm) >> 12 (its low 12 bits+ are the PC's, in-circuit)+ CSR reads (stubs) LUI rd, 0, or MISA rd; anything else ILLEGAL+ ECALL RS1 = 17 (a7), RS2 = 10 (a0)+ digit-file ops (R-type) RD, RS1, RS2+ BPRD k: RS2 (bits 0..4) + S0..S1 (bits 5..6); S7..S9 = dD++The digit file: 8 registers d0..d7 of 128 decimal digits (index 0 least significant). Custom-0+(opcode 0x0B, R-type, funct7 = 0) takes its operands from rs2 = dD | dS << 3 | ovw << 6 | n << 7+| off << 13 | m << 20 (registers dD / dS, the overwrite flag, n = digits 0..63, off 0..127,+m = min(multiplier, 9); the common words fit a one-token value); custom-1 (opcode 0x2B, I-type) is+BPRD:++ BMAC f3 0 for i < n: v = (ovw ? 0 : dD[i + off]) + dS[i] * m + c; dD[i + off] = v % 10,+ c = v // 10 (c = 0 at i = 0); rd = c (add: m = 1; mul-row; ovw: copy / scale /+ zero with m = 0)+ BMSB f3 1 for i < n: v = dD[i + off] - dS[i] * m - b; dD[i + off] = v % 10, b = the borrow;+ rd = b (sub: m = 1; the div-digit step)+ BLD f3 2 for i < n: dD[i + off] = min(mem8[(rs1 - 1 - i) & 0x7FFF] & 15, 9); rd = 0+ (the ASCII digits ending at rs1 - 1, least significant last)+ BGET f3 3 rd = dD[off]+ BSCAN f3 4 rd = the number n (<= 63) of ASCII digits at mem8[(rs1 + i) & 0x7FFF], i = 0..+ BPRD 0x2B putchar('0' + dD[k]), k = imm[6:0], dD = rd & 7 (I-type)++Rows past 127 read as 0 and are not written. A row op's record is a BD token per digit (the+digit and the carry / borrow out), then rd's value record.+"""++from dataclasses import dataclass, field++from chipformer.rv import spec as S++UOPS = ["ILLEGAL", "LUI", "AUIPC", "MISA",+ "ADDI", "SLTI", "SLTIU", "XORI", "ORI", "ANDI", "SLLI", "SRLI", "SRAI",+ "ADD", "SUB", "SLL", "SLT", "SLTU", "XOR", "SRL", "SRA", "OR", "AND",+ "LB", "LH", "LW", "LBU", "LHU", "SB", "SH", "SW",+ "BEQ", "BNE", "BLT", "BGE", "BLTU", "BGEU", "JAL", "JALR",+ "ECALL", "EBREAK", "FENCE",+ "MUL", "MULH", "MULHSU", "MULHU", "DIV", "DIVU", "REM", "REMU",+ "SH1ADD", "SH2ADD", "SH3ADD"]+CUSTOM = {"BMAC": 64, "BMSB": 65, "BLD": 66, "BGET": 67, "BPRD": 68,+ "BSCAN": 69} # uops 64..127: custom+CSR_STUBS = {0xC00: 0, 0xC01: 0, 0xC02: 0, 0xC80: 0, 0xC81: 0, 0xC82: 0, 0xF14: 0,+ 0x301: S.MISA} # cycle, time, instret (+h), mhartid, misa+U = {n: i for i, n in enumerate(UOPS)} | CUSTOM+UOPS = UOPS + list(CUSTOM)+BN_ROW = {"BMAC", "BMSB", "BLD"}+WRITES_RD = {"LUI", "AUIPC", "MISA", "ADDI", "SLTI", "SLTIU", "XORI", "ORI", "ANDI", "SLLI",+ "SRLI", "SRAI", "ADD", "SUB", "SLL", "SLT", "SLTU", "XOR", "SRL", "SRA", "OR", "AND",+ "LB", "LH", "LW", "LBU", "LHU", "JAL", "JALR",+ "MUL", "MULH", "MULHSU", "MULHU", "DIV", "DIVU", "REM", "REMU",+ "SH1ADD", "SH2ADD", "SH3ADD"}+MUL_OPS = {"MUL", "MULH", "MULHSU", "MULHU"}+DIV_OPS = {"DIV", "DIVU", "REM", "REMU"}+ZBA = {"SH1ADD": 1, "SH2ADD": 2, "SH3ADD": 3}+IMM_OPS = {"ADDI", "SLTI", "SLTIU", "XORI", "ORI", "ANDI", "SLLI", "SRLI", "SRAI"}+BRANCH_OPS = {"BEQ", "BNE", "BLT", "BGE", "BLTU", "BGEU"}+LOAD_OPS = {"LB", "LH", "LW", "LBU", "LHU"}+STORE_OPS = {"SB", "SH", "SW"}+M32 = 0xFFFFFFFF+++def sx(v: int, bits: int) -> int:+ v &= (1 << bits) - 1+ return v - (1 << bits) if v >> (bits - 1) else v+++@dataclass+class Inst:+ op: str+ rd: int = 0+ rs1: int = 0+ rs2: int = 0+ imm: int = 0 # sign-extended immediate (branch/jump: byte offset)+ dd: int = 0 # digit-file registers and the overwrite flag (custom ops)+ ds: int = 0+ ovw: int = 0+++def decode(w: int) -> Inst:+ opc, rd, f3, rs1, rs2, f7 = w & 0x7F, w >> 7 & 31, w >> 12 & 7, w >> 15 & 31, w >> 20 & 31, w >> 25+ i_imm = sx(w >> 20, 12)+ s_imm = sx((w >> 25) << 5 | (w >> 7 & 31), 12)+ b_imm = sx((w >> 31 & 1) << 12 | (w >> 7 & 1) << 11 | (w >> 25 & 63) << 5 | (w >> 8 & 15) << 1, 13)+ j_imm = sx((w >> 31 & 1) << 20 | (w >> 12 & 0xFF) << 12 | (w >> 20 & 1) << 11+ | (w >> 21 & 0x3FF) << 1, 21)+ if opc == 0x37:+ return Inst("LUI", rd, imm=sx(w & 0xFFFFF000, 32))+ if opc == 0x17:+ return Inst("AUIPC", rd, imm=sx(w & 0xFFFFF000, 32))+ if opc == 0x6F:+ return Inst("JAL", rd, imm=j_imm)+ if opc == 0x67 and f3 == 0:+ return Inst("JALR", rd, rs1, imm=i_imm)+ if opc == 0x63 and f3 not in (2, 3):+ return Inst(["BEQ", "BNE", "", "", "BLT", "BGE", "BLTU", "BGEU"][f3], 0, rs1, rs2, b_imm)+ if opc == 0x03 and f3 in (0, 1, 2, 4, 5):+ return Inst(["LB", "LH", "LW", "", "LBU", "LHU"][f3], rd, rs1, imm=i_imm)+ if opc == 0x23 and f3 in (0, 1, 2):+ return Inst(["SB", "SH", "SW"][f3], 0, rs1, rs2, s_imm)+ if opc == 0x13:+ if f3 == 1 and f7 == 0:+ return Inst("SLLI", rd, rs1, imm=rs2)+ if f3 == 5 and f7 in (0, 0x20):+ return Inst("SRAI" if f7 else "SRLI", rd, rs1, imm=rs2)+ if f3 not in (1, 5):+ return Inst(["ADDI", "", "SLTI", "SLTIU", "XORI", "", "ORI", "ANDI"][f3], rd, rs1, imm=i_imm)+ if opc == 0x33:+ if f7 == 0:+ return Inst(["ADD", "SLL", "SLT", "SLTU", "XOR", "SRL", "OR", "AND"][f3], rd, rs1, rs2)+ if f7 == 0x20 and f3 in (0, 5):+ return Inst("SUB" if f3 == 0 else "SRA", rd, rs1, rs2)+ if f7 == 1:+ return Inst(["MUL", "MULH", "MULHSU", "MULHU", "DIV", "DIVU", "REM", "REMU"][f3], rd, rs1, rs2)+ if f7 == 0x10 and f3 in (2, 4, 6):+ return Inst({2: "SH1ADD", 4: "SH2ADD", 6: "SH3ADD"}[f3], rd, rs1, rs2)+ if opc == 0x0B and f3 < 5 and f7 == 0:+ return Inst(["BMAC", "BMSB", "BLD", "BGET", "BSCAN"][f3], rd, rs1, rs2)+ if opc == 0x2B and f3 == 0:+ return Inst("BPRD", imm=w >> 20 & 0x7F, dd=rd & 7)+ if w == 0x00000073:+ return Inst("ECALL")+ if opc == 0x73 and f3 in (1, 2, 3, 5, 6, 7):+ return Inst(["", "CSRRW", "CSRRS", "CSRRC", "", "CSRRWI", "CSRRSI", "CSRRCI"][f3], rd, rs1,+ imm=w >> 20)+ if w == 0x00100073:+ return Inst("EBREAK")+ if opc == 0x0F:+ return Inst("FENCE")+ return Inst("ILLEGAL")+++def alu(op: str, a: int, b: int) -> int:+ """Register-register / register-immediate results (a, b as unsigned 32-bit)."""+ sa, sb = sx(a, 32), sx(b, 32)+ if op in ("ADD", "ADDI"):+ r = a + b+ elif op == "SUB":+ r = a - b+ elif op in ("SLL", "SLLI"):+ r = a << (b & 31)+ elif op in ("SRL", "SRLI"):+ r = a >> (b & 31)+ elif op in ("SRA", "SRAI"):+ r = sa >> (b & 31)+ elif op in ("SLT", "SLTI"):+ r = int(sa < sb)+ elif op in ("SLTU", "SLTIU"):+ r = int(a < b)+ elif op in ("XOR", "XORI"):+ r = a ^ b+ elif op in ("OR", "ORI"):+ r = a | b+ elif op in ("AND", "ANDI"):+ r = a & b+ elif op == "MUL":+ r = a * b+ elif op == "MULH":+ r = (sa * sb) >> 32+ elif op == "MULHSU":+ r = (sa * b) >> 32+ elif op == "MULHU":+ r = (a * b) >> 32+ elif op == "DIV":+ r = -1 if sb == 0 else (sa if (sa, sb) == (-1 << 31, -1) else int(sa / sb))+ elif op == "DIVU":+ r = M32 if b == 0 else a // b+ elif op == "REM":+ r = sa if sb == 0 else (0 if (sa, sb) == (-1 << 31, -1) else sa - sb * int(sa / sb))+ elif op == "REMU":+ r = a if b == 0 else a % b+ elif op in ZBA:+ r = (a << ZBA[op]) + b+ else:+ raise ValueError(op)+ return r & M32+++def taken(op: str, a: int, b: int) -> bool:+ sa, sb = sx(a, 32), sx(b, 32)+ return {"BEQ": a == b, "BNE": a != b, "BLT": sa < sb, "BGE": sa >= sb,+ "BLTU": a < b, "BGEU": a >= b}[op]+++def nibbles_used(v: int) -> int:+ """How many of the 8 nibble prefixes v >> 4i (i = 0..7) are nonzero."""+ return sum(1 for i in range(8) if v >> 4 * i)+++def div_start(op: str, a: int, b: int) -> int:+ """k0, the first quotient digit a DIV-class record computes (digits 0..k0-1 are 0).++ |A| < 16^dA and |B| >= 16^(dB-1), so |A| / |B| has at most dA - dB + 1 hex digits. dA counts+ the nonzero prefixes of a's raw bits (8 when a negative signed operand sets bit 31: an+ overestimate, which is safe) and dB those of |B| - 1 for a negative signed b, else of b (an+ underestimate when b < 0, also safe). b = 0 computes all 8 digits (the quotient is all ones)."""+ sb = op in ("DIV", "REM") and b >> 31+ da = nibbles_used(a)+ db = nibbles_used((~b & M32) if sb else b)+ return 0 if b == 0 else min(max(7 - da + db, 0), 8)+++def div_parts(op: str, a: int, b: int) -> tuple[int, list[int], int]:+ """(k0, the quotient digits k0..7 of |A| / |B|, the result) of a DIV-class instruction."""+ signed = op in ("DIV", "REM")+ sa, sb = signed and a >> 31 == 1, signed and b >> 31 == 1+ ua, ub = (-a) & M32 if sa else a, (-b) & M32 if sb else b+ q = ua // ub if ub else M32+ k0 = div_start(op, a, b)+ assert q >> 4 * (8 - k0) == 0 if k0 < 8 else q == 0, (op, a, b, k0)+ digits = [q >> 4 * (7 - k) & 15 for k in range(k0, 8)]+ qs = (-q) & M32 if sa != sb and b else q # the signed quotient+ res = qs if op in ("DIV", "DIVU") else (a - qs * b) & M32+ assert res == alu(op, a, b), (op, a, b)+ return k0, digits, res+++def div_record(op: str, a: int, b: int) -> list[int]:+ k0, digits, res = div_parts(op, a, b)+ return ([S.tok("QS", k0)] + [S.tok("QD", (k0 + i) << 4 | q) for i, q in enumerate(digits)]+ + S.value_record(res))+++class Fault(Exception):+ pass+++@dataclass+class Rv32:+ """Plain RV32IM + Zba with a flat byte memory (dict pages) and ecall exit/putchar."""++ mem: dict[int, int] = field(default_factory=dict)+ x: list[int] = field(default_factory=lambda: [0] * 32)+ pc: int = 0+ out: bytearray = field(default_factory=bytearray)+ halted: bool = False+ exit_code: int | None = None+ steps: int = 0++ def load(self, addr: int, n: int) -> int:+ return int.from_bytes(bytes(self.mem.get(addr + i, 0) for i in range(n)), "little")++ def store(self, addr: int, v: int, n: int) -> None:+ for i in range(n):+ self.mem[addr + i] = v >> 8 * i & 0xFF++ def step(self) -> Inst:+ w = self.load(self.pc, 4)+ ins = decode(w)+ op, x = ins.op, self.x+ nxt = self.pc + 4+ a, b = x[ins.rs1], x[ins.rs2]+ res = None+ if op == "LUI":+ res = ins.imm & M32+ elif op == "AUIPC":+ res = (self.pc + ins.imm) & M32+ elif op in ("JAL", "JALR"):+ res = nxt+ nxt = (self.pc + ins.imm) & M32 if op == "JAL" else (a + ins.imm) & M32 & ~1+ elif op in BRANCH_OPS:+ if taken(op, a, b):+ nxt = (self.pc + ins.imm) & M32+ elif op == "EBREAK":+ self.halted = True+ elif op in LOAD_OPS:+ ea = (a + ins.imm) & M32+ n = {"LB": 1, "LBU": 1, "LH": 2, "LHU": 2, "LW": 4}[op]+ v = self.load(ea, n)+ res = (sx(v, 8 * n) if op in ("LB", "LH") else v) & M32+ elif op in STORE_OPS:+ ea = (a + ins.imm) & M32+ self.store(ea, b, {"SB": 1, "SH": 2, "SW": 4}[op])+ elif op == "ECALL":+ if x[17] == 93:+ self.halted, self.exit_code = True, x[10]+ elif x[17] in (1, 64): # putchar(a0) / write(1, a1, a2)+ if x[17] == 1:+ self.out.append(x[10] & 0xFF)+ else:+ self.out += bytes(self.load(x[11] + i, 1) for i in range(x[12]))+ res = None+ else:+ self.halted = True+ elif op == "FENCE":+ pass+ elif op == "ILLEGAL" or op.startswith("CSR"):+ raise Fault(f"unsupported instruction {w:#010x} at {self.pc:#x}")+ elif op in IMM_OPS:+ res = alu(op, a, ins.imm & M32)+ else:+ res = alu(op, a, b)+ if res is not None and ins.rd:+ x[ins.rd] = res+ self.pc = nxt & M32+ self.steps += 1+ return ins+++# --- the circuit's ROM ------------------------------------------------------------------------+++def csr_read(ins: Inst) -> int | None:+ """The value of a supported pure CSR read (csrr), else None (an illegal-instruction fault)."""+ pure = ins.op in ("CSRRS", "CSRRC", "CSRRSI", "CSRRCI") and ins.rs1 == 0+ return CSR_STUBS.get(ins.imm) if pure else None+++def in_rom(t: int) -> bool:+ return t % 4 == 0 and S.ROM_BASE <= t < S.ROM_BASE + 4 * S.ROM_WORDS+++def pack(uop: str, rd: int = 0, rs1: int = 0, rs2: int = 0, spare: int = 0) -> int:+ """The ROM word from its uop, register slots and spare bits S0..S9."""+ b0 = U[uop] | (spare & 1) << 7+ b1 = rd << 3 | (spare >> 1 & 7)+ b2 = rs1 << 3 | (spare >> 4 & 7)+ b3 = rs2 << 3 | (spare >> 7 & 7)+ return b0 | b1 << 8 | b2 << 16 | b3 << 24+++def rom_word(w: int, pc: int) -> int:+ """The pre-decoded ROM word of the instruction word w at pc (format in the module doc)."""+ ins = decode(w)+ op, imm = ins.op, ins.imm+ if op.startswith("CSR"):+ v = csr_read(ins)+ if v is None:+ return pack("ILLEGAL")+ return pack("MISA", ins.rd) if v else pack("LUI", ins.rd)+ if op in ("LUI", "AUIPC"):+ up = (imm if op == "LUI" else pc + imm) >> 12 & 0xFFFFF+ return pack(op, ins.rd, up & 31, up >> 5 & 31, up >> 10)+ if op in IMM_OPS or op in LOAD_OPS or op == "JALR":+ i = imm & 0xFFF+ return pack(op, ins.rd, ins.rs1, i & 31, i >> 5)+ if op in STORE_OPS:+ i = imm & 0xFFF+ return pack(op, ins.rs2, ins.rs1, i & 31, i >> 5)+ if op in BRANCH_OPS:+ t = (pc + imm) & M32+ off = (imm >> 2) & 0x7FF # signed word offset, 11 bits+ bad = int(not in_rom(t))+ return pack(op, off & 31, ins.rs1, ins.rs2, off >> 5 | bad << 6)+ if op == "JAL":+ t = (pc + imm) & M32+ wi = (t - S.ROM_BASE) >> 2 & 0xFFFF+ bad = int(not in_rom(t))+ return pack(op, ins.rd, wi & 31, wi >> 5 & 31, wi >> 10 | bad << 6)+ if op == "ECALL":+ return pack(op, 0, 17, 10)+ if op in CUSTOM:+ if op == "BPRD":+ return pack(op, 0, 0, imm & 31, imm >> 5 | ins.dd << 7)+ return pack(op, ins.rd, ins.rs1, ins.rs2)+ if op in ("EBREAK", "FENCE", "ILLEGAL"):+ return pack(op)+ return pack(op, ins.rd, ins.rs1, ins.rs2) # R-type+++# --- the traced machine -----------------------------------------------------------------------+++@dataclass+class Machine:+ """State the trace commits: registers, RAM words, anchor, n. ROM and DROM are read-only."""++ rom: list[int] # instruction words from ROM_BASE+ ram: list[int] # words from RAM_BASE+ x: list[int]+ anchor: int # PC word index of the latest J / R+ n: int = 0+ halted: bool = False+ steps: int = 0 # instructions executed+ drom: bytes = b"" # read-only data from DROM_BASE+ bn: list = field(default_factory=lambda: [[0] * 128 for _ in range(8)]) # the digit file++ @classmethod+ def from_program(cls, prog, entry: int | None = None) -> "Machine":+ ram = [0] * S.RAM_WORDS+ off = prog.data_base - S.RAM_BASE+ data = bytes(off) + prog.data+ data += bytes(-len(data) % 4)+ for i in range(0, len(data), 4):+ ram[i // 4] = int.from_bytes(data[i:i + 4], "little")+ x = [0] * 32+ x[2] = S.SP_INIT+ rom = list(prog.text) + [0] * (S.ROM_WORDS - len(prog.text))+ e = prog.entry if entry is None else entry+ rodata = getattr(prog, "rodata", b"")+ assert len(rodata) <= 4 * S.DROM_WORDS, "rodata does not fit the DROM"+ return cls(rom, ram, x, (e - S.ROM_BASE) >> 2, drom=bytes(rodata))++ out: bytearray = field(default_factory=bytearray) # putchar output+ fault: tuple[int, int] | None = None # (cause, pc) of the fault that halted the run+ exit_code: int | None = None++ @property+ def pc(self) -> int:+ return S.ROM_BASE + 4 * (self.anchor + self.n)++ def ram_index(self, ea: int) -> int | None:+ return (ea - S.RAM_BASE) >> 2 if 0 <= ea - S.RAM_BASE < 4 * S.RAM_WORDS else None++ text: list[bytes] | None = None # the P3 text buffer rows (TEXT_ROW bytes each), if any++ def text_word(self, ea: int) -> int | None:+ """The word of the text buffer at ea (0 for unwritten rows), None outside it."""+ o = ea - S.TEXT_ADDR+ if self.text is None or not 0 <= o < S.TEXT_ROW * S.TEXT_ROWS:+ return None+ row = self.text[o // S.TEXT_ROW]+ return int.from_bytes(row[o % S.TEXT_ROW & ~3:][:4].ljust(4, bytes(1)), "little")++ def drom_word(self, ea: int) -> int | None:+ """The DROM word holding byte ea, or None outside the DROM."""+ o = ea - S.DROM_BASE+ if not 0 <= o < 4 * S.DROM_WORDS:+ return None+ o &= ~3+ return int.from_bytes(self.drom[o:o + 4].ljust(4, b"\0"), "little")++ def record(self) -> list[int]:+ """The next record's tokens; commits it (state as after its last token)."""+ if self.n == S.N_MAX:+ w = self.anchor + self.n+ self.anchor, self.n = w, 0+ return [S.tok("R", w)]+ pc = self.pc+ word = self.rom[(pc - S.ROM_BASE) >> 2]+ ins = decode(word)+ op, x = ins.op, self.x+ a, b = x[ins.rs1], x[ins.rs2]+ self.steps += 1++ def fault(cause):+ self.halted, self.fault = True, (S.FAULTS.index(cause), pc)+ return S.fault_record(S.FAULTS.index(cause), pc)++ def commit_rd(v):+ self.n += 1+ if ins.rd:+ x[ins.rd] = v & M32+ return S.value_record(v)+ return [S.tok("NT")]++ def jump(target, link):+ if not in_rom(target):+ return fault("bad jump")+ if link and ins.rd:+ x[ins.rd] = (pc + 4) & M32+ self.anchor, self.n = (target - S.ROM_BASE) >> 2, 0+ return [S.tok("J", self.anchor)]++ if op in ("LUI", "AUIPC") or op.startswith("CSR"):+ v = ins.imm if op == "LUI" else pc + ins.imm if op == "AUIPC" else csr_read(ins)+ return fault("illegal") if v is None else commit_rd(v)+ if op in ("JAL", "JALR"):+ target = (pc + ins.imm) & M32 if op == "JAL" else (a + ins.imm) & M32 & ~1+ return jump(target, True)+ if op in BRANCH_OPS:+ if taken(op, a, b):+ return jump((pc + ins.imm) & M32, False)+ self.n += 1+ return [S.tok("NT")]+ if op in LOAD_OPS:+ ea = (a + ins.imm) & M32+ n = {"LB": 1, "LBU": 1, "LH": 2, "LHU": 2, "LW": 4}[op]+ if ea % n:+ return fault("misaligned load")+ i = self.ram_index(ea)+ word = self.ram[i] if i is not None else self.drom_word(ea)+ if word is None:+ word = self.text_word(ea)+ if word is None:+ return fault("load out of range")+ wv = word >> 8 * (ea & 3) & ((1 << 8 * n) - 1)+ return commit_rd(sx(wv, 8 * n) if op in ("LB", "LH") else wv)+ if op in STORE_OPS:+ ea = (a + ins.imm) & M32+ n = {"SB": 1, "SH": 2, "SW": 4}[op]+ if ea % n:+ return fault("misaligned store")+ i = self.ram_index(ea)+ if i is None:+ return fault("store out of range")+ w4 = i >> 4 & 1+ self.n += 1+ if op == "SW":+ self.ram[i] = b+ return [S.tok(f"ST{w4}")]+ sh, mask = 8 * (ea & 3), (1 << 8 * n) - 1+ m = self.ram[i] & ~(mask << sh) & M32 | (b & mask) << sh+ self.ram[i] = m+ return [S.tok("V", m & 0x7FFF), S.tok("V", m >> 15 & 0x7FFF), S.tok("V", m >> 30),+ S.tok(f"SS{w4}")]+ if op == "ECALL":+ a, b = x[17], x[10] # the call (a7) and its argument (a0)+ if a == 1: # putchar(a0): a byte, or (256..) a special token of the P3 bridge+ if b & 0x1FF < 256:+ self.out.append(b & 0xFF)+ self.n += 1+ return [S.tok("BYTE", b & 0x1FF)]+ if a == 93: # exit(a0)+ self.halted, self.exit_code = True, b+ self.n += 1+ return [S.tok("HALT")]+ return fault("ecall")+ if op == "EBREAK":+ return fault("ebreak")+ if op == "FENCE":+ self.n += 1+ return [S.tok("NT")]+ if op == "ILLEGAL":+ return fault("illegal")+ if op in CUSTOM:+ return self.digit_op(ins, a, b)+ if op in DIV_OPS:+ self.n += 1+ if not ins.rd:+ return [S.tok("NT")]+ x[ins.rd] = div_parts(op, a, b)[2]+ return div_record(op, a, b)+ if op in IMM_OPS:+ return commit_rd(alu(op, a, ins.imm & M32))+ return commit_rd(alu(op, a, b))++ def ram_byte(self, ea: int) -> int:+ """A byte of RAM at ea mod 32 KB (the digit-file ops' addressing)."""+ ea &= 0x7FFF+ return self.ram[ea >> 2] >> 8 * (ea & 3) & 0xFF++ def digit_op(self, ins: Inst, a: int, b: int) -> list[int]:+ """A digit-file instruction's record (see the module doc)."""+ op, f, x = ins.op, self.bn, self.x+ self.n += 1+ if op == "BPRD":+ c = 48 + f[ins.dd][ins.imm]+ self.out.append(c)+ return [S.tok("BYTE", c)]+ if op == "BSCAN":+ n = 0+ while n < 63 and 48 <= self.ram_byte(a + n) <= 57:+ n += 1+ toks = [S.tok("SC")] * n+ if ins.rd:+ x[ins.rd] = n+ return toks + S.value_record(n)+ return toks + [S.tok("NT")]+ dd, ds, ovw = b & 7, b >> 3 & 7, b >> 6 & 1+ n, off, m = b >> 7 & 63, b >> 13 & 127, min(b >> 20 & 15, 9)+ if op == "BGET":+ v = f[dd][off]+ if ins.rd:+ x[ins.rd] = v+ return S.value_record(v)+ return [S.tok("NT")]+ toks, c = [], 0+ for i in range(n):+ r = i + off+ d = f[dd][r] if r < 128 else 0+ if op == "BLD":+ v, c = min(self.ram_byte(a - 1 - i) & 15, 9), 0+ elif op == "BMAC":+ v = (0 if ovw else d) + f[ds][i] * m + c+ v, c = v % 10, v // 10+ else:+ v = d - f[ds][i] * m - c+ v, c = v % 10, (v % 10 - v) // 10+ if r < 128:+ f[dd][r] = v+ toks.append(S.tok("BD", v | c << 4))+ if ins.rd:+ x[ins.rd] = c+ return toks + S.value_record(c)+ return toks + [S.tok("NT")]++ def run(self, max_records: int = 10**6) -> list[int]:+ """The whole trace from GO to HALT."""+ toks = [S.tok("GO")]+ while not self.halted and len(toks) < max_records:+ toks += self.record()+ return toks+++def trace_stats(toks: list[int]) -> dict:+ fams = [S.family(t)[0] for t in toks]+ # one final token per instruction, except R refreshes; a fault's FLT .. HALT is one+ instr = sum(1 for f in fams if f in S.FINAL and f != "R")+ return {"tokens": len(toks), "instructions": instr,+ "tokens_per_instruction": len(toks) / max(instr, 1),+ "refreshes": fams.count("R")}diff --git a/experimental/riscv/chipformer/rv/replay.py b/experimental/riscv/chipformer/rv/replay.pynew file mode 100644index 0000000..d7d9b65--- /dev/null+++ b/experimental/riscv/chipformer/rv/replay.py@@ -0,0 +1,92 @@+"""Offline IR replay of rv1 eval records (no model needed).++ python -m chipformer.rv.replay MODEL_DIR records.jsonl [--firmware ELF] [--limit N]++Each record's prompt and generated ids (evalarith stores them) become the circuit's token+stream; the reference machine (fed the prompt as history and Qwen's own tokens) must produce the+same chip tokens and echoes, and the IR circuit must own exactly the reference's positions and+predict every token there.+"""++import argparse+import json+from pathlib import Path++import numpy as np++from chipformer.q35 import ir+from chipformer.q35.ir import Target+from chipformer.rv import bridge as BR+from chipformer.rv import core, elf, ref+from chipformer.rv import spec as S+++def replay_ids(c, T, v0, prompt, gen, firmware) -> dict:+ stream = [i - v0 if i >= v0 else BR.text_token(i) for i in prompt + gen]+ m = ref.Machine.from_program(elf.program(firmware))+ BR.park(m)+ ses = BR.Session(T, m)+ ses.history(prompt)+ skip = {x for x in range(len(prompt)) if prompt[x] == T.TCC}+ pos = len(prompt)+ while pos < len(stream):+ t = stream[pos]+ if not S.is_text(t):+ return {"ok": False, "where": pos, "error": "chip token without a run"}+ k = len(ses.toks)+ ses.qwen([t - S.TEXT_TOKEN])+ pos += 1+ added = ses.toks[k + 1:]+ if added:+ got = stream[pos:pos + len(added)]+ if got != added[:len(got)]:+ return {"ok": False, "where": pos, "error": "stream differs from the reference"}+ pos += len(added)+ toks = ses.toks[:len(stream)]+ assert toks == stream+ m2 = ref.Machine.from_program(elf.program(firmware))+ BR.park(m2)+ core.snapshot(c, m2)+ carry, mats = {}, []+ for a in range(0, len(stream), 2048):+ mats.append(ir.evaluate(c, stream[a:a + 2048], carry=carry))+ mat = np.concatenate(mats)+ pred = core.predictions(c, mat)+ own = np.rint(ir.ev(c.owner, mat)).astype(int)+ bad_own = int(sum(own[x] != int(ses.own[x]) for x in range(len(stream))))+ bad = int(sum(1 for x in range(len(stream) - 1)+ if ses.own[x] and x not in skip and pred[x] != stream[x + 1]))+ return {"ok": bad == 0 and bad_own == 0, "positions": len(stream),+ "owned": int(sum(ses.own)) - len(skip), "wrong_predictions": bad,+ "owner_mismatch": bad_own, "runs": len(ses.runs)}+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("model")+ ap.add_argument("records")+ ap.add_argument("--firmware", default="tests/data/rv/calc-bridge.elf")+ ap.add_argument("--limit", type=int, default=10**9)+ args = ap.parse_args()+ meta = json.loads((Path(args.model) / "chipformer.json").read_text())+ T = BR.Text(Path(args.model) / "tokenizer.json")+ cfg = json.loads((Path(args.model) / "config.json").read_text())+ c = core.build(Target(cfg["linear_num_key_heads"], cfg["linear_num_value_heads"],+ cfg["intermediate_size"]), T)+ tot = {"records": 0, "ok": 0, "positions": 0, "owned": 0}+ for line in Path(args.records).read_text().splitlines()[: args.limit]:+ r = json.loads(line)+ if "gen_ids" not in r:+ continue+ res = replay_ids(c, T, meta["chip_token_offset"], r["prompt_ids"], r["gen_ids"],+ args.firmware)+ print(r["id"], json.dumps(res), flush=True)+ tot["records"] += 1+ tot["ok"] += res["ok"]+ tot["positions"] += res.get("positions", 0)+ tot["owned"] += res.get("owned", 0)+ print(json.dumps(tot))+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/spec.py b/experimental/riscv/chipformer/rv/spec.pynew file mode 100644index 0000000..b8a5aed--- /dev/null+++ b/experimental/riscv/chipformer/rv/spec.py@@ -0,0 +1,159 @@+"""Tokens, records and the address map of the RV32 machine.++A trace is one record per instruction. Machine state changes only at a record's last token (its+*commit*); the PC is implied: PC = ROM_BASE + 4 * (anchor + n), where the anchor is the payload of+the latest J or R token and n counts the commits since then (an R refresh comes before the+instruction when n reaches 63).++Token families (chip vocabulary ids from 0):++ V 15-bit chunk of a value, not final+ F0 final; the value is this payload sign-extended from bit 14 (a 1-token value record)+ F1 final; after one V: value bits 15..29, sign-extended from bit 29+ F2 final; after two V: value bits 30..31+ J final; jump: anchor = payload (a 16-bit ROM word index), n = 0+ R final; anchor refresh: anchor = payload, n = 0, nothing executes+ BYTE final; putchar (ecall a7 = 1): a0 & 511 (0..255 a byte; 256.. a special token, P3)+ FLT not final; a fault: payload = cause (FAULTS), then 8 HEX digits of the PC, then HALT+ HEX not final; a hex digit of a fault record+ QS not final; DIV/DIVU/REM/REMU: k0, the index of the first quotient digit computed (0..8)+ QD not final; a quotient digit: payload k << 4 | q (digit k of 8, most significant first)+ BD not final; a digit-file row step (custom ops): payload digit | carry << 4+ NT, ST0, ST1, SS0, SS1, HALT final controls (ST/SS: the RAM value head, address bit 6)+ GO not final: the start of a run (after a state snapshot)+ SC not final: BSCAN found one more ASCII digit++Records:++ rd-writing instruction (ALU, MUL, load, LUI, AUIPC, CSR read) F0 | V F1 | V V F2 (by size)+ DIV / DIVU / REM / REMU with rd != x0 QS QD.. then the value record+ BMAC / BMSB / BLD (n digits) BD x n, then F0(carry) | NT+ BGET F0(digit) | NT+ BSCAN (n digit chars) SC x n, then F0(n) | NT+ BPRD BYTE('0' + digit)+ the same with rd = x0, not-taken branch, FENCE NT+ taken branch, JAL, JALR J (the link is in-circuit)+ SW ST0 / ST1+ SB, SH V V V SS0 / SS1 (merged word)+ ECALL a7 = 1 (putchar a0) BYTE+ ECALL a7 = 93 (exit) HALT+ a fault FLT(cause) HEX x 8 HALT+ before the instruction, when n == 63 R++Memory (P2):++ RAM 0x0000_0000 + 32 KB words; data / bss / heap from 0x100, the stack below 0x8000+ (data pointers below 16 KB fit a 1-token value)+ DROM 0x0002_0000 + 128 KB read-only data (.rodata), bytes; loads only+ ROM 0x0004_0000 + 256 KB code only, pre-decoded (ref.rom_word), filled from a snapshot++The digit file (custom-0 / custom-1 ops, see ref.py): 8 registers of 128 decimal digits (least+significant first), outside the address space, saved in the snapshot like RAM.++A DIV-class record: QS(k0), then QD(k << 4 | q_k) for k = k0..7, then the result's value record.+q_k are the hex digits of |A| / |B| (most significant first); k0 skips digits that are known to+be 0 from the operands' sizes (see ref.div_start), so small quotients take few tokens.+"""++FAMILY_SIZES = [("V", 1 << 15), ("F0", 1 << 15), ("F1", 1 << 15), ("F2", 4), ("J", 1 << 16),+ ("R", 1 << 16), ("BYTE", 512), ("FLT", 8), ("HEX", 16), ("QS", 16), ("QD", 128),+ ("BD", 256)]+FAMILIES = {}+_base = 0+for _n, _s in FAMILY_SIZES:+ FAMILIES[_n] = (_base, _s)+ _base += _s+CTRL = _base+CONTROL_NAMES = ["NT", "ST0", "ST1", "SS0", "SS1", "HALT", "GO", "SC"]+CONTROLS = {n: CTRL + i for i, n in enumerate(CONTROL_NAMES)}+VOCAB = CTRL + len(CONTROL_NAMES)+FINAL = {"F0", "F1", "F2", "J", "R", "BYTE", "NT", "ST0", "ST1", "SS0", "SS1", "HALT"}+PAYLOAD_BITS = {n: (s - 1).bit_length() for n, s in FAMILY_SIZES}++FAULTS = ["illegal", "ebreak", "misaligned load", "misaligned store", "load out of range",+ "store out of range", "bad jump", "ecall"]++ROM_BASE, ROM_WORDS = 0x40000, 1 << 16 # 256 KB of code (two sublayers)+DROM_BASE, DROM_WORDS = 0x20000, 1 << 15 # 128 KB of read-only data (one sublayer)+RAM_BASE, RAM_WORDS = 0x0, 1 << 13 # 32 KB (two sublayers)+DATA_BASE = 0x100 # .data starts here (address 0 stays a null pointer)+SP_INIT = RAM_BASE + 4 * RAM_WORDS+N_MAX = 63 # the R refresh comes before the instruction when n == N_MAX+MISA = 0x40001100 # RV32, I and M+# (P3) the text buffer, loads only: row i (1..CAP_MAX) holds the i-th token of the latest tool+# call's text, its UTF-8 bytes NUL-padded to TEXT_ROW, at TEXT_ADDR + TEXT_ROW * i+TEXT_ADDR, TEXT_ROW, TEXT_ROWS = 0x8000, 16, 128+# the capture counter's thermometer weights are 8 * 2^m: m < 88 keeps them below bf16's max+# after the emitter's 2^36 fold+CAP_MAX = 80+TEXT_TOKEN = 1 << 24 # (P3) a Qwen text token in the circuit's token stream: TEXT_TOKEN | qwen_id+++def is_text(t: int) -> bool:+ return t >= TEXT_TOKEN+++def family(t: int) -> tuple[str, int]:+ """(family or control name, payload) of a token."""+ if t >= CTRL:+ if t >= VOCAB:+ raise ValueError(f"not an rv token: {t}")+ return CONTROL_NAMES[t - CTRL], 0+ for name, (base, size) in FAMILIES.items():+ if base <= t < base + size:+ return name, t - base+ raise ValueError(f"not an rv token: {t}")+++def tok(name: str, payload: int = 0) -> int:+ if name in CONTROLS:+ return CONTROLS[name]+ base, size = FAMILIES[name]+ assert 0 <= payload < size, (name, payload)+ return base + payload+++def is_final(t: int) -> bool:+ return family(t)[0] in FINAL+++def value_record(v: int) -> list[int]:+ """Tokens of a 32-bit value: as few chunks as its sign-extended size needs."""+ v &= 0xFFFFFFFF+ s = v - (1 << 32) if v >> 31 else v+ c0, c1, c2 = v & 0x7FFF, v >> 15 & 0x7FFF, v >> 30+ if -(1 << 14) <= s < 1 << 14:+ return [tok("F0", c0)]+ if -(1 << 29) <= s < 1 << 29:+ return [tok("V", c0), tok("F1", c1)]+ return [tok("V", c0), tok("V", c1), tok("F2", c2)]+++def decode_value(tokens: list[int]) -> int:+ """Inverse of value_record (the commit's view: the final token and the V before it)."""+ names = [family(t) for t in tokens]+ (fam, p) = names[-1]+ if fam == "F0":+ v = p | (0xFFFF8000 if p >> 14 else 0)+ elif fam == "F1":+ v = names[-2][1] | p << 15 | (0xC0000000 if p >> 14 else 0)+ elif fam == "F2":+ v = names[-3][1] | names[-2][1] << 15 | p << 30+ else:+ raise ValueError(fam)+ return v & 0xFFFFFFFF+++def fault_record(cause: int, pc: int) -> list[int]:+ return ([tok("FLT", cause)] + [tok("HEX", pc >> 4 * (7 - i) & 15) for i in range(8)]+ + [tok("HALT")])+++def sign32(v: int) -> int:+ v &= 0xFFFFFFFF+ return v - (1 << 32) if v >> 31 else v+++def name(t: int) -> str:+ fam, p = family(t)+ return fam if fam in CONTROLS else f"{fam}({p})"diff --git a/experimental/riscv/chipformer/rv/textcheck.py b/experimental/riscv/chipformer/rv/textcheck.pynew file mode 100644index 0000000..fdb8e3c--- /dev/null+++ b/experimental/riscv/chipformer/rv/textcheck.py@@ -0,0 +1,90 @@+"""Text fidelity of a merged rv1 checkpoint against its base Qwen3.5.++ python -m chipformer.rv.textcheck BASE MERGED [--out dist/rv-text.json]++On report.PROMPTS: base greedy continuations (32 tokens), then teacher-forced next-token+distributions over Qwen's vocabulary on prompt + continuation. KL(base || merged) is compared+with the noise floor KL(base sdpa || base eager attention). Also: the merged model's own greedy+continuations (plain HF generate) and the merged tokenizer's encodings against the base's.+"""++import argparse+import gc+import json++import torch++from chipformer.q35.report import PROMPTS, encode++N_NEW = 32+SAMPLES = ["Hello, world! 123456789 * 987654321 = ?", " indented\n\tcode() # comment",+ "naïve café — 東京 🙂", "<tool_call>\n<function=calc>\n<parameter=expression>\n1+2\n"+ "</parameter>\n</function>\n</tool_call>"]++[email protected]_grad()+def logprobs(model, seqs, nq, dev):+ out = []+ for g in seqs:+ lg = model(input_ids=torch.tensor([g], device=dev)).logits[0, :, :nq].float()+ out.append(torch.log_softmax(lg, -1).half().cpu())+ return out+++def kl(a, b):+ return [float((p.float().exp() * (p.float() - q.float())).sum(-1).mean()) for p, q in zip(a, b)]+++def main() -> None:+ from transformers import AutoModelForImageTextToText, AutoTokenizer, Qwen3_5ForCausalLM++ ap = argparse.ArgumentParser()+ ap.add_argument("base")+ ap.add_argument("merged")+ ap.add_argument("--out", default="dist/rv-text.json")+ args = ap.parse_args()+ dev = "cuda"+ tb = AutoTokenizer.from_pretrained(args.base)+ tm = AutoTokenizer.from_pretrained(args.merged)+ tok_same = [tb(s).input_ids == tm(s).input_ids for s in SAMPLES]+ prompts = [encode(tb, k, t) for k, t in PROMPTS]+ nq = len(tb)++ base = AutoModelForImageTextToText.from_pretrained(args.base, dtype=torch.bfloat16).to(dev)+ seqs = []+ with torch.no_grad():+ for ids in prompts:+ g = base.generate(torch.tensor([ids], device=dev), max_new_tokens=N_NEW,+ do_sample=False)+ seqs.append(g[0].tolist())+ lb = logprobs(base, seqs, nq, dev)+ del base+ gc.collect()+ torch.cuda.empty_cache()+ eager = AutoModelForImageTextToText.from_pretrained(args.base, dtype=torch.bfloat16,+ attn_implementation="eager").to(dev)+ le = logprobs(eager, seqs, nq, dev)+ del eager+ gc.collect()+ torch.cuda.empty_cache()+ merged = Qwen3_5ForCausalLM.from_pretrained(args.merged, dtype=torch.bfloat16).to(dev)+ lm = logprobs(merged, seqs, nq, dev)+ same_greedy = []+ with torch.no_grad():+ for ids, g in zip(prompts, seqs):+ h = merged.generate(torch.tensor([ids], device=dev), max_new_tokens=N_NEW,+ do_sample=False)+ same_greedy.append(h[0].tolist() == g)+ k_m, k_e = kl(lb, lm), kl(lb, le)+ top1 = [float((p.argmax(-1) == q.argmax(-1)).float().mean()) for p, q in zip(lb, lm)]+ res = {"kl_merged_mean": sum(k_m) / len(k_m), "kl_noise_floor_mean": sum(k_e) / len(k_e),+ "kl_merged": k_m, "kl_noise_floor": k_e, "top1_agree": top1,+ "greedy_identical": same_greedy, "tokenizer_identical": tok_same}+ print(json.dumps({k: v for k, v in res.items() if k.endswith("mean")+ or k in ("greedy_identical", "tokenizer_identical")}, indent=1))+ with open(args.out, "w") as f:+ json.dump(res, f, indent=1)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/timing.py b/experimental/riscv/chipformer/rv/timing.pynew file mode 100644index 0000000..f369a1b--- /dev/null+++ b/experimental/riscv/chipformer/rv/timing.py@@ -0,0 +1,169 @@+"""P0.5 timing: per-token decode cost of the RV32 core's chip layers at Qwen3.5-4B width.++ python -m chipformer.rv.timing [--hidden 3072] [--steps 300]++Builds the chip layers (8 here) at the given hidden size with 4B head shapes, then times batch-1+decode steps over a real trace (the snapshot injected): the dense HF modules, then the same+model after fast.compact (each linear map reads only its nonzero block, exact). Reports ms per+token, tokens/s and instructions/s (at the measured tokens per instruction).+"""++import argparse+import time++import torch++from chipformer.q35 import fast+from chipformer.q35.emit import Layout+from chipformer.rv import core, elf, hf, ref+from chipformer.rv.measure import DATA+++def timed_decode(model, c, ids, toks, steps):+ dev = next(model.parameters()).device+ x = torch.tensor([[ids[t] for t in toks]], device=dev)+ with torch.no_grad():+ out = model(input_ids=torch.tensor([[ids[None]]], device=dev), use_cache=True)+ cache = out.past_key_values+ hf.inject(c, cache)+ for t in range(20): # warm-up+ model(input_ids=x[:, t:t + 1], past_key_values=cache, use_cache=True)+ if dev.type == "cuda":+ torch.cuda.synchronize()+ t0 = time.time()+ for t in range(20, 20 + steps):+ model(input_ids=x[:, t:t + 1], past_key_values=cache, use_cache=True)+ if dev.type == "cuda":+ torch.cuda.synchronize()+ return (time.time() - t0) / steps+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--hidden", type=int, default=3072)+ ap.add_argument("--steps", type=int, default=1000)+ ap.add_argument("--prog", default="c-crc32")+ ap.add_argument("--executor-only", action="store_true")+ ap.add_argument("--batches", default="1,8,32", help="executor batch sizes (P1.6)")+ ap.add_argument("--sparse", default="0", help="executors: 0 dense blocks, 1 sparse, "+ "2 the Triton executor v2 (e.g. 0,1,2)")+ ap.add_argument("--hf-compile", action="store_true",+ help="also time rv-tiny's HF modules: static cache + torch.compile")+ args = ap.parse_args()+ c = core.build()+ prog = elf.program(DATA / f"{args.prog}.elf")+ core.snapshot(c, ref.Machine.from_program(prog))+ toks = ref.Machine.from_program(prog).run(10**6)+ n_instr = ref.trace_stats(toks)["instructions"]+ tpi = len(toks) / n_instr+ m = next(m for m in range(3, 16) if args.hidden % m == 0 and _is_pow4(args.hidden // m))+ dev = "cuda" if torch.cuda.is_available() else "cpu"+ batches = [int(b) for b in args.batches.split(",")]+ if args.hf_compile:+ hf_compiled(c, prog, toks, steps_of(args, toks), tpi)+ hf_compiled(c, prog, toks, steps_of(args, toks), tpi, compact=True)+ if args.executor_only:+ for sp in args.sparse.split(","):+ if sp == "2":+ run_executor2(c, toks, steps_of(args, toks), tpi, batches)+ else:+ run_executor(c, toks, steps_of(args, toks), tpi, dev, batches, sp == "1")+ return None+ import chipformer.rv.hf as H++ orig = H.pick_layout+ H.pick_layout = lambda n: Layout(hidden=args.hidden, m=m)+ try:+ model, _, ids = hf.build_model(c, toks)+ finally:+ H.pick_layout = orig+ model = model.to(dev)+ steps = steps_of(args, toks)+ dense = timed_decode(model, c, ids, toks, steps)+ counts = fast.compact(model, qwen=False, n_chip=c.n_layers())+ comp = timed_decode(model, c, ids, toks, steps)+ for name, s in (("dense", dense), ("compact", comp)):+ print(f"{name:8s} {1e3 * s:7.3f} ms/token {1 / s:8.0f} tok/s "+ f"{1 / s / tpi:8.0f} instr/s (hidden {args.hidden}, {c.n_layers()} layers, "+ f"{tpi:.2f} tok/instr)", flush=True)+ print("compact weight counts", counts)+ del model+ torch.cuda.empty_cache()+ run_executor(c, toks, steps, tpi, dev, batches)+++def hf_compiled(c, prog, toks, steps, tpi, compact=False):+ """rv-tiny's stock HF modules decoding free-running: a StaticCache (snapshot injected) and+ torch.compile(mode="reduce-overhead"), i.e. CUDA graphs. Checks the generated tokens."""+ from transformers import StaticCache++ from chipformer.rv import spec as S+ from chipformer.rv import tiny++ model, _ = tiny.build(c)+ model = model.cuda()+ if compact: # every linear map (and the tied embedding / output head) reads its nonzero block+ fast.compact(model, qwen=True, n_chip=c.n_layers())+ cache = StaticCache(config=model.config, max_cache_len=steps + 64)+ with torch.no_grad():+ model(input_ids=torch.tensor([[S.VOCAB]], device="cuda"), past_key_values=cache,+ use_cache=True, cache_position=torch.tensor([0], device="cuda"))+ core.snapshot(c, ref.Machine.from_program(prog))+ hf.inject(c, cache)+ fwd = torch.compile(model.forward, mode="reduce-overhead", fullgraph=False)+ tok = torch.tensor([[toks[0]]], device="cuda")+ got, t0 = [], None+ for t in range(steps):+ if t == 20:+ torch.cuda.synchronize()+ t0 = time.time()+ out = fwd(input_ids=tok, past_key_values=cache, use_cache=True,+ cache_position=torch.tensor([t + 1], device="cuda"))+ tok = out.logits[:, -1:].argmax(-1)+ got.append(tok)+ torch.cuda.synchronize()+ s = (time.time() - t0) / (steps - 20)+ got = torch.cat(got, 1)[0].tolist()+ ok = got == toks[1:steps + 1]+ print(f"HF (rv-tiny, hidden {model.layout.hidden}, StaticCache + torch.compile "+ f"reduce-overhead{', fast.compact' if compact else ''}) {1e3 * s:7.3f} ms/token {1 / s:8.0f} tok/s {1 / s / tpi:8.0f} "+ f"instr/s free-running == reference: {ok}", flush=True)+ del model, cache+ torch.cuda.empty_cache()+++def steps_of(args, toks):+ return min(args.steps, len(toks) - 25)+++def run_executor(c, toks, steps, tpi, dev, batches=(1,), sparse=False):+ from chipformer.rv import fastexec++ kind = "sparse" if sparse else "dense blocks"+ n = fastexec.check(c, toks[:300], dev, sparse=sparse)+ print(f"executor ({kind}) == IR on {n} positions ({dev})", flush=True)+ for b in batches:+ s = fastexec.timed(c, toks, steps, graph=True, compile_=True, batch=b, sparse=sparse)+ print(f"executor {kind} batch {b:3d} (compiled + CUDA graph) {1e3 * s:7.3f} ms/step "+ f"{b / s:9.0f} tok/s {b / s / tpi:9.0f} instr/s", flush=True)+++def run_executor2(c, toks, steps, tpi, batches=(1,)):+ from chipformer.rv import fastexec, tritonexec++ n = tritonexec.check(c, toks[:200])+ print(f"executor v2 (triton) == IR on {n} positions", flush=True)+ for b in batches:+ s = fastexec.timed2(c, toks, steps, batch=b)+ print(f"executor v2 (triton) batch {b:3d} (CUDA graph) {1e3 * s:7.3f} ms/step "+ f"{b / s:9.0f} tok/s {b / s / tpi:9.0f} instr/s", flush=True)+++def _is_pow4(x):+ while x > 1 and x % 4 == 0:+ x //= 4+ return x == 1+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/tiny.py b/experimental/riscv/chipformer/rv/tiny.pynew file mode 100644index 0000000..c9850e0--- /dev/null+++ b/experimental/riscv/chipformer/rv/tiny.py@@ -0,0 +1,113 @@+"""rv-tiny: the RV32 core as a standalone stock Qwen3_5ForCausalLM.++ python -m chipformer.rv.tiny --out dist/rv-tiny++Qwen3.5-4B's DeltaNet head shapes (16 key heads, 32 value heads, 128 dims) and MLP width (9216),+the circuit's layers, and one zero full-attention layer at the end (HF's cache counts positions+there). The vocabulary is the chip tokens (spec.py) plus PAD, a zero row: a run starts after a+one-token PAD prefix, with the program's snapshot written into the cache (firmware.py). The+output head is tied to the embedding: a row's code dims hold CODE_AMP * (2 * bit - 1) of its+token's code, and after the exit the residual holds the predicted code, so the argmax is the+token whose code bits all agree.+"""++import json+from pathlib import Path++import numpy as np+import torch++from chipformer.q35.emit import check_bf16, emit_layers+from chipformer.q35.tiny import BASE, CODE_GAIN, pick_layout+from chipformer.rv import core+from chipformer.rv import spec as S+++def embedding_matrix(c, lay, n_vocab: int) -> torch.Tensor:+ """Rows of every chip token (fast: a token's features are linear in its payload bits within+ its family), the PAD row (zero) and padding rows (zero)."""+ feats = sorted(c.token_fns)+ cols = [lay.dim(d) for d in feats]++ def vec(t):+ return np.array([float(c.feature_value(d, t)) for d in feats])++ e = np.zeros((n_vocab, lay.hidden), dtype=np.float32)+ for name, (base, size) in S.FAMILIES.items():+ v0 = vec(base)+ nb = (size - 1).bit_length()+ deltas = np.stack([vec(base + (1 << i)) - v0 for i in range(nb)]) if nb else None+ p = np.arange(size)+ rows = np.tile(v0, (size, 1))+ if size <= 4096: # small families: any features (e.g. QS / QD's digit index)+ rows = np.stack([vec(base + i) for i in range(size)])+ elif nb:+ bits = (p[:, None] >> np.arange(nb)[None, :]) & 1+ rows = rows + bits @ deltas+ e[base:base + size][:, cols] = rows+ e[base:base + size, lay.off:lay.off + lay.m] = lay.ballast+ for t in (base + size - 1, base + size // 3): # spot checks of the linearity+ assert (vec(t) == rows[t - base]).all(), (name, t)+ for name, t in S.CONTROLS.items():+ e[t][cols] = vec(t)+ e[t, lay.off:lay.off + lay.m] = lay.ballast+ e = torch.tensor(e)+ check_bf16("embedding", e)+ return e+++def build(c=None, dtype=torch.bfloat16):+ """(model, circuit): a Qwen3_5ForCausalLM; token id t is chip token t, id S.VOCAB is PAD."""+ from transformers import GenerationConfig, Qwen3_5ForCausalLM, Qwen3_5TextConfig++ c = c or core.build()+ lay = pick_layout(len(c.dims))+ n_chip = c.n_layers()+ n_vocab = -(-(S.VOCAB + 1) // 64) * 64+ tc = dict(BASE, hidden_size=lay.hidden, vocab_size=n_vocab, num_hidden_layers=n_chip + 1,+ layer_types=["linear_attention"] * n_chip + ["full_attention"],+ linear_num_key_heads=c.target.key_heads, linear_num_value_heads=c.target.value_heads,+ intermediate_size=c.target.intermediate, eos_token_id=S.tok("HALT"),+ pad_token_id=S.VOCAB, bos_token_id=S.VOCAB)+ sd = emit_layers(c, lay, tc, 0, "model.", None, set(c.keep))+ sd["model.embed_tokens.weight"] = embedding_matrix(c, lay, n_vocab)+ norm = torch.full((lay.hidden,), -1.0) # (1 + w) = 0: the output head sees only the code+ for d in c.keep:+ norm[lay.dim(d)] = CODE_GAIN+ check_bf16("final norm", norm)+ sd["model.norm.weight"] = norm+ model = Qwen3_5ForCausalLM(Qwen3_5TextConfig(**tc))+ for p in model.parameters():+ p.data.zero_()+ missing, unexpected = model.load_state_dict(sd, strict=False)+ assert not unexpected, unexpected[:5]+ assert all(k.startswith(f"model.layers.{n_chip}.") or k == "lm_head.weight" for k in missing)+ model.tie_weights()+ model.generation_config = GenerationConfig(eos_token_id=S.tok("HALT"), pad_token_id=S.VOCAB,+ do_sample=False)+ model.layout = lay+ return model.to(dtype).eval(), c+++def main() -> None:+ import argparse++ ap = argparse.ArgumentParser(description="Save the RV32 core as a standalone checkpoint")+ ap.add_argument("--out", default="dist/rv-tiny")+ args = ap.parse_args()+ model, c = build()+ out = Path(args.out)+ model.save_pretrained(out)+ lay = model.layout+ meta = {"machine": "chipformer rv32 (RV32IM minus DIV)", "vocab": S.VOCAB, "pad": S.VOCAB,+ "families": {k: list(v) for k, v in S.FAMILIES.items()}, "controls": S.CONTROLS,+ "hidden": lay.hidden, "ballast_dims": lay.m, "chip_layers": c.n_layers(),+ "circuit": c.stats(), "rom_base": S.ROM_BASE, "rom_words": S.ROM_WORDS,+ "ram_base": S.RAM_BASE, "ram_words": S.RAM_WORDS}+ (out / "chipformer.json").write_text(json.dumps(meta, indent=1))+ n = sum(p.numel() for p in model.parameters())+ print(f"{out}: {c.stats()}, hidden {lay.hidden}, {n / 1e6:.0f}M parameters")+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/tinycheck.py b/experimental/riscv/chipformer/rv/tinycheck.pynew file mode 100644index 0000000..f754c7e--- /dev/null+++ b/experimental/riscv/chipformer/rv/tinycheck.py@@ -0,0 +1,90 @@+"""P1.5 checks of rv-tiny (a stock Qwen3_5ForCausalLM):++ python -m chipformer.rv.tinycheck [--save dist/rv-tiny] [--device cuda]++1. The residual before the exit layer equals the IR evaluator's on every circuit dim, at every+ position (bf16; one prefill with the snapshot in the cache).+2. Free-running ``generate`` from GO reproduces the reference trace token for token.+"""++import argparse+import time++import numpy as np+import torch++from chipformer.q35 import ir+from chipformer.rv import core, elf, firmware, ref, tiny+from chipformer.rv import spec as S+from chipformer.rv.measure import DATA+from chipformer.rv.programs import program+++def residual_check(model, c, prog) -> int:+ dev = next(model.parameters()).device+ toks = ref.Machine.from_program(prog).run(10**6)+ core.snapshot(c, ref.Machine.from_program(prog))+ want = ir.evaluate(c, toks, check=False)+ # (twice: on a new shape fla's Triton autotuner runs the kernels several times, and they+ # update the recurrent state in place, so the first call's result is not to be used)+ for _ in range(2):+ cache = firmware.load_cache(model, c, ref.Machine.from_program(prog))+ with torch.no_grad():+ o = model(input_ids=torch.tensor([toks], device=dev), past_key_values=cache,+ use_cache=True, output_hidden_states=True)+ h = o.hidden_states[c.n_layers() - 1][0].double().cpu().numpy()+ lay = model.layout+ dims = [d for d in range(1, len(c.dims))+ if c.dims[d].level < 2 * c.n_layers() - 1 and d not in c.keep]+ bad = [d for d in dims if (h[:, lay.dim(d)] != want[:, d]).any()]+ if bad:+ d = bad[0]+ t = int(np.nonzero(h[:, lay.dim(d)] != want[:, d])[0][0])+ nd = [e for e in range(len(c.dims)) if c.dims[e].name.startswith(("nd==", "v>", "j>"))]+ raise AssertionError(f"{[c.dims[e].name for e in bad[:5]]} first at {t} "+ f"({S.name(toks[t])}): hf {h[t, lay.dim(d)]} ir {want[t, d]}; "+ + "; ".join(f"{c.dims[e].name} hf {h[t, lay.dim(e)]} ir {want[t, e]}"+ for e in nd if h[t, lay.dim(e)] != want[t, e]))+ return len(dims) * len(toks)+++def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--save", default=None)+ ap.add_argument("--device", default="cuda" if torch.cuda.is_available() else "cpu")+ ap.add_argument("--progs", default="signext,call,memcpy,rv32ui-lb,rv32um-mulh,c-crc32p,"+ "c-strops")+ args = ap.parse_args()+ t = time.time()+ model, c = tiny.build()+ n = sum(p.numel() for p in model.parameters())+ print(f"built rv-tiny: hidden {model.layout.hidden}, {c.n_layers() + 1} layers, "+ f"{n / 1e6:.0f}M parameters ({time.time() - t:.0f} s)", flush=True)+ if args.save:+ import json+ from pathlib import Path++ model.save_pretrained(args.save)+ meta = {"vocab": S.VOCAB, "pad": S.VOCAB, "families": {k: list(v) for k, v in+ S.FAMILIES.items()},+ "controls": S.CONTROLS, "circuit": c.stats(), "hidden": model.layout.hidden}+ (Path(args.save) / "chipformer.json").write_text(json.dumps(meta, indent=1))+ print(f"saved {args.save}", flush=True)+ model = model.to(args.device)+ for name in args.progs.split(","):+ prog = program(name) if "-" not in name else elf.program(DATA / f"{name}.elf")+ t = time.time()+ cells = residual_check(model, c, prog)+ want = ref.Machine.from_program(prog).run(10**6)+ firmware.run(model, c, ref.Machine.from_program(prog), max_new=4) # (autotune warm-up)+ got = firmware.run(model, c, ref.Machine.from_program(prog), max_new=len(want) + 5)+ ok = got == want[1:]+ out = firmware.output_text(got)+ print(f"{name:14s} residual == IR ({cells} cells); generate == reference: {ok} "+ f"({len(got)} tokens{', output ' + repr(out[:40]) if out else ''}; "+ f"{time.time() - t:.1f} s)", flush=True)+ assert ok+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/chipformer/rv/tritonexec.py b/experimental/riscv/chipformer/rv/tritonexec.pynew file mode 100644index 0000000..f3bdcda--- /dev/null+++ b/experimental/riscv/chipformer/rv/tritonexec.py@@ -0,0 +1,305 @@+"""Executor v2: the circuit's step as fused Triton kernels, one per sublayer, under one CUDA graph.++Same semantics as fastexec.Executor (the IR's arithmetic in fp32, every value exact), with the+weights kept as sparse rows (CSR, padded per sublayer to a power-of-2 width KP so a row is one+2D gather instead of a loop of dependent loads):++ MLP a program per block of units: a and b are sparse dot products with the residual, then+ coef * a * relu(b) goes (atomically) into the output buffer+ heads a program per value head: q / k / v / z are sparse dot products over the residual and+ its lag history (the conv taps folded into the weights); then decay (only the rows the+ head's keys reach), the one-row write (key = argmax), the read of the query's rows only+ (at most QHOT of them, found by argmax), the ballast norm, the gate and the output+ columns into the output buffer+ apply residual += output buffer; after a head sublayer its lag history moves on++ Executor2(c, batch).step(x) -> x after the whole circuit (in place)+"""++import numpy as np+import torch+import triton+import triton.language as tl++from chipformer.dsl.graph import ONE+from chipformer.q35.ir import FFN, HD, Head+++def _csr(rows):+ """CSR arrays (row_ptr, idx, w) of a list of {col: weight}."""+ rp, ix, w = [0], [], []+ for r in rows:+ for k, v in r.items():+ if v:+ ix.append(k)+ w.append(v)+ rp.append(len(ix))+ if not ix:+ ix, w = [0], [0.0]+ return np.array(rp, np.int32), np.array(ix, np.int32), np.array(w, np.float32)+++def _kp(rp) -> int:+ """The padded row width: the next power of 2 of the longest row (at least 2)."""+ return max(2, 1 << int(max(np.diff(rp).max(initial=0), 1) - 1).bit_length())+++def _ffn_kernel(x_ptr, y_ptr, D, U, rpa, ixa, wa, rpb, ixb, wb, coef, outd,+ BLOCK: tl.constexpr, KA: tl.constexpr, KB: tl.constexpr):+ pid = tl.program_id(0)+ bat = tl.program_id(1)+ u = pid * BLOCK + tl.arange(0, BLOCK)+ m = u < U+ xb = x_ptr + bat * D+ a0 = tl.load(rpa + u, mask=m, other=0)+ na = tl.load(rpa + u + 1, mask=m, other=0) - a0+ ka = tl.arange(0, KA)+ ma = m[:, None] & (ka[None, :] < na[:, None])+ ia = tl.load(ixa + a0[:, None] + ka[None, :], mask=ma, other=0)+ av = tl.sum(tl.load(wa + a0[:, None] + ka[None, :], mask=ma, other=0.0)+ * tl.load(xb + ia, mask=ma, other=0.0), axis=1)+ b0 = tl.load(rpb + u, mask=m, other=0)+ nb = tl.load(rpb + u + 1, mask=m, other=0) - b0+ kb = tl.arange(0, KB)+ mb = m[:, None] & (kb[None, :] < nb[:, None])+ ib = tl.load(ixb + b0[:, None] + kb[None, :], mask=mb, other=0)+ bv = tl.sum(tl.load(wb + b0[:, None] + kb[None, :], mask=mb, other=0.0)+ * tl.load(xb + ib, mask=mb, other=0.0), axis=1)+ val = tl.load(coef + u, mask=m, other=0.0) * av * tl.maximum(bv, 0.0)+ od = tl.load(outd + u, mask=m, other=0)+ tl.atomic_add(y_ptr + bat * D + od, val, mask=m)+++def _lag_load(xb, hb, i, m, D):+ """The lagged residual: index lag * D + d (lag 0: x, lags 1..3: the history)."""+ lo = i < D+ return tl.where(lo, tl.load(xb + i, mask=m & lo, other=0.0),+ tl.load(hb + i - D, mask=m & (i >= D), other=0.0))+++def _rows(xb, hb, D, rp, ix, w, base, ch, KP: tl.constexpr):+ """relu of 128 sparse rows (row base + ch) over the lagged residual."""+ r0 = tl.load(rp + base + ch)+ n = tl.load(rp + base + ch + 1) - r0+ kk = tl.arange(0, KP)+ mk = kk[None, :] < n[:, None]+ i = tl.load(ix + r0[:, None] + kk[None, :], mask=mk, other=0)+ acc = tl.sum(tl.load(w + r0[:, None] + kk[None, :], mask=mk, other=0.0)+ * _lag_load(xb, hb, i, mk, D), axis=1)+ return tl.maximum(acc, 0.0)+++def _scalar(xb, hb, D, rp, ix, w, vh):+ lane = tl.arange(0, 32)+ s0 = tl.load(rp + vh)+ n = tl.load(rp + vh + 1) - s0+ mk = lane < n+ i = tl.load(ix + s0 + lane, mask=mk, other=0)+ return tl.sum(tl.load(w + s0 + lane, mask=mk, other=0.0) * _lag_load(xb, hb, i, mk, D), axis=0)+++def _head_kernel(x_ptr, h_ptr, S_ptr, y_ptr, D, V,+ qrp, qix, qw, krp, kix, kw, vrp, vix, vw, zrp, zix, zw,+ arp, aix, aw, brp, bix, bw, bconst, odim, ow, khead, nrows,+ KQ: tl.constexpr, KK: tl.constexpr, KV: tl.constexpr, KZ: tl.constexpr,+ QHOT: tl.constexpr):+ vh = tl.program_id(0)+ bat = tl.program_id(1)+ ch = tl.arange(0, 128)+ xb = x_ptr + bat * D+ hb = h_ptr + bat * 3 * D+ h = tl.load(khead + vh)+ k = _rows(xb, hb, D, krp, kix, kw, h * 128, ch, KK)+ v = _rows(xb, hb, D, vrp, vix, vw, vh * 128, ch, KV)+ z = _rows(xb, hb, D, zrp, zix, zw, vh * 128, ch, KZ)+ q = _rows(xb, hb, D, qrp, qix, qw, h * 128, ch, KQ)+ bv = _scalar(xb, hb, D, brp, bix, bw, vh) + tl.load(bconst + vh)+ Sb = S_ptr + (bat * V + vh) * 16384+ nr = tl.load(nrows + vh) # > 0: the head decays, in its first nr rows+ if nr > 0:+ av = _scalar(xb, hb, D, arp, aix, aw, vh)+ fac = tl.where(av == 0.0, 1.0, tl.where(av == 110.0, 0.5, 0.0))+ if fac != 1.0:+ for r in range(nr):+ tl.store(Sb + r * 128 + ch, tl.load(Sb + r * 128 + ch) * fac)+ kmax = tl.max(k, axis=0)+ krow = tl.argmax(k, axis=0)+ if (kmax > 0.0) & (bv == 1.0):+ tl.store(Sb + krow * 128 + ch, v)+ tl.debug_barrier()+ # the read: the sum of the rows whose query is set (multi-hot queries have equal values)+ o = tl.zeros([128], dtype=tl.float32)+ for _ in range(QHOT):+ qm = tl.max(q, axis=0)+ qi = tl.argmax(q, axis=0)+ if qm > 0.0:+ o += tl.load(Sb + qi * 128 + ch)+ q = tl.where(ch == qi, 0.0, q)+ ref = tl.sum(tl.where(ch < 2, o * o, 0.0), axis=0)+ norm = tl.math.sqrt_rn(ref / 128.0)+ out = tl.where(ref > 0.0, tl.math.div_rn(o, tl.where(ref > 0.0, norm, 1.0)), 0.0)+ wv = tl.load(ow + vh * 128 + ch)+ dd = tl.load(odim + vh * 128 + ch)+ tl.atomic_add(y_ptr + bat * D + dd, out * z * wv, mask=wv != 0.0)+++def _apply_kernel(x_ptr, y_ptr, h_ptr, D, HIST: tl.constexpr, BLOCK: tl.constexpr):+ """x += y, y = 0; with HIST, the lag history (h: [3, D] per row, h[0] = lag 1) moves on+ first, the input x becoming lag 1."""+ pid = tl.program_id(0)+ bat = tl.program_id(1)+ i = pid * BLOCK + tl.arange(0, BLOCK)+ m = i < D+ xo = x_ptr + bat * D + i+ x = tl.load(xo, mask=m, other=0.0)+ if HIST:+ hb = h_ptr + bat * 3 * D + i+ h1 = tl.load(hb + D, mask=m, other=0.0)+ h0 = tl.load(hb, mask=m, other=0.0)+ tl.store(hb + 2 * D, h1, mask=m)+ tl.store(hb + D, h0, mask=m)+ tl.store(hb, x, mask=m)+ yo = y_ptr + bat * D + i+ tl.store(xo, x + tl.load(yo, mask=m, other=0.0), mask=m)+ tl.store(yo, tl.zeros([BLOCK], dtype=tl.float32), mask=m)+++class Executor2:+ BLOCK = 64++ def __init__(self, c, batch: int = 1, device="cuda"):+ self.c, self.B, self.dev = c, batch, device+ D = len(c.dims)+ self.D = D+ t = lambda a, dt=torch.int32: torch.tensor(np.asarray(a), dtype=dt, device=device)++ def dev3(arr):+ return [t(arr[0]), t(arr[1]), t(arr[2], torch.float32)]++ self.steps = []+ for lv in sorted({n.level for n in c.nodes}):+ nodes = [n for n in c.nodes if n.level == lv]+ if isinstance(nodes[0], FFN):+ units = [(float(cf), a, b, n.out) for n in nodes for cf, a, b in n.terms]+ ra = _csr([{d: float(v) for d, v in a.terms.items()} for _, a, _, _ in units])+ rb = _csr([{d: float(v) for d, v in b.terms.items()} for _, _, b, _ in units])+ self.steps.append({"kind": "ffn", "U": len(units), "a": dev3(ra), "b": dev3(rb),+ "KA": _kp(ra[0]), "KB": _kp(rb[0]),+ "coef": t([u[0] for u in units], torch.float32),+ "out": t([u[3] for u in units])})+ continue+ heads = [n for n in nodes if isinstance(n, Head)]+ vh = [(h, vd) for h, n in enumerate(heads) for vd in n.vheads()]+ V = len(vh)++ def lagged(ch, D=D):+ return {lag * D + d: float(cf) * tw for lag, tw in enumerate(ch.taps) if tw+ for d, cf in ch.expr.terms.items()}++ qrows, krows = [], []+ for n in heads:+ for s in range(HD):+ qrows.append(lagged(n.q[s]) if s in n.q else {})+ krows.append(lagged(n.k[s]) if s in n.k else {})+ vrows, zrows, arows, brows, bconst, odim, ow, khead, nrows = ([] for _ in range(9))+ for h, vd in vh:+ for col in range(HD):+ vrows.append(lagged(vd.v[col]) if col in vd.v else {})+ zrows.append({d: float(cf) for d, cf in vd.z[col].terms.items()}+ if col in vd.z else {})+ arows.append({d: float(cf) for d, cf in vd.a.terms.items()})+ if vd.beta is None:+ brows.append({})+ bconst.append(1.0)+ else:+ brows.append({d: float(cf) for d, cf in vd.beta.terms.items()})+ bconst.append(0.0)+ od, w_ = np.zeros(HD, np.int32), np.zeros(HD, np.float32)+ for col, d, w in vd.out:+ od[col], w_[col] = d, w+ odim.append(od)+ ow.append(w_)+ khead.append(h)+ # a decaying head: only the rows its keys (or its initial state) reach+ reach = set(heads[h].k) | {r for x in (heads[h].init or []) for r in x}+ nrows.append(max(reach) + 1 if vd.a.terms and reach else 0)+ assert max(len(r) for r in arows + brows) <= 32+ q, k, v, z = (_csr(r) for r in (qrows, krows, vrows, zrows))+ st = {"kind": "head", "V": V, "heads": heads, "q": dev3(q), "k": dev3(k),+ "v": dev3(v), "z": dev3(z), "a": dev3(_csr(arows)), "b": dev3(_csr(brows)),+ "KQ": _kp(q[0]), "KK": _kp(k[0]), "KV": _kp(v[0]), "KZ": _kp(z[0]),+ "QHOT": max(getattr(n, "qhot", 2) for n in heads),+ "bconst": t(bconst, torch.float32), "odim": t(np.concatenate(odim)),+ "ow": t(np.concatenate(ow), torch.float32), "khead": t(khead),+ "nrows": t(nrows), "S": torch.zeros(batch, V, HD, HD, device=device),+ "hist": torch.zeros(batch, 3, D, device=device)} # lags 1..3+ self.steps.append(st)+ self.y = torch.zeros(batch, D, device=device)+ self.feat_ids = sorted(c.token_fns)++ def load_init(self, row: int = 0):+ for st in self.steps:+ if st["kind"] != "head":+ continue+ buf = np.zeros(st["S"].shape[1:], np.float32)+ j = 0+ for n in st["heads"]:+ init = n.init or []+ for x in range(len(n.vheads())):+ for r, vec in (init[x] if x < len(init) else {}).items():+ buf[j, r] = vec+ j += 1+ st["S"][row].copy_(torch.from_numpy(buf))++ def reset(self, rows=None):+ for st in self.steps:+ if st["kind"] == "head":+ (st["hist"].zero_() if rows is None else st["hist"][rows].zero_())++ def embed(self, toks):+ out = np.zeros((len(toks), self.D))+ out[:, ONE] = 1.0+ for d in self.feat_ids:+ out[:, d] = [float(self.c.feature_value(d, t)) for t in toks]+ return torch.tensor(out, dtype=torch.float32, device=self.dev)++ def step(self, x):+ """x [B, D] (in place: the embedding in, the residual after every node out)."""+ B, D = x.shape[0], self.D+ grid_apply = (triton.cdiv(D, 1024), B)+ for st in self.steps:+ if st["kind"] == "ffn":+ _ffn_kernel[(triton.cdiv(st["U"], self.BLOCK), B)](+ x, self.y, D, st["U"], *st["a"], *st["b"], st["coef"], st["out"],+ BLOCK=self.BLOCK, KA=st["KA"], KB=st["KB"])+ _apply_kernel[grid_apply](x, self.y, x, D, HIST=False, BLOCK=1024)+ continue+ _head_kernel[(st["V"], B)](+ x, st["hist"], st["S"], self.y, D, st["V"], *st["q"], *st["k"], *st["v"],+ *st["z"], *st["a"], *st["b"], st["bconst"], st["odim"], st["ow"], st["khead"],+ st["nrows"], KQ=st["KQ"], KK=st["KK"], KV=st["KV"], KZ=st["KZ"],+ QHOT=st["QHOT"])+ _apply_kernel[grid_apply](x, self.y, st["hist"], D, HIST=True, BLOCK=1024)+ return x+++def check(c, toks, device="cuda") -> int:+ """Executor2 == ir.evaluate on every dim at every position."""+ from chipformer.q35 import ir++ ex = Executor2(c, 1, device)+ ex.load_init()+ emb = ex.embed(toks)+ want = ir.evaluate(c, toks, check=False)+ for t in range(len(toks)):+ x = emb[t:t + 1].clone()+ got = ex.step(x)[0].double().cpu().numpy()+ bad = np.nonzero(got != want[t])[0]+ assert not len(bad), (t, [(c.dims[d].name, got[d], want[t, d]) for d in bad[:5]])+ return len(toks)diff --git a/experimental/riscv/chipformer/session.py b/experimental/riscv/chipformer/session.pynew file mode 100644index 0000000..b1c9d95--- /dev/null+++ b/experimental/riscv/chipformer/session.py@@ -0,0 +1,72 @@+"""Host loop: drive any next-token model as a CHIP-8 machine.++The host only ever (1) feeds the boot prompt, (2) injects TICK tokens and key+writes right after a STEP, and (3) keeps a mirror of the cells built from the+model's own writes, for the screen. It never computes machine semantics.+"""++from collections.abc import Callable++from chipformer import spec+from chipformer.ref.emu import Chip8, apply_writes, display_pixels+from chipformer.trace import boot_tokens, encode_writes++MAX_RECORD = 2 * 40 + 2 # longer than any valid record+++class Session:+ def __init__(self, feed: Callable[[int], int], rom: bytes, quirks=None):+ """``feed(token)`` processes one position and returns the predicted next token."""+ self.feed = feed+ self.cells = [0] * spec.NUM_CELLS+ self.tokens: list[int] = []+ self.pred = -1+ boot = Chip8(rom, quirks)+ apply_writes(self.cells, boot.state_writes())+ for t in boot_tokens(boot.cells):+ self._push(t)++ def _push(self, tok: int) -> None:+ self.tokens.append(tok)+ self.pred = self.feed(tok)++ def _inject(self, toks: list[int]) -> None:+ for t in toks:+ self._push(t)++ def _generate_record(self) -> list[tuple[int, int]]:+ """Let the model write until it emits STEP; apply its writes to the mirror."""+ writes = []+ for _ in range(MAX_RECORD):+ tok = self.pred+ if tok == spec.STEP:+ self._push(tok)+ apply_writes(self.cells, writes)+ return writes+ if not spec.is_addr(tok):+ raise RuntimeError(f"model emitted {spec.token_name(tok)} where an ADDR or STEP "+ f"was expected (position {len(self.tokens)})")+ self._push(tok)+ val = self.pred+ if not spec.is_val(val):+ raise RuntimeError(f"model emitted {spec.token_name(val)} after an ADDR")+ self._push(val)+ writes.append((tok - spec.ADDR_BASE, val - spec.VAL_BASE))+ raise RuntimeError("record never ended")++ def step(self) -> list[tuple[int, int]]:+ return self._generate_record()++ def tick(self) -> list[tuple[int, int]]:+ self._inject([spec.TICK])+ return self._generate_record()++ def set_key(self, key: int, down: bool) -> None:+ if bool(self.cells[spec.KEYS + key]) == down:+ return+ w = [(spec.KEYS + key, int(down))]+ self._inject(encode_writes(w) + [spec.STEP])+ apply_writes(self.cells, w)++ def pixels(self) -> list[list[int]]:+ return display_pixels(self.cells)diff --git a/experimental/riscv/chipformer/spec.py b/experimental/riscv/chipformer/spec.pynew file mode 100644index 0000000..76c03af--- /dev/null+++ b/experimental/riscv/chipformer/spec.py@@ -0,0 +1,121 @@+"""Machine layout and token vocabulary shared by every component.++All machine state lives in one flat array of cells. A cell holds an integer in+0..MAX_VALUE. The trace the transformer reads and writes is a sequence of+write events ``[ADDR a][VAL v]`` grouped into records terminated by ``STEP``.+See docs/trace_format.md for the full specification.+"""++# --- cell address map -------------------------------------------------------+RAM = 0x0000+RAM_SIZE = 0x1000+V = 0x1000 # V0..VF+I = 0x1010 # 12-bit index register+PC = 0x1011 # 12-bit program counter+SP = 0x1012 # stack pointer, 0..15+DT = 0x1013 # delay timer+ST = 0x1014 # sound timer+RNG = 0x1015 # 12-bit LFSR state+CLS = 0x1016 # writing this cell clears the display+QUIRKS = 0x1017 # 6 quirk flags (0/1), meaning as Octo's flags of the same name; see docs/quirks.md+QUIRK_NAMES = ("shift", "loadstore", "clip", "logic", "jump", "vforder")+Q_SHIFT, Q_LOADSTORE, Q_CLIP, Q_LOGIC, Q_JUMP, Q_VFORDER = (QUIRKS + i for i in range(6))+STACK = 0x1020 # 16 return addresses+KEYS = 0x1030 # 16 key states, 0 or 1+DISPLAY = 0x1100 # 32 rows x 8 bytes, MSB is the leftmost pixel+NULL = 0x1200 # sink for writes that must exist but have no effect+NUM_CELLS = 0x1201++DISPLAY_W = 64+DISPLAY_H = 32+DISPLAY_ROW_BYTES = DISPLAY_W // 8++# --- values -----------------------------------------------------------------+NUM_VALUES = 0x1000+MAX_VALUE = NUM_VALUES - 1++# --- token vocabulary -------------------------------------------------------+VAL_BASE = 0 # VAL v -> v+ADDR_BASE = NUM_VALUES # ADDR a -> ADDR_BASE + a+STEP = ADDR_BASE + NUM_CELLS+TICK = STEP + 1+BOOT = STEP + 2+VOCAB_SIZE = BOOT + 1++# --- machine constants ------------------------------------------------------+PROGRAM_START = 0x200+FONT_BASE = 0x050+RNG_SEED = 0xACE+RNG_MASK = 0x829 # Galois taps for x^12 + x^6 + x^4 + x + 1 (period 4095)++# Named quirk profiles: the set of enabled flags. All flags off = Octo's defaults.+PROFILES = {+ "octo": (),+ "chip48": ("shift", "loadstore", "clip"), # HP48 CHIP-48 / SCHIP-era games, e.g. Pong+ "cosmac": ("logic", "clip"), # original COSMAC VIP interpreter+}+++def quirk_cells(quirks: "str | tuple[str, ...] | list[str] | None") -> dict[int, int]:+ """Cell values for a profile name or a list of enabled flag names."""+ if quirks is None:+ quirks = ()+ elif isinstance(quirks, str):+ quirks = PROFILES[quirks] if quirks in PROFILES else tuple(q for q in quirks.split(",") if q)+ unknown = set(quirks) - set(QUIRK_NAMES)+ if unknown:+ raise ValueError(f"unknown quirks {sorted(unknown)}; known: {QUIRK_NAMES}")+ return {QUIRKS + i: int(name in quirks) for i, name in enumerate(QUIRK_NAMES)}++FONT = bytes([+ 0xF0, 0x90, 0x90, 0x90, 0xF0, # 0+ 0x20, 0x60, 0x20, 0x20, 0x70, # 1+ 0xF0, 0x10, 0xF0, 0x80, 0xF0, # 2+ 0xF0, 0x10, 0xF0, 0x10, 0xF0, # 3+ 0x90, 0x90, 0xF0, 0x10, 0x10, # 4+ 0xF0, 0x80, 0xF0, 0x10, 0xF0, # 5+ 0xF0, 0x80, 0xF0, 0x90, 0xF0, # 6+ 0xF0, 0x10, 0x20, 0x40, 0x40, # 7+ 0xF0, 0x90, 0xF0, 0x90, 0xF0, # 8+ 0xF0, 0x90, 0xF0, 0x10, 0xF0, # 9+ 0xF0, 0x90, 0xF0, 0x90, 0x90, # A+ 0xE0, 0x90, 0xE0, 0x90, 0xE0, # B+ 0xF0, 0x80, 0x80, 0x80, 0xF0, # C+ 0xE0, 0x90, 0x90, 0x90, 0xE0, # D+ 0xF0, 0x80, 0xF0, 0x80, 0xF0, # E+ 0xF0, 0x80, 0xF0, 0x80, 0x80, # F+])+++def lfsr_step(s: int) -> int:+ lsb = s & 1+ s >>= 1+ if lsb:+ s ^= RNG_MASK+ return s+++def val_token(v: int) -> int:+ assert 0 <= v <= MAX_VALUE, v+ return VAL_BASE + v+++def addr_token(a: int) -> int:+ assert 0 <= a < NUM_CELLS, a+ return ADDR_BASE + a+++def is_val(tok: int) -> bool:+ return VAL_BASE <= tok < ADDR_BASE+++def is_addr(tok: int) -> bool:+ return ADDR_BASE <= tok < STEP+++def token_name(tok: int) -> str:+ if is_val(tok):+ return f"={tok - VAL_BASE:#x}"+ if is_addr(tok):+ return f"@{tok - ADDR_BASE:#06x}"+ return {STEP: "STEP", TICK: "TICK", BOOT: "BOOT"}.get(tok, f"?{tok}")diff --git a/experimental/riscv/chipformer/trace.py b/experimental/riscv/chipformer/trace.pynew file mode 100644index 0000000..4026634--- /dev/null+++ b/experimental/riscv/chipformer/trace.py@@ -0,0 +1,118 @@+"""Canonical token traces: encoding, parsing and replay.++A trace is::++ BOOT <state dump writes> STEP+ <record> STEP <record> STEP ...++where each record is one of+ instruction: <writes> (generated by the model)+ timer tick: TICK <DT write> <ST write> (TICK injected by the host)+ input: <KEY cell write> (injected whole by the host)++and every write is the token pair ``ADDR a, VAL v``.+"""++from dataclasses import dataclass, field++from chipformer import spec+from chipformer.ref.emu import Chip8, Writes, apply_writes+from chipformer.spec import BOOT, STEP, TICK+++def encode_writes(writes: Writes) -> list[int]:+ out = []+ for a, v in writes:+ out += [spec.addr_token(a), spec.val_token(v)]+ return out+++def boot_tokens(cells: list[int]) -> list[int]:+ from chipformer.ref.emu import state_writes++ return [BOOT] + encode_writes(state_writes(cells)) + [STEP]+++@dataclass+class Record:+ kind: str # "boot" | "instr" | "tick"+ writes: Writes = field(default_factory=list)+++def parse(tokens: list[int], allow_reboot: bool = False) -> list[Record]:+ """Split a trace into records, validating its grammar.++ ``allow_reboot`` accepts later BOOT records (context rollover in a dumped session).+ """+ records: list[Record] = []+ cur: Record | None = None+ i = 0+ while i < len(tokens):+ t = tokens[i]+ if t == STEP:+ if cur is None:+ raise ValueError(f"empty record before STEP at {i}")+ records.append(cur)+ cur = None+ i += 1+ elif t in (BOOT, TICK):+ if cur is not None:+ raise ValueError(f"{spec.token_name(t)} inside a record at {i}")+ if t == BOOT and records and not allow_reboot:+ raise ValueError(f"BOOT after start at {i}")+ cur = Record("boot" if t == BOOT else "tick")+ i += 1+ elif spec.is_addr(t):+ if i + 1 >= len(tokens) or not spec.is_val(tokens[i + 1]):+ raise ValueError(f"ADDR not followed by VAL at {i}")+ if cur is None:+ cur = Record("instr")+ cur.writes.append((t - spec.ADDR_BASE, tokens[i + 1] - spec.VAL_BASE))+ i += 2+ else:+ raise ValueError(f"unexpected token {spec.token_name(t)} at {i}")+ if cur is not None:+ raise ValueError("trace ends inside a record")+ if not records or records[0].kind != "boot":+ raise ValueError("trace must start with BOOT")+ return records+++def replay(tokens: list[int]) -> list[int]:+ """Rebuild the machine state from a trace alone."""+ cells = [0] * spec.NUM_CELLS+ for r in parse(tokens):+ apply_writes(cells, r.writes)+ return cells+++class Machine:+ """Drives the reference emulator and records the canonical trace.++ This is the ground truth the transformer must reproduce token for token.+ """++ def __init__(self, rom: bytes, quirks: "str | tuple[str, ...] | None" = None):+ self.emu = Chip8(rom, quirks)+ self.tokens = boot_tokens(self.emu.cells)+ self.generated = [False] * len(self.tokens) # tokens the model must predict++ def _emit(self, tokens: list[int], generated: bool) -> None:+ self.tokens += tokens+ self.generated += [generated] * len(tokens)++ def step(self) -> Writes:+ w = self.emu.step()+ self._emit(encode_writes(w) + [STEP], True)+ return w++ def tick(self) -> Writes:+ w = self.emu.tick()+ self._emit([TICK], False)+ self._emit(encode_writes(w) + [STEP], True)+ return w++ def set_key(self, key: int, down: bool) -> None:+ if bool(self.emu.cells[spec.KEYS + key]) == down:+ return+ self._emit(encode_writes(self.emu.set_key(key, down)) + [STEP], False)diff --git a/experimental/riscv/chipformer/web.py b/experimental/riscv/chipformer/web.pynew file mode 100644index 0000000..6d0a81b--- /dev/null+++ b/experimental/riscv/chipformer/web.py@@ -0,0 +1,197 @@+"""Publish the demo assets into the website, or assemble the Hugging Face Space.++ uv run python -m chipformer.web build --blog D:/Blog+ uv run python -m chipformer.web space --out dist/space [--build-wasm]++`build` builds the WebAssembly engine, then writes:+ <blog>/public/chipformer-assets/ engine.wasm, model.chipformer, roms/, thumbs/, ATTRIBUTION.md+ <blog>/components/chipformer/data.ts generated game list, machine constants, model facts+`space` writes a self-contained static site: the sources in space/ plus the same assets and a+generated data.js. It reuses the built engine.wasm unless --build-wasm is given.+The weights repo stays the source of truth; the blog and the Space only receive build outputs.+"""++import argparse+import hashlib+import json+import os+import shutil+import subprocess+from pathlib import Path++import numpy as np++from chipformer.games import GAMES, load_manifest, write_attribution++ROOT = Path(__file__).resolve().parent.parent+WASM = ROOT / "engine" / "target" / "wasm32-unknown-unknown" / "release" / "chipformer_engine.wasm"+PHOSPHOR, SCREEN_BG = (221, 245, 158), (11, 16, 13)+THUMB_INSTR = {"blinky": 20000, "super-pong": 800, "tetris": 120000} # a more telling frame+++def build_wasm() -> None:+ subprocess.run(["cargo", "build", "--release", "--lib", "--target", "wasm32-unknown-unknown",+ "--no-default-features"], cwd=ROOT / "engine", check=True)+++def thumbnail(game: dict, out: Path, instr: int = 2500) -> None:+ """Render a frame of the game from a verified engine run, in the site's phosphor palette."""+ from PIL import Image++ from chipformer import spec+ from chipformer.check import verify_trace+ from chipformer.ref.emu import apply_writes, display_pixels+ from chipformer.trace import parse++ exe = ROOT / "engine" / "target" / "release" / ("engine.exe" if os.name == "nt" else "engine")+ dump = out.with_suffix(".u32")+ subprocess.run([str(exe), "run", str(ROOT / "model"), str(GAMES / game["file"]),+ "--instr", str(instr), "--ipf", str(game["ipf"]), "--quirks", ",".join(game["quirks"]),+ "--seed", "11", "--dump", str(dump)], check=True, capture_output=True)+ tokens = np.fromfile(dump, dtype="<u4").tolist()+ dump.unlink()+ _, err = verify_trace(tokens)+ if err:+ raise RuntimeError(f"{game['id']}: trace does not verify: {err}")+ cells = [0] * spec.NUM_CELLS+ for rec in parse(tokens, allow_reboot=True):+ apply_writes(cells, rec.writes)+ img = Image.new("RGB", (spec.DISPLAY_W, spec.DISPLAY_H))+ img.putdata([PHOSPHOR if p else SCREEN_BG for row in display_pixels(cells) for p in row])+ img.resize((spec.DISPLAY_W * 8, spec.DISPLAY_H * 8), Image.NEAREST).save(out)+++def publish_assets(assets: Path) -> dict:+ """Copy the engine, model, featured ROMs, thumbnails and credits into `assets`; return the+ generated site data (games, layers, model facts, machine constants)."""+ from chipformer.compile import load_sparse++ for sub in ("roms", "thumbs"):+ (assets / sub).mkdir(parents=True, exist_ok=True)+ shutil.copyfile(WASM, assets / "engine.wasm")+ shutil.copyfile(ROOT / "model" / "model.chipformer", assets / "model.chipformer")+ _, config = load_sparse(ROOT / "model" / "model.chipformer")++ manifest = load_manifest()+ featured = sorted((g for g in manifest if "featured" in g), key=lambda g: g["featured"])+ games = []+ for g in featured:+ shutil.copyfile(GAMES / g["file"], assets / "roms" / f"{g['id']}.ch8")+ thumbnail(g, assets / "thumbs" / f"{g['id']}.png", THUMB_INSTR.get(g["id"], 2500))+ games.append({k: g.get(k) for k in ("id", "title", "authors", "year", "description", "license",+ "source", "quirks", "ipf", "keys", "controls", "tap")} | {"gameOver": g.get("game_over")})+ write_attribution(featured, assets / "ATTRIBUTION.md")++ digest = hashlib.sha1((assets / "engine.wasm").read_bytes()+ + (assets / "model.chipformer").read_bytes()).hexdigest()[:10]+ m = config["machine"]+ facts = {+ "layers": config["n_layers"], "dModel": config["d_model"],+ "heads": sum(lay["heads"] for lay in config["layers"]),+ "ffnUnits": sum(lay["ffn_units"] for lay in config["layers"]),+ "params": config["params"], "nonzero": config["nonzero_params"],+ "vocab": config["vocab_size"], "modelBytes": (assets / "model.chipformer").stat().st_size,+ "wasmBytes": (assets / "engine.wasm").stat().st_size,+ }+ return {+ "digest": digest,+ "machine": {k: m[k] for k in ("VAL_BASE", "ADDR_BASE", "STEP", "TICK", "BOOT", "NULL", "CLS", "PC",+ "RNG", "KEYS", "DISPLAY", "QUIRKS", "QUIRK_NAMES")},+ "facts": facts,+ "library": {"total": len(manifest), "verified": sum(g.get("status") == "ok" for g in manifest)},+ "layers": [{"heads": lay["head_names"], "units": lay["ffn_units"]} for lay in config["layers"]],+ "games": games,+ }+++def build(blog: Path) -> None:+ build_wasm()+ assets = blog / "public" / "chipformer-assets"+ d = publish_assets(assets)+ digest, machine, facts, library, layers, games = (+ d[k] for k in ("digest", "machine", "facts", "library", "layers", "games"))+ ts = "\n".join([+ "// Generated by chipformer/web.py in the chipformer repo. Do not edit by hand.",+ "",+ 'export const ASSET_BASE = "/chipformer-assets";',+ f'export const ASSET_VERSION = "{digest}";',+ "",+ f"export const MACHINE = {json.dumps(machine, indent=2)} as const;",+ "",+ f"export const MODEL_FACTS = {json.dumps(facts, indent=2)} as const;",+ "",+ f"export const LIBRARY = {json.dumps(library)} as const;",+ "",+ "/** Per layer: the names of its attention heads (in model order) and its FFN unit count. */",+ f"export const LAYERS: {{ heads: string[]; units: number }}[] = {json.dumps(layers)};",+ "",+ "export type KeyRole = \"up\" | \"down\" | \"left\" | \"right\" | \"a\" | \"b\" | \"player2Up\" | \"player2Down\";",+ "",+ "export type Game = {",+ " id: string;",+ " title: string;",+ " authors: string[];",+ " year: string | null;",+ " description: string;",+ " license: string;",+ " source: string;",+ " quirks: string[];",+ " ipf: number;",+ " keys: Partial<Record<KeyRole, number>>;",+ " controls: string;",+ " tap: KeyRole[];",+ " gameOver: { rows: number[]; cols: number[]; seconds: number; message: string } | null;",+ "};",+ "",+ f"export const GAMES: Game[] = {json.dumps(games, indent=2, ensure_ascii=False)};",+ "",+ ])+ (blog / "components" / "chipformer").mkdir(parents=True, exist_ok=True)+ (blog / "components" / "chipformer" / "data.ts").write_text(ts, encoding="utf-8")+ print(f"published {len(games)} games, engine {facts['wasmBytes'] / 1e3:.0f} kB, "+ f"model {facts['modelBytes'] / 1e6:.2f} MB -> {assets}")+++def space(out: Path, rebuild_wasm: bool = False) -> None:+ """Assemble the static Hugging Face Space: space/ sources + assets + a generated data.js."""+ if rebuild_wasm or not WASM.exists():+ build_wasm()+ if out.exists():+ shutil.rmtree(out)+ shutil.copytree(ROOT / "space", out)+ d = publish_assets(out / "assets")+ js = "\n".join([+ "// Generated by chipformer/web.py (python -m chipformer.web space). Do not edit by hand.",+ "",+ 'export const ASSET_BASE = "./assets";',+ f'export const ASSET_VERSION = "{d["digest"]}";',+ f"export const MACHINE = {json.dumps(d['machine'])};",+ f"export const MODEL_FACTS = {json.dumps(d['facts'])};",+ f"export const LIBRARY = {json.dumps(d['library'])};",+ "/** Per layer: the names of its attention heads (in model order) and its FFN unit count. */",+ f"export const LAYERS = {json.dumps(d['layers'])};",+ f"export const GAMES = {json.dumps(d['games'], indent=1, ensure_ascii=False)};",+ "",+ ])+ (out / "data.js").write_text(js, encoding="utf-8")+ size = sum(f.stat().st_size for f in out.rglob("*") if f.is_file())+ print(f"space: {len(d['games'])} games, {size / 1e6:.2f} MB -> {out}")+++def main() -> None:+ ap = argparse.ArgumentParser()+ ap.add_argument("command", choices=["build", "space"])+ ap.add_argument("--blog", type=Path, help="build: the website repo to publish into")+ ap.add_argument("--out", type=Path, default=ROOT / "dist" / "space", help="space: output directory")+ ap.add_argument("--build-wasm", action="store_true", help="space: rebuild engine.wasm first")+ args = ap.parse_args()+ if args.command == "build":+ if not args.blog:+ ap.error("build needs --blog")+ build(args.blog)+ else:+ space(args.out, args.build_wasm)+++if __name__ == "__main__":+ main()diff --git a/experimental/riscv/docs/rv32-p3-status.md b/experimental/riscv/docs/rv32-p3-status.mdnew file mode 100644index 0000000..97fd546--- /dev/null+++ b/experimental/riscv/docs/rv32-p3-status.md@@ -0,0 +1,51 @@+# RV32 P3 status (deferred until P2 lands)++Branch `rv-p3`, from rv-p1 `de634e7`. Code: `chipformer/rv/{bridge,merge,chat,evalarith,replay,textcheck}.py`,+the bridge in `core.build(text=...)`, `env/progs/calc.c`, `tests/test_rv_bridge.py`. Design notes are in+`docs/rv32.md` (P3 section).++## Built and verified+- **The bridge in the circuit:** tool-call text capture into a text buffer read as memory at 0x8000; GO at+ `</tool_call>`; putchar echoed as Qwen tokens (bytes plus `<tool_response>`, `</tool_response>`,+ `<think>`, `</think>`); owner, tau and mu; the HALT echo. Still 11 chip layers, 73 heads.+- **IR tests (`tests/test_rv_bridge.py`, 6 tests):** pass. The circuit equals the reference on owner bits and+ every owned token, including bignum calls, prefilled history and two calls in one stream. The rv P1 suites+ (`test_rv_ref`, `test_rv_core`) still pass.+- **Merged Qwen3.5-4B (layout rv1):** a text-only `Qwen3_5ForCausalLM`, hidden 3328 (no overlay needed), 15 GB.+- **End to end on GPU:** Qwen calls `calc`, the machine inside the weights computes+ 123456789 × 987654321 = 121932631112635269, and Qwen answers it. The lockstep check against the reference+ machine was clean, and the stock head's argmax agreed with the circuit at every owned position.+- **Arith n=40** (evals' seeded set, exact scoring), on an L4 box AWS reclaimed afterwards, so its result+ files are gone:++ | Run | Correct | Notes |+ |---|---|---|+ | rv1 | **38/40** | 54 calc calls, all exact; 0 lockstep errors; 0 head mismatches; median about 3.5k chip tokens per call, about 70 s per problem on an L4 |+ | base Qwen + host-executed calc tool (same response format) | 39/40 | |++ Both rv1 misses are Qwen's own: one re-calls in a loop, the other re-subtracts a result the computer+ already returned. The base-without-tool run was not finished.+- **Bug found and fixed:** the select dims outlived the exit and derailed Qwen's norms after a run. The+ merged circuit now keeps only code, tau and mu after the exit.++## Not done / next+- **0.8B does not fit this core:** it has 1 value head per key head and 3584 MLP units, while the core+ needs 2 value heads per key head and up to 6198 units in one sublayer. It would need a core variant;+ I'd defer it to after P2.+- **Not run yet:**+ - the text-fidelity check (`rv/textcheck.py`, KL vs base against the sdpa/eager noise floor)+ - the offline IR replay of eval records (`rv/replay.py`)+ - the base-without-tool control+- Rebase onto rv-p2 (DIV, Zba, 256 KB ROM, custom bignum uops); then run DeepMind sets via the+ `evalarith`-style harness.+- **Speed:** about 50-60 chip tokens/s on the eager HF chip-only path. The compact executor inside+ generation isn't wired in.+- **Capture window:** CAP_MAX = 80 tokens. The thermometer weights are 8·2^m, which overflows bf16 at+ 96 after the emitter's 2^36 fold; a wider window needs a two-level counter.++## Box state+- **q35-batch:** nothing of mine is running; GPU idle. My files:+ - `~/wrv3` (code)+ - `~/weights/dist/rv-q4` (the merged checkpoint, 15 GB; the disk is at 97%, delete it if space is needed)+ - `~/rvres` (`chat.log`, plus a partial `arith.rv1.jsonl`)+- **q35-s1 / q35-s2:** terminated by AWS (spot).diff --git a/experimental/riscv/docs/rv32.md b/experimental/riscv/docs/rv32.mdnew file mode 100644index 0000000..8be24cb--- /dev/null+++ b/experimental/riscv/docs/rv32.md@@ -0,0 +1,523 @@+# RV32 inside Qwen3.5++`chipformer/rv/` compiles a RISC-V core into stock Qwen3.5 Gated DeltaNet + SwiGLU layers with+the same q35 compiler as the CHIP-8 machine (`chipformer/q35/ir.py`, `prims.py`, `emit.py`).+P2 (branch `rv-p2`) is first, then P1 (branch `rv-p1`), then the P0 feasibility study+(branch `rv-p0`).++## P2: RV32IM + Zba, a software stack (MicroPython, calc)++### What changed (ISA-visible parts first, commit `77fa2ae`)+- **Memory map v2:**+ - RAM: 32 KB at 0. It holds .data, TLS, .bss, the heap, and the stack below 0x8000.+ - DROM: 128 KB of read-only data at 0x20000 (.rodata; loads only).+ - ROM: 256 KB of code at 0x40000.+ - The linker scripts are `env/rom.ld` and `env/qemu.ld`.+- **ROM word:** one pre-decoded 4-byte word per instruction (`ref.rom_word`):+ - Layout: uop | S0, RD slot | S1-3, RS1 slot | S4-6, RS2 slot | S7-9.+ - Every format fits, with static facts folded in: branch targets are word offsets with a+ bad-target bit, JAL targets are absolute, AUIPC's upper bits are precomputed, and the CSR+ stubs are uops.+ - It decodes in one MLP stage. The two value heads of a ROM key head now hold different+ words, so 256 KB fits in the same 2 sublayers that held 128 KB.+ - Stores take their data register through a third regfile read port (the RD slot), so I-type+ and S-type share one immediate layout.+- **DIV / DIVU / REM / REMU** in hardware.+ - The record is `QS(k0)`, then `QD(k << 4 | q_k)` for k = k0..7 (the hex digits of+ |A| / |B|, most significant first), then the result's value record.+ - k0 = 7 - dA + dB (from the operands' nibble counts) skips digits that must be 0.+ - Random operands average 4.1 tokens per DIV.+ - DIV by 0 and the overflow case follow RISC-V.+ - C builds use `-march=rv32im_zba` with picolibc's rv32im libc.+- **Zba:** sh1add / sh2add / sh3add.+ - MicroPython measured -4% with Zba+Zbb, and Zba was almost all of it.+ - Zbb is not added.+- **The digit file:** 8 registers of 128 decimal digits, outside the address space and part of+ the snapshot. The custom-0 / custom-1 ops (`ref.py` has the exact semantics):+ - `BMAC`: `D[i+off] = (ovw ? 0 : D[i+off]) + S[i]*m + c`. This one op covers add, mul-row,+ copy, scale and zero.+ - `BMSB`: `D[i+off] -= S[i]*m + b`. This covers sub and the div-digit step.+ - `BLD`: loads decimal digits from a RAM byte buffer.+ - `BSCAN`: the length of the ASCII digit run at an address.+ - `BGET`: reads one digit into a register.+ - `BPRD`: prints one digit as a BYTE token. Printing uses a sled of BPRD instructions, one+ token per digit.+ - All operands come in one register: `d | s<<3 | ovw<<6 | n<<7 | off<<13 | m<<20`. The+ common words fit a one-token value.+ - A row op is a `BD(digit | carry << 4)` token per digit, then rd's value (the final carry).+- **Tokens:**+ - New families: QS (16), QD (128) and BD (256), plus the control SC.+ - The vocabulary is now 230,068 rows plus PAD.++### The circuit (commit `3c33d0f`)+- **Size:**+ - **11 layers**, the same as P1.+ - 2,679 dims and 99 heads (at most 16 per sublayer).+ - 42,012 MLP units, at most 8,149 per sublayer out of 9,216.+ - **1,040,595 nonzero weights**; the emitted standalone hidden size is 2816.+ - With unlimited capacity it is still 11 layers. The depth is the dependency chain (DIV's+ compare signs at L10, the exit at L11), so list scheduling (`ir.Graph.reschedule`) cannot+ remove a layer here.++| Layer | Attention (key heads) | MLP (units) |+|---|---|---|+| L1 | pc, count, j, v (4) | thermometers (306) |+| L2 | — | PC word, link, digits-so-far nd (1,376) |+| L3-L4 | ROM 16 + 16 (256 KB) | decode (4,138) |+| L5 | registers x3, quotient heads x4 (7) | adder s1, MUL / REM products, DIV signs, digit-file rows (7,876) |+| L6 | digit file: 2 read ports x 4 (8) | adder s2, shifter, logic, \|A\|>>4s, Q x \|B\| and m x \|B\| products (6,621) |+| L7-L8 | RAM 16 + 16 (32 KB) | multiplier s3-s4, DIV product splits and compare columns, loads, Zba, branch target (3,751 / 7,923) |+| L9 | DROM 16 (128 KB) | multiplier s5, DROM bytes to bits, DIV lookahead, digit / carry (8,149) |+| L10 | — | the 15 DIV compare signs, rd value select (1,074) |+| L11 | — | the output code (798) |++- **DIV**, at the position computing digit k (s = 7 - k):+ - q_k = #{m in 1..15 : D >= m|B|}, where D = (|A| >> 4s) - 16 Q |B| and Q is the digits+ so far.+ - Each of the 15 tests is the sign (bit 39) of D - m|B| mod 2^40. It is built from+ un-normalized nibble-product columns in 3 MLP stages: column split, lookahead, sign.+ - The column offsets are solved so that every column stays in [0, 255].+ - |B| = (B xor sb) + sb, with the +1 folded into nibble 0 (range 0..16).+ - |A| >> 4s is a mux with the ~A + 1 correction.+ - The quotient heads (L5: 4 key heads x 2 value heads, one per digit, one row each) store+ each digit at its QD token in one copy per alignment. One-hot reads gated by nd return+ the digits so far, right-aligned.+ - A first version read 8 rows of one head at once. rv-tiny's fla chunk kernel then left+ fp32 residues (2e-8) on c-crc32p: the multi-hot query's 1/sqrt(8) is not exact, and the+ rows are rewritten often. The IR's read emulation reads rows directly, so it cannot+ see this. Every read is one-hot or 2-hot (the ROM's) again.+ - REM is A -+ |Q| B mod 2^32 through the multiplier: Q replaces A as the product input, and+ the column forms change.+- **DROM:** byte tables read at L9, like the code ROM. Bits come out of the L9 MLP; the lane and+ sign select happen in the value select.+- **Digit file:** port 1 reads row j (the record index); port 2 reads row j + off or k. Writes go+ to both ports' copies at port 2's row from t-1.++### Gate results+- **Compliance:** riscv-tests **rv32ui (38), rv32um (all 8, including div / divu / rem / remu)+ and rv32uzba (3) are green.**+ - Teacher-forced in the IR, with the HF-kernel read emulation on traces under 1,500 tokens.+ - Also free-running on the GPU.+ - DIV fuzz (edge values, all operand sizes, signs, rd = x0) is exact in the IR. The+ reference's DIV decomposition is checked against the ISA on 80k random cases.+ - The local suite (test_rv_core, test_rv_ref, test_rv_fuzz) passes: 228 tests, 14 skipped.+- **Programs:** all 61 ELFs give the same exit code and stdout under the reference model as+ under qemu-riscv32. Free-running on the L4 (the executor generates every token, which must+ equal the reference trace):++| Program | Instructions | Tokens | tok/instr | P1 instructions |+|---|---|---|---|---|+| MicroPython s5 (18-line snippet) | 492,526 | 610,544 | 1.24 | — |+| MicroPython s1 (`print(2**100)`) | 39,888 | 50,150 | 1.26 | — |+| fib(25) | 2,114,629 | 2,174,558 | 1.03 | 2,117,133 |+| qsort 1000 | 199,157 | 240,176 | 1.21 | 256,975 |+| sieve below 10^4 | 123,535 | 176,225 | 1.43 | 126,897 |+| mini-gmp 20x20 | 14,767 | 22,671 | 1.54 | 15,967 |+| crc32 printf | 4,109 | 6,283 | 1.53 | 9,998 |+| crc32, sort, strops, sieve(2000) + 7 riscv-tests (div, rem, Zba) | 44,722 | | | |++ - The two free-running batches total **3.03M instructions**.+ - Fuzz (now with DIV, Zba and DROM loads): 64 programs, 152,848 instructions.+ - A bigger fuzz run: 128 programs at 300 loop iterations, **2,986,619 instructions,+ exact**.+ - **Altogether P2 free-ran 15.3M instructions exactly**: the programs above, the fuzz+ runs, the 10k calc cases and the v2 runs.+- **MicroPython** (P2.4a, `env/build_mpy.sh`, `env/mpy/`) is rv_tool's best build under 256 KB,+ for this machine:+ - The 3 source patches, 5 hot files at -O3, fastlib, computed goto, the map cache, no GC+ clear, parse chunk 512.+ - rv32im_zba, with the script compiled into the DROM and output through putchar ecalls.+ - Sizes: text 205 KB (ROM), rodata 26 KB (DROM), bss 2 KB, and a GC heap of about 23 KB.+ - s1 and s5 are free-running exact. rv_tool measured 38,964 / 490,092 instructions with+ Zba+Zbb; this build takes 39,888 / 492,526.+- **calc** (P2.4b, `env/calc/`, `chipformer/rv/calc.py`): C firmware on the digit file.+ - It handles + - * / // % ** ^ gcd() and parentheses with Python's semantics, on values up to+ 62 digits.+ - It has a fast path for "A op B" and a shunting-yard loop otherwise.+ - Long division estimates each quotient digit from the window's top 3 digits and the divisor's+ top 2 (at most 2 too big), then runs BMSB and an add-back if needed.+ - **10,000 random cases equal Python on the reference machine.**+ - **All 10,000 are also free-running exact on the circuit** (the executor on the L4):+ 8,951,798 instructions and 12,298,128 tokens in 19 min.+ - Six expressions are teacher-forced in the IR with the kernel check+ (test_rv_core.test_calc).+ - Tokens per case (from the text in RAM to the end of the print), against CHIP-8 v15 (the+ median of 20; CHIP-8 counts from after its CALL record, so its CALL record, about+ 2 tokens per digit plus 10, is added):++| Call | RV32 calc tokens (instructions) | of which digit-file / print tokens | CHIP-8 v15 after CALL | CHIP-8 incl. CALL |+|---|---|---|---|---|+| add 4+3 | 254 (202) | 25 | 30 | ~54 |+| add 12+12 | 303 (210) | 76 | 55 | ~113 |+| sub 8-5 | 289 (218) | 44 | 42 | ~78 |+| mul 2x2 | 340 (283) | 20 | 44 | ~62 |+| mul 4x3 | 377 (296) | 38 | 68 | ~92 |+| mul 8x8 | 520 (352) | 113 | 206 | ~248 |+| mul 12x12 | 684 (402) | 217 | 394 | ~452 |+| div 4/4 | 435 (352) | 23 | 526 | ~552 |+| div 6/2 | 660 (510) | 52 | 570 | ~596 |+| div 12/4 | 882 (618) | 114 | 751 | ~793 |++ - The digit work itself (BD / SC / BYTE / QS / QD tokens) costs about 1 token per digit per+ row, about the same as the CHIP-8 macros.+ - What costs more is the C around it: about 200-250 tokens per call for the dispatch,+ register bookkeeping and the call / return of the bignum routines, against CHIP-8's 22-byte+ BIOS.+ - On random 1-24-digit operands the means are: add 358, sub 432, mul 680, div 1,117,+ mod 1,056, gcd 3,394, pow 1,731, and mixed 2-4-operand expressions 991.+ - Closing the gap would mean writing the dispatcher in assembly, or giving more of the+ control to the digit-file ops.+- **Executor v2** (P2.5; NVIDIA L4, crc32 printf at 1.53 tokens per instruction):++| Path | Batch | Per step | tok/s | instr/s |+|---|---|---|---|---|+| P1 executor (dense weight blocks), P2 circuit | 1 | 1.066 ms | 938 | 614 |+| sparse weights (gather, product, scatter-add), torch.compile + CUDA graph | 1 | 0.261 ms | 3,836 | 2,509 |+| sparse | 8 | 0.427 ms | 18,738 | 12,254 |+| sparse | 32 | 1.132 ms | 28,261 | 18,483 |+| **executor v2: Triton, one fused kernel per sublayer, CUDA graph** | **1** | **0.177 ms** | **5,657** | **3,699** |+| executor v2 | 8 | 0.441 ms | 18,150 | 11,870 |+| executor v2 | 32 | 1.502 ms | 21,309 | 13,936 |++ - **Batch 1 is now 3.7k instructions/s, 1.8x P1's 2.0k**, on a circuit twice the size+ (and 6x the P1 executor on the P2 circuit).+ - At batch 8 the two paths are level; at batch 32 the sparse torch path is ahead (27k+ tok/s).+ - v2's value-head programs (one per vhead and row) are latency-bound and scale badly+ with batch. The next levers are several rows per program, and one persistent kernel+ for the whole step.+ - v2 is checked free-running: the fuzz set (64 programs, 152,848 instructions) and+ MicroPython s1, gmpmul, crc32p, rv32um-div and rv32um-rem (58,880 instructions),+ all exact.++ - The P2 circuit's dense weight blocks are 165 MB per step, against 0.78M nonzeros, so+ batch 1 is bandwidth-bound. Sparse rows cut that about 25x.+ - `tritonexec.Executor2` fuses each sublayer into one Triton kernel under one CUDA graph:+ - MLP: a block of units per program.+ - Heads: one program per value head: projections over the lagged residual, decay, the+ one-row write, the k-hot read with the ballast norm, and the output columns.+ - Plus an apply kernel.+ - It equals the IR on every dim.+- **rv-tiny on the GPU:** a stock `Qwen3_5ForCausalLM`, 2,112M parameters, hidden 2816,+ 12 layers, saved on q35-eval in `~/wrv2/dist/rv-tiny-p2`.+ - bf16 with the fla kernels, for rv32um-div, rv32um-rem, rv32uzba-sh2add and c-crc32p: the+ residual before the exit equals the IR on every dim, and free-running `generate`+ reproduces the reference trace. crc32p's 6,282 generated tokens print `crc32=414fa339`.+ - tinycheck now warms up fla's autotuner before each check. On a new shape the autotuner+ reruns kernels that update the recurrent state in place, which corrupted the first+ result.+ - Longer traces (gmpmul, 22k tokens) run out of memory in tinycheck's one-shot prefill:+ the logits alone are 22k x 230k. This is a limit of the check, not of the model.+- **Re-checked after the one-hot quotient heads** (free-running on the final circuit):+ - MicroPython s5 and s1, qsort, crc32p, gmpmul and rv32um div / divu / rem / remu:+ 750,680 instructions with executor v2.+ - 2,000 calc cases: 1,773,438 instructions.+ - The fuzz set: 64 programs, 152,848 instructions.+ - All exact.++### Not in P2+- **Zbb:** it measured under 0.5% for MicroPython.+- **RAM:** it stays at 32 KB. That is in the asked 32-64 KB range, and MicroPython's rodata now+ sits in the DROM.+- **The text bridge to Qwen:** that is P3. The calc input sits in RAM (`calc_in`, 0x100) for+ now; P3's capture will write it.+- **calc's C overhead** (above).+- **A persistent single-kernel step:** v2 uses one kernel per sublayer instead.++## P1: the RV32IM core (minus DIV), standalone++### The core+- **ISA:** all of RV32I, plus MUL / MULH / MULHSU / MULHU through a 5-stage multiplier. DIV and REM+ are an illegal-instruction fault; C code uses gcc `-march=rv32i_zmmul`, which sends DIV/REM to+ libgcc. FENCE commits as NT.+- **System:**+ - `ecall` a7 = 1 is putchar: the record is one `BYTE(a0 & 255)` token.+ - `ecall` a7 = 93 is exit (`HALT`).+ - The CSR stubs are pure reads, folded into the ROM as constants: cycle / time / instret (+h)+ read 0, mhartid 0, misa 0x40001100. Any CSR write is illegal.+- **Faults halt and print.** The record is `FLT(cause)`, the PC's 8 hex digits (`HEX`), then+ `HALT`. The causes are illegal, ebreak, misaligned load / store, load / store out of range, bad+ jump (misaligned or outside ROM; static targets are flagged in the ROM image) and unknown ecall.+- **Sizes:**+ - ROM 128 KB in 2 sublayers (32 key heads).+ - RAM 32 KB in 2 sublayers (32 key heads × 2 value heads × 128 rows, words).+ - The address map: RAM at 0 (data, rodata, TLS, bss and heap from 0x100, the stack below+ 0x8000), ROM at 0x10000.+- **Tokens:**+ - V / F0 / F1 have 15-bit payloads; J / R have 16 bits; BYTE 8; FLT 3; HEX 4; there are 7+ controls.+ - The vocabulary is **229,667 rows**.+ - The uop byte keeps 64..127 free for the custom-0/1 bignum ops of P2.+- **Circuit:**+ - **11 layers** (the exit is folded into the output sublayer), 1,253 IR dims, 69 heads,+ 19,576 MLP units (at most 6,198 per sublayer), **501,396 nonzero weights**.+ - A j counter head (2^-j, tokens of the current record) replaces P0's previous-token head.++| Layer | Attention | MLP |+|---|---|---|+| L1 | pc, count, j | thermometers |+| L2 | — | PC word, link |+| L3-L4 | ROM | field decode, immediate, link |+| L5 | registers | adder s1, logic, shift s1, multiplier s1 (products), store data |+| L6 | — | adder s2, shift s2, multiplier s2 (product nibbles) |+| L7-L8 | RAM | multiplier s3-s4, loads, merge, compares, faults |+| L9 | — | multiplier s5 |+| L10 | — | rd value and size |+| L11 | — | output code + exit |++### Gate results+- **Compliance in the IR:** riscv-tests **rv32ui (38 tests) and rv32um mul / mulh / mulhsu / mulhu,+ all green**, teacher-forced, with HF-kernel read emulation on traces under 1,500 tokens.+ fence_i and ma_data are excluded, and so is rv32um div / rem (no DIV).+- **Other IR tests:**+ - 10 fault programs, one per cause plus variants.+ - The CSR stubs and putchar.+ - ALU / branch / MUL fuzz.+ - The n = 63 refresh.+ - Snapshots at n > 0.+ - C programs: crc32 printing through picolibc printf (13.4k tokens), strops, and the+ **mini-gmp 20×20-digit multiply (16k instructions)**. The last two take 12-44k tokens, so+ they run under `RV_SLOW=1`.+- **Reference vs qemu-riscv32:**+ - 54 ELFs (gcc 13.2; C with picolibc for rv32i, libgcc): exit codes and stdout equal qemu's.+ - The traced machine ends in the ISA emulator's exact register state.+- **C programs verified free-running** (the executor generates every token itself; each must+ equal the reference trace up to HALT):++ | Program | Instructions | Output |+ |---|---|---|+ | fib(25), recursive | 2,117,133 | `fib(25)=75025` |+ | qsort 1000 ints (picolibc) | 256,975 | sorted, hash 72e4eb11 |+ | sieve below 10^4 | 126,897 | `primes below 10000: 1229` |+ | mini-gmp 20×20 multiply | 15,967 | `mul=1219326311370217952249809480012498094790` |+ | crc32, sort, strops, sieve(2000) | 51,459 | |+ | **total** | **2,568,431** | all equal |++ sudoku-easy (8.4M instructions) passes the qemu / reference check but was not free-run.+- **Fuzz** (`chipformer/rv/fuzz.py`): random fault-free RV32IM programs covering every ALU,+ shift and MUL op, all load / store widths through masked addresses, branches, calls, putchar,+ LUI / AUIPC. **64 programs, 1,481,677 instructions free-running, all equal.**+- **In total, 4.05M instructions** are verified free-running.+- **`rv-tiny`** (`python -m chipformer.rv.tinycheck --save dist/rv-tiny`; saved on q35-eval in+ `~/wrv/dist/rv-tiny`):+ - **The model:** a stock `Qwen3_5ForCausalLM`, 960M parameters, hidden 1280, 12 layers (11+ chip + 1 zero full-attention), Qwen3.5-4B's DeltaNet head shapes, vocabulary 229,667 + PAD,+ with tied embeddings. The code dims of each token row score bit agreement.+ - **`firmware.py`:** ELF → `ref.Machine` → `core.snapshot` → the HF cache, after a zero PAD+ prefix, then plain greedy `generate`.+ - **Checks:** the bf16 residual before the exit layer equals the IR on every circuit dim at+ every position, and **free-running `generate` reproduces the reference trace exactly**.+ - On GPU with the fla kernels: 7 programs, including crc32, whose 13.4k generated tokens+ print `crc32=414fa339`.+ - On CPU with the torch kernels: 3 programs.+- **Speed** (NVIDIA L4, crc32 trace at 1.34 tokens per instruction):++ | Path | Per step | tok/s | instr/s |+ |---|---|---|---|+ | executor, batch 1 (torch.compile + CUDA graph) | 0.37 ms | 2,706 | 2,017 |+ | executor, batch 8 | 0.60 ms | 13,367 | 9,963 |+ | executor, batch 32 | 0.85 ms | 37,743 | 28,131 |+ | rv-tiny HF modules, StaticCache + torch.compile (reduce-overhead) | 8.9 ms | 112 | 83 |+ | the same after `fast.compact` (nonzero blocks only) | 1.64 ms | 609 | 454 |++ - **The executor** (`fastexec.py`) now has batch rows and free-running generation (decode plus+ embedding on the GPU, one graph per step).+ - **The stock HF path is not within 2× of the executor.** Compacted and compiled it is 4.4×+ slower at batch 1, and 24× without compaction. The dense layers read 0.96B parameters per+ token, including the tied 229k-row output head. After compaction it is launch-bound: each+ layer runs conv, l2norm, the fla GDN kernel, gated RMSNorm and the MLP.+ - Both rows of the HF path generate the exact reference trace.+- **Tokens per instruction:** 1.0-1.55 on the C programs (1.40 averaged over crc32, sieve, sort+ and strops); 1.19 on the P0.6 small-C mix.++### Not in P1+- DIV/REM in hardware (the P2 micro-code).+- The text I/O bridge to Qwen (P3). The BYTE family stands in for Qwen's byte tokens.+- causal-conv1d (it doesn't build on q35-eval).++# P0: feasibility (branch `rv-p0`)++Everything below is measured on `rv-p0` unless marked *estimate*.++## Verdict++| Criterion (plan P0.8) | Result | |+|---|---|---|+| IR exact | 38 riscv-tests rv32ui + 4 gcc -O2 C programs (up to 31.7k instructions / 44k tokens) + 480 fuzzed ALU ops + branch fuzz, teacher-forced: 0 wrong tokens. Every head read also replayed as the HF torch kernels compute it (2^-3.5 q scale, bf16 cast of o, gated RMSNorm): exact | GO |+| Real HF modules | stock `Qwen3_5ForCausalLM` (bf16, torch reference kernels, CPU), snapshot injected into the cache: residual == IR on every IR dim at every position, chunked prefill and token-by-token decode | GO |+| Depth ≤ 12 (P1), ≤ 32 (P2) | the P0 skeleton is **8 layers**; P1 extrapolates to ~11-13, P2 to ~16-20 (below) | GO |+| Width / text-visible dims ≤ 3072 | 884 IR dims, all chip-only (overlayable); text-visible = output code (19) + mu/tau/ballast until the P3 bridge adds capture state | GO |+| Byte columns exact on fla kernels | **bf16: bit-exact** on an NVIDIA L4 with fla 0.5.2: `chunk_gated_delta_rule` (prefill) and `fused_recurrent_gated_delta_rule` (decode) on 9 programs, every IR dim at every position, including crc32's 4,075 positions (`gpucheck --decode`). **fp32: 0 wrong tokens** on the same 18 runs, but the residual is not bit-exact; see "fp32" below | GO (bf16) |+| Compact executor ≥ 3k instr/s | **3.6k instr/s on an NVIDIA L4** (5,562 tok/s, 0.18 ms per token at crc32's 1.55 tokens per instruction; about 4.7k at the small-C mix's 1.19): the circuit's own executor (`fastexec.py`), torch.compile'd and captured as one CUDA graph, its replays checked exact against the IR | GO |++## Design as built++### Trace format (`spec.py`)+One record per instruction; state changes only at a record's last token, and the PC is implied:+PC = ROM_BASE + 4·(anchor + n), with the anchor the payload of the latest J/R token and n the commits+since (an R refresh is emitted before the instruction when n reaches 63).++| Record | Tokens |+|---|---|+| rd-writing (ALU, LUI, AUIPC, loads) | `F0` · `V F1` · `V V F2` by the value's sign-extended size (15-bit chunks) |+| rd = x0, not-taken branch, FENCE | `NT` |+| taken branch, JAL, JALR | `J(word)`; the link rd = PC+4 is computed in-circuit |+| SW | `ST0`/`ST1` (the RAM value head = address bit 6); the data is rs2 from t-1, in-circuit |+| SB, SH | `V V V SS0/SS1`: the merged word (old bytes merged at the top of the circuit) |+| ECALL, EBREAK, illegal | `HALT` |++Vocabulary: V, F0, F1, J, R × 32,768 + F2 × 4 + 7 controls = **163,851 rows**.++### The commit rule+Every position computes the instruction its *next* token belongs to: at a record's last token+the circuit already works on the next instruction; inside a record it recomputes the current one.+A write takes its **key from t-1** (a conv tap), its **write gate beta from the token at t** (beta =+sigmoid(256 g − 128) with g ∈ {≤0, 1}, exactly 0 or 1), and its **value from token payloads at lags+0..3 or from dims computed at t-1 below the writing layer**. The payload-to-bit wiring needs no+constants (which would break at the sequence start): for "V bit at t-1 when t is F1" the channel is+P = −Vb + isF1 + isV with taps (1, −1), which equals the bit exactly when t is F1 and is ≤ 0+otherwise. Information only flows down through tokens, so nothing writes a value computed above+its own layer at the same position; that is why the PC is implied rather than written.++### Memory+| Table | Layout | Read | Write |+|---|---|---|---|+| ROM (code) | 16 key heads × 2 value heads; row pairs (p, p+64); 8 bytes per instruction **pre-decoded** at load: (uop, rdw, rs1, rs2) in value head 0, imm in value head 1; byte-valued columns | 2-hot query {p, p+64} (effective q scale 2^-4, exact for bytes); the word is selected by z per column block, so no select MLP | snapshot only |+| registers | 2 key heads (rs1, rs2 ports), 32 rows, 109 columns per row (payload sources per bit + the link) | one-hot query; a bit = sum of its columns | key rdw(t-1), beta = F0+F1+F2+J |+| RAM | 16 key heads × 2 value heads × 128 rows × 1 word = 16 KB per sublayer; key head = EA[5:2], value head = EA[6], row = EA[13:7] | one-hot query, z picks the value head | key = address(t-1) AND store(t-1); beta = ST_x/SS_x |+| pc, count, prev | 1 head each | | J/R payload; 2^-n decay counter; token kinds at lags 1-3 |++The ROM is a compiled image, so static facts are folded in (`ref.rom_fields`): rdw = rd only if+the instruction writes rd, AUIPC/LUI are constants, branch and JAL targets are absolute words.+The uop byte has room for the custom-0/custom-1 bignum ops planned for P2.++Address map: **RAM at 0** (.data from 0x100, the stack below 0x4000), **ROM at 0x10000**. RAM at+the bottom makes data and stack pointers 1-token values: 1.42 → 1.19 tokens per instruction on+the small-C mix.++### Datapath (8 layers)+| Layer | Attention | MLP (units) |+|---|---|---|+| L1 | pc, count, prev (3 heads) | n thermometer, record position j (129) |+| L2 | — | PC word w = anchor + n, link w+1, thresholds + carry into bit 6 (1,072) |+| L3 | ROM (16) | field one-hots, immediate bits, gated link (4,660) |+| L4 | registers (2) | adder stage 1 for 4 operand modes (generate / propagate / digit bits per nibble), logic ops, shifter stage 1, store data (5,216) |+| L5 | — | adder stage 2 (carries and result bits in one stage), shifter stage 2 (1,008) |+| L6 | RAM (16) | load lanes / sign extension, SB/SH merge, SLT/SLTU/zero (492) |+| L7 | — | rd value select, its size class, branch decision (190) |+| L8 | — | output code: family id + 15-bit payload (175) |++The 32-bit add is 2 MLP sublayers: nibble sums are free linear forms, then stage 1 thresholds them+per operand mode (with the mode as an AND literal, so no operand mux layer), then stage 2 applies+the carry sources as AND literals on the digit bits.++## Measurements (`python -m chipformer.rv.measure --nonzeros`)+- **Size:** 884 IR dims, 37 heads, 12,942 MLP units; **298,676 nonzero weights** in the 8 chip layers.+ At 4B width the dense chip layers hold 1.08B parameters, almost all zeros. The ROM, RAM and+ registers live in the recurrent state, not in the weights.+- **Capacity per sublayer:** ROM 64 KB (pre-decoded, 8 bytes per instruction), RAM 16 KB (words).+- **Tokens per instruction:**+ - Measured on gcc -O2 C programs (crc32, sieve, sort, strops): **1.40**. Byte stores are 4 tokens+ and sieve is store-heavy.+ - The small-C mix from P0.6 gives **1.19**, against 3-5 for CHIP-8.+- **Payload width:**++| Chunk bits | Value records (1/2/3 tokens) | Tokens per instruction, mix | Vocabulary rows | Embedding at 3072 wide |+|---|---|---|---|---|+| 12 | 20.4k / 4.2k / 2.6k | 1.44 (2-token jumps) | 20.7k | 0.13 GB |+| **15** (built) | 23.6k / 2.2k / 1.4k | **1.19** | 164k | 1.01 GB |+| 16 | 23.7k / 3.5k / — | 1.11 (SB/SH as V+SS) | 459k | 2.82 GB |++ Recommendation: keep 15-bit V/F and make **J/R 16-bit** (+65k rows), since 256 KB ROMs need+ 16-bit word indices (MicroPython with float is 229 KB).+- **HF cache snapshot (R11):** injecting the initial states into `DynamicCache` recurrent states+ after a zero (text-token) prefix works for prefill and decode (`hf.py`). Static cache not tried.+- **Speed** (`python -m chipformer.rv.timing --hidden 3072`, batch 1, crc32 trace at 1.55 tokens+ per instruction):++ | Path (NVIDIA L4) | ms/token | tok/s | instr/s |+ |---|---|---|---|+ | HF modules, dense, eager, DynamicCache (4B width: hidden 3072, 16/32 heads, 9216 units) | 11.0 | 91 | 59 |+ | the same after `fast.compact` (nonzero blocks only) | 12.5 | 80 | 52 |+ | circuit executor (`fastexec.py`), eager | 5.1 | 194 | 126 |+ | executor, one CUDA graph per step | 0.53 | 1,898 | 1,226 |+ | **executor, torch.compile + CUDA graph** | **0.18** | **5,562** | **3,594** |++ - **What the executor is.** It runs the IR's arithmetic in fp32 over the circuit's live+ structure:+ - each sublayer's projections as one matmul over the input dims they read;+ - one-hot key writes as row scatters, ≤2-hot reads as top-2 row gathers;+ - decay only on the counter.+ - **How it is checked.** It equals `ir.evaluate` on every dim at every position, eager on CPU and+ on GPU, and its CUDA-graph replays are checked against the IR on 300 positions.+ - **It replaces the HF modules for chip-only steps.** Same arithmetic, not the same kernels.+ The HF rows stay eager and launch-bound here; the speed fork's static-cache CUDA-graph decode+ path (`decode_rows`, about 1.3 ms for a 16-layer chip step on an L40S) was not applied to+ this model.+ - **L40S *estimate*.** By bandwidth alone (864 vs 300 GB/s) the executor row would be about+ 16k tok/s on an L40S. These steps are launch- and latency-bound, so the real gain should be+ smaller; it is an upper estimate.+ - **Next.** A persistent single-kernel step (Triton/CUDA) over the 0.3M nonzeros is the next+ step if more is needed.++## GPU numerics (P0.2, NVIDIA L4, fla 0.5.2, `gpucheck --decode`)++Programs: signext, call, memcpy, rv32ui-lb / -sh / -jalr / -sra, c-crc32 (4,075 positions) and+c-strops. Each runs as one chunked prefill (`chunk_gated_delta_rule`) and token by token+(`fused_recurrent_gated_delta_rule`), with the snapshot injected into the cache.++- **bf16** (the merged model's dtype): **bit-exact.** Every IR dim matches at every position in+ all 18 runs. That covers the byte-column 2-hot ROM reads, the sibling value heads' write gates,+ the lag-1 key commits through prefill-then-decode, and the counter's decays through the chunk+ kernel's exp(cumsum).+- **fp32: 0 wrong tokens** in all 18 runs, but 70-400 dims per run are not bit-exact (1-ulp+ noise).+ - **Why.** Without the final bf16 cast, the kernel's q scale 1/sqrt(128) and the chunk kernel's+ exp(cumsum) decays leave about 2^-23 relative error, e.g. a counter read of 0.2500611 for 0.25.+ - **Why the tokens are still right.** The circuit's AND/threshold margins absorb it.+ - **Status.** fp32 decoding is correct, not bit-exact. Making it bit-exact would need power-of-2+ q scales the kernels don't have; bf16 is the target dtype.++## Options measured or costed+- **8-byte granule RAM rows** (the spike fork's alternative): 2× RAM per sublayer, but SW becomes+ 4 tokens (+0.27 tokens per instruction on the mix). RAM needs are tiny (P0.6: at most 16 KB+ touched), so word rows stay.+- **ROM density:** pre-decoded 64 KB per sublayer. 256 KB = 4 sublayers, which adds about 3 layers+ of fetch depth. The alternative is raw 4-byte words at 128 KB per sublayer plus one decode+ MLP stage. Pick for P1 against the depth budget.+- **SB/SH cost** is 4 tokens with 15-bit payloads; 2 with 16-bit (V + SS16).++## P1 / P2 extrapolation (*estimates*)+- **P1** (all RV32I, MUL via 5 parallel MLP stages, putchar/exit, 128 KB ROM, 32-64 KB RAM): MUL+ runs beside the adder/RAM path (L4-L8), so it adds no depth. The ROM grows to 2 sublayers (+1),+ RAM to 2-4 sublayers (+1..3) and output select (+1), giving **11-13 layers**.+- **P2** (256 KB ROM, micro-coded DIV, CARRY/instret, custom bignum ops): **16-20 layers**.++## Gaps+- **causal-conv1d is untested.** It doesn't build on q35-eval: its CUDA toolkit is 12.8, while+ torch is built for cu13.0. The conv runs through the torch fallback, as in every run above.+- **Not in the P0 circuit:** MUL*, DIV*/REM*, putchar, faults (misaligned, out of range), text I/O.+- `fence_i` (self-modifying code) and `ma_data` (misaligned access) are skipped, as planned.++## Files+- `chipformer/rv/`:+ - `spec.py`: tokens and records.+ - `ref.py`: the ISA emulator, `rom_fields`, the traced `Machine`.+ - `asm.py`: assembler.+ - `elf.py`: ELF loader.+ - `programs.py`: assembly test programs.+ - `core.py`: the circuit and snapshot.+ - `hf.py`: HF modules plus cache injection.+ - `measure.py`, `gpucheck.py` (GPU numerics), `timing.py`.+ - `fastexec.py`: the compact circuit executor (the P0.5 prototype).+ - `env/`: riscv_test.h, link scripts, crt0, C programs, `build_tests.sh` (WSL).+- `tests/`:+ - `test_rv_ref.py`: ref == qemu-riscv32 on 44 ELFs; Machine == ISA.+ - `test_rv_core.py`: the circuit, teacher-forced.+ - `test_rv_hf.py`.+ - `data/rv/`: the built ELFs and qemu results.+- **IR additions** (`q35/ir.py`):+ - from store-v1: sibling value heads, per-value-head beta, `into`.+ - new here: head `init` (snapshot) states, byte-valued and decay levels (snapshots at n > 0, tested mid-run), `kernel_check` (HF kernel+ emulation), beta g ≤ 0.+- `q35/prims.py`: `thead`.diff --git a/experimental/riscv/pyproject.toml b/experimental/riscv/pyproject.tomlnew file mode 100644index 0000000..0c6eebd--- /dev/null+++ b/experimental/riscv/pyproject.toml@@ -0,0 +1,33 @@+[project]+name = "chipformer"+version = "0.1.0"+description = "A hand-constructed transformer that executes CHIP-8 programs"+requires-python = ">=3.11"+dependencies = [+ "numpy>=1.26",+ "safetensors>=0.8.0",+]++[project.optional-dependencies]+play = ["pygame>=2.5"]+hf = [+ "torch>=2.14.1",+ "transformers>=5.19.0",+]++[dependency-groups]+dev = [+ "pillow>=12.3.0",+ "pytest>=8",+ "ruff>=0.6",+]++[build-system]+requires = ["hatchling"]+build-backend = "hatchling.build"++[tool.ruff]+line-length = 100++[tool.pytest.ini_options]+testpaths = ["tests"]diff --git a/experimental/riscv/tests/data/rv/c-crc32.elf b/experimental/riscv/tests/data/rv/c-crc32.elfnew file mode 100644index 0000000..5d18ce7Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-crc32.elf differdiff --git a/experimental/riscv/tests/data/rv/c-crc32.qemu.elf b/experimental/riscv/tests/data/rv/c-crc32.qemu.elfnew file mode 100644index 0000000..ae09b33Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-crc32.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/c-crc32p.elf b/experimental/riscv/tests/data/rv/c-crc32p.elfnew file mode 100644index 0000000..171a741Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-crc32p.elf differdiff --git a/experimental/riscv/tests/data/rv/c-crc32p.qemu.elf b/experimental/riscv/tests/data/rv/c-crc32p.qemu.elfnew file mode 100644index 0000000..816404cBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-crc32p.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/c-fib.elf b/experimental/riscv/tests/data/rv/c-fib.elfnew file mode 100644index 0000000..79b2fddBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-fib.elf differdiff --git a/experimental/riscv/tests/data/rv/c-fib.qemu.elf b/experimental/riscv/tests/data/rv/c-fib.qemu.elfnew file mode 100644index 0000000..0b89decBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-fib.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/c-gmpmul.elf b/experimental/riscv/tests/data/rv/c-gmpmul.elfnew file mode 100644index 0000000..9d968c3Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-gmpmul.elf differdiff --git a/experimental/riscv/tests/data/rv/c-gmpmul.qemu.elf b/experimental/riscv/tests/data/rv/c-gmpmul.qemu.elfnew file mode 100644index 0000000..effe55cBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-gmpmul.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/c-qsort.elf b/experimental/riscv/tests/data/rv/c-qsort.elfnew file mode 100644index 0000000..69a1877Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-qsort.elf differdiff --git a/experimental/riscv/tests/data/rv/c-qsort.qemu.elf b/experimental/riscv/tests/data/rv/c-qsort.qemu.elfnew file mode 100644index 0000000..747971cBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-qsort.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/c-sieve.elf b/experimental/riscv/tests/data/rv/c-sieve.elfnew file mode 100644index 0000000..495b694Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-sieve.elf differdiff --git a/experimental/riscv/tests/data/rv/c-sieve.qemu.elf b/experimental/riscv/tests/data/rv/c-sieve.qemu.elfnew file mode 100644index 0000000..46a43f2Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-sieve.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/c-sieve4.elf b/experimental/riscv/tests/data/rv/c-sieve4.elfnew file mode 100644index 0000000..3b44636Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-sieve4.elf differdiff --git a/experimental/riscv/tests/data/rv/c-sieve4.qemu.elf b/experimental/riscv/tests/data/rv/c-sieve4.qemu.elfnew file mode 100644index 0000000..ceec675Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-sieve4.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/c-sort.elf b/experimental/riscv/tests/data/rv/c-sort.elfnew file mode 100644index 0000000..d3b537fBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-sort.elf differdiff --git a/experimental/riscv/tests/data/rv/c-sort.qemu.elf b/experimental/riscv/tests/data/rv/c-sort.qemu.elfnew file mode 100644index 0000000..5b2285cBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-sort.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/c-strops.elf b/experimental/riscv/tests/data/rv/c-strops.elfnew file mode 100644index 0000000..f39809aBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-strops.elf differdiff --git a/experimental/riscv/tests/data/rv/c-strops.qemu.elf b/experimental/riscv/tests/data/rv/c-strops.qemu.elfnew file mode 100644index 0000000..2e0b075Binary files /dev/null and b/experimental/riscv/tests/data/rv/c-strops.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/c-sudoku.elf b/experimental/riscv/tests/data/rv/c-sudoku.elfnew file mode 100644index 0000000..8d8867fBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-sudoku.elf differdiff --git a/experimental/riscv/tests/data/rv/c-sudoku.qemu.elf b/experimental/riscv/tests/data/rv/c-sudoku.qemu.elfnew file mode 100644index 0000000..d3ec57bBinary files /dev/null and b/experimental/riscv/tests/data/rv/c-sudoku.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/calc-bridge.elf b/experimental/riscv/tests/data/rv/calc-bridge.elfnew file mode 100644index 0000000..3c1dce8Binary files /dev/null and b/experimental/riscv/tests/data/rv/calc-bridge.elf differdiff --git a/experimental/riscv/tests/data/rv/calc.elf b/experimental/riscv/tests/data/rv/calc.elfnew file mode 100644index 0000000..7f80bdcBinary files /dev/null and b/experimental/riscv/tests/data/rv/calc.elf differdiff --git a/experimental/riscv/tests/data/rv/mpy-s1.elf b/experimental/riscv/tests/data/rv/mpy-s1.elfnew file mode 100644index 0000000..1156f1dBinary files /dev/null and b/experimental/riscv/tests/data/rv/mpy-s1.elf differdiff --git a/experimental/riscv/tests/data/rv/mpy-s1.qemu.elf b/experimental/riscv/tests/data/rv/mpy-s1.qemu.elfnew file mode 100644index 0000000..d0914fcBinary files /dev/null and b/experimental/riscv/tests/data/rv/mpy-s1.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/mpy-s5.elf b/experimental/riscv/tests/data/rv/mpy-s5.elfnew file mode 100644index 0000000..8e0dc4aBinary files /dev/null and b/experimental/riscv/tests/data/rv/mpy-s5.elf differdiff --git a/experimental/riscv/tests/data/rv/mpy-s5.qemu.elf b/experimental/riscv/tests/data/rv/mpy-s5.qemu.elfnew file mode 100644index 0000000..b3f5d91Binary files /dev/null and b/experimental/riscv/tests/data/rv/mpy-s5.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/qemu.json b/experimental/riscv/tests/data/rv/qemu.jsonnew file mode 100644index 0000000..dd37ce9--- /dev/null+++ b/experimental/riscv/tests/data/rv/qemu.json@@ -0,0 +1,254 @@+{+ "c-crc32": {+ "exit": 57,+ "stdout": ""+ },+ "c-crc32p": {+ "exit": 0,+ "stdout": "crc32=414fa339\n"+ },+ "c-fib": {+ "exit": 0,+ "stdout": "fib(25)=75025\n"+ },+ "c-gmpmul": {+ "exit": 0,+ "stdout": "mul=1219326311370217952249809480012498094790\n"+ },+ "c-qsort": {+ "exit": 0,+ "stdout": "qsort 1000: err=0 min=4940 max=16772127 hash=72e4eb11\n"+ },+ "c-sieve": {+ "exit": 47,+ "stdout": ""+ },+ "c-sieve4": {+ "exit": 0,+ "stdout": "primes below 10000: 1229\n"+ },+ "c-sort": {+ "exit": 253,+ "stdout": ""+ },+ "c-strops": {+ "exit": 180,+ "stdout": ""+ },+ "c-sudoku": {+ "exit": 0,+ "stdout": "solved=1 steps=37652 row0=534678912\n"+ },+ "rv32ui-add": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-addi": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-and": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-andi": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-auipc": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-beq": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-bge": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-bgeu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-blt": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-bltu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-bne": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-jal": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-jalr": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-lb": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-lbu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-ld_st": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-lh": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-lhu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-lui": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-lw": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-or": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-ori": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-sb": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-sh": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-simple": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-sll": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-slli": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-slt": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-slti": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-sltiu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-sltu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-sra": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-srai": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-srl": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-srli": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-st_ld": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-sub": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-sw": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-xor": {+ "exit": 0,+ "stdout": ""+ },+ "rv32ui-xori": {+ "exit": 0,+ "stdout": ""+ },+ "rv32um-div": {+ "exit": 0,+ "stdout": ""+ },+ "rv32um-divu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32um-mul": {+ "exit": 0,+ "stdout": ""+ },+ "rv32um-mulh": {+ "exit": 0,+ "stdout": ""+ },+ "rv32um-mulhsu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32um-mulhu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32um-rem": {+ "exit": 0,+ "stdout": ""+ },+ "rv32um-remu": {+ "exit": 0,+ "stdout": ""+ },+ "rv32uzba-sh1add": {+ "exit": 0,+ "stdout": ""+ },+ "rv32uzba-sh2add": {+ "exit": 0,+ "stdout": ""+ },+ "rv32uzba-sh3add": {+ "exit": 0,+ "stdout": ""+ },+ "mpy-s1": {+ "exit": 0,+ "stdout": "1267650600228229401496703205376\r\n"+ },+ "mpy-s5": {+ "exit": 0,+ "stdout": "[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47]\r\n[('brown', 1), ('dog', 1), ('fox', 1), ('jumps', 1), ('lazy', 1), ('over', 1), ('quick', 1), ('the', 2)]\r\nquick\r\n"+ }+}\ No newline at end of filediff --git a/experimental/riscv/tests/data/rv/rv32ui-add.elf b/experimental/riscv/tests/data/rv/rv32ui-add.elfnew file mode 100644index 0000000..3a9e2ceBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-add.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-add.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-add.qemu.elfnew file mode 100644index 0000000..8c74892Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-add.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-addi.elf b/experimental/riscv/tests/data/rv/rv32ui-addi.elfnew file mode 100644index 0000000..a59f265Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-addi.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-addi.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-addi.qemu.elfnew file mode 100644index 0000000..cb189e6Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-addi.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-and.elf b/experimental/riscv/tests/data/rv/rv32ui-and.elfnew file mode 100644index 0000000..cdcbcc4Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-and.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-and.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-and.qemu.elfnew file mode 100644index 0000000..6ba693bBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-and.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-andi.elf b/experimental/riscv/tests/data/rv/rv32ui-andi.elfnew file mode 100644index 0000000..d5e4b93Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-andi.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-andi.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-andi.qemu.elfnew file mode 100644index 0000000..32f860bBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-andi.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-auipc.elf b/experimental/riscv/tests/data/rv/rv32ui-auipc.elfnew file mode 100644index 0000000..c2b5301Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-auipc.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-auipc.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-auipc.qemu.elfnew file mode 100644index 0000000..62c9860Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-auipc.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-beq.elf b/experimental/riscv/tests/data/rv/rv32ui-beq.elfnew file mode 100644index 0000000..956f419Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-beq.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-beq.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-beq.qemu.elfnew file mode 100644index 0000000..c1b0fe4Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-beq.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-bge.elf b/experimental/riscv/tests/data/rv/rv32ui-bge.elfnew file mode 100644index 0000000..a52d9d9Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-bge.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-bge.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-bge.qemu.elfnew file mode 100644index 0000000..4208bc4Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-bge.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-bgeu.elf b/experimental/riscv/tests/data/rv/rv32ui-bgeu.elfnew file mode 100644index 0000000..e7ce9bfBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-bgeu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-bgeu.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-bgeu.qemu.elfnew file mode 100644index 0000000..b8a7d55Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-bgeu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-blt.elf b/experimental/riscv/tests/data/rv/rv32ui-blt.elfnew file mode 100644index 0000000..63d5c21Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-blt.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-blt.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-blt.qemu.elfnew file mode 100644index 0000000..43335a3Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-blt.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-bltu.elf b/experimental/riscv/tests/data/rv/rv32ui-bltu.elfnew file mode 100644index 0000000..9e9bf80Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-bltu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-bltu.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-bltu.qemu.elfnew file mode 100644index 0000000..0314199Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-bltu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-bne.elf b/experimental/riscv/tests/data/rv/rv32ui-bne.elfnew file mode 100644index 0000000..feff7b2Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-bne.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-bne.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-bne.qemu.elfnew file mode 100644index 0000000..15c9194Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-bne.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-jal.elf b/experimental/riscv/tests/data/rv/rv32ui-jal.elfnew file mode 100644index 0000000..644982fBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-jal.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-jal.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-jal.qemu.elfnew file mode 100644index 0000000..8af2437Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-jal.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-jalr.elf b/experimental/riscv/tests/data/rv/rv32ui-jalr.elfnew file mode 100644index 0000000..29d34edBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-jalr.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-jalr.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-jalr.qemu.elfnew file mode 100644index 0000000..7cd71c7Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-jalr.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lb.elf b/experimental/riscv/tests/data/rv/rv32ui-lb.elfnew file mode 100644index 0000000..c1f4f0cBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lb.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lb.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-lb.qemu.elfnew file mode 100644index 0000000..b2e0785Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lb.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lbu.elf b/experimental/riscv/tests/data/rv/rv32ui-lbu.elfnew file mode 100644index 0000000..e691aa5Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lbu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lbu.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-lbu.qemu.elfnew file mode 100644index 0000000..07e74e8Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lbu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-ld_st.elf b/experimental/riscv/tests/data/rv/rv32ui-ld_st.elfnew file mode 100644index 0000000..d9125cdBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-ld_st.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-ld_st.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-ld_st.qemu.elfnew file mode 100644index 0000000..eaf9ac4Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-ld_st.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lh.elf b/experimental/riscv/tests/data/rv/rv32ui-lh.elfnew file mode 100644index 0000000..af4de60Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lh.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lh.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-lh.qemu.elfnew file mode 100644index 0000000..2ab7147Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lh.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lhu.elf b/experimental/riscv/tests/data/rv/rv32ui-lhu.elfnew file mode 100644index 0000000..b8d9b53Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lhu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lhu.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-lhu.qemu.elfnew file mode 100644index 0000000..a47c2e5Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lhu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lui.elf b/experimental/riscv/tests/data/rv/rv32ui-lui.elfnew file mode 100644index 0000000..47b401dBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lui.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lui.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-lui.qemu.elfnew file mode 100644index 0000000..4f33b79Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lui.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lw.elf b/experimental/riscv/tests/data/rv/rv32ui-lw.elfnew file mode 100644index 0000000..4753a21Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lw.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-lw.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-lw.qemu.elfnew file mode 100644index 0000000..489b72eBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-lw.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-or.elf b/experimental/riscv/tests/data/rv/rv32ui-or.elfnew file mode 100644index 0000000..3dec97bBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-or.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-or.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-or.qemu.elfnew file mode 100644index 0000000..b5f7f06Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-or.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-ori.elf b/experimental/riscv/tests/data/rv/rv32ui-ori.elfnew file mode 100644index 0000000..4f4a774Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-ori.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-ori.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-ori.qemu.elfnew file mode 100644index 0000000..7f9b9b4Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-ori.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sb.elf b/experimental/riscv/tests/data/rv/rv32ui-sb.elfnew file mode 100644index 0000000..17828acBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sb.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sb.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-sb.qemu.elfnew file mode 100644index 0000000..3150975Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sb.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sh.elf b/experimental/riscv/tests/data/rv/rv32ui-sh.elfnew file mode 100644index 0000000..25ebb30Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sh.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sh.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-sh.qemu.elfnew file mode 100644index 0000000..e5da74bBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sh.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-simple.elf b/experimental/riscv/tests/data/rv/rv32ui-simple.elfnew file mode 100644index 0000000..234cf97Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-simple.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-simple.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-simple.qemu.elfnew file mode 100644index 0000000..0665b59Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-simple.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sll.elf b/experimental/riscv/tests/data/rv/rv32ui-sll.elfnew file mode 100644index 0000000..f91b4e5Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sll.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sll.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-sll.qemu.elfnew file mode 100644index 0000000..828b141Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sll.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-slli.elf b/experimental/riscv/tests/data/rv/rv32ui-slli.elfnew file mode 100644index 0000000..812603eBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-slli.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-slli.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-slli.qemu.elfnew file mode 100644index 0000000..ef74c5bBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-slli.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-slt.elf b/experimental/riscv/tests/data/rv/rv32ui-slt.elfnew file mode 100644index 0000000..1e2e580Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-slt.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-slt.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-slt.qemu.elfnew file mode 100644index 0000000..f7c7351Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-slt.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-slti.elf b/experimental/riscv/tests/data/rv/rv32ui-slti.elfnew file mode 100644index 0000000..33b3158Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-slti.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-slti.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-slti.qemu.elfnew file mode 100644index 0000000..0ae368eBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-slti.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sltiu.elf b/experimental/riscv/tests/data/rv/rv32ui-sltiu.elfnew file mode 100644index 0000000..84b3219Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sltiu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sltiu.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-sltiu.qemu.elfnew file mode 100644index 0000000..7b1b2ceBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sltiu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sltu.elf b/experimental/riscv/tests/data/rv/rv32ui-sltu.elfnew file mode 100644index 0000000..c59495dBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sltu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sltu.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-sltu.qemu.elfnew file mode 100644index 0000000..5a836ddBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sltu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sra.elf b/experimental/riscv/tests/data/rv/rv32ui-sra.elfnew file mode 100644index 0000000..251141dBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sra.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sra.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-sra.qemu.elfnew file mode 100644index 0000000..2d8e5e5Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sra.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-srai.elf b/experimental/riscv/tests/data/rv/rv32ui-srai.elfnew file mode 100644index 0000000..fc942ddBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-srai.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-srai.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-srai.qemu.elfnew file mode 100644index 0000000..f72beccBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-srai.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-srl.elf b/experimental/riscv/tests/data/rv/rv32ui-srl.elfnew file mode 100644index 0000000..6d90216Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-srl.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-srl.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-srl.qemu.elfnew file mode 100644index 0000000..efd5a91Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-srl.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-srli.elf b/experimental/riscv/tests/data/rv/rv32ui-srli.elfnew file mode 100644index 0000000..e0b82feBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-srli.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-srli.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-srli.qemu.elfnew file mode 100644index 0000000..ca631a7Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-srli.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-st_ld.elf b/experimental/riscv/tests/data/rv/rv32ui-st_ld.elfnew file mode 100644index 0000000..6bc5a55Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-st_ld.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-st_ld.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-st_ld.qemu.elfnew file mode 100644index 0000000..9f04bd2Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-st_ld.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sub.elf b/experimental/riscv/tests/data/rv/rv32ui-sub.elfnew file mode 100644index 0000000..6c73619Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sub.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sub.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-sub.qemu.elfnew file mode 100644index 0000000..3c8fda6Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sub.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sw.elf b/experimental/riscv/tests/data/rv/rv32ui-sw.elfnew file mode 100644index 0000000..869c33dBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sw.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-sw.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-sw.qemu.elfnew file mode 100644index 0000000..2a099acBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-sw.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-xor.elf b/experimental/riscv/tests/data/rv/rv32ui-xor.elfnew file mode 100644index 0000000..3d29d62Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-xor.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-xor.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-xor.qemu.elfnew file mode 100644index 0000000..c84238dBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-xor.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-xori.elf b/experimental/riscv/tests/data/rv/rv32ui-xori.elfnew file mode 100644index 0000000..1017a79Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-xori.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32ui-xori.qemu.elf b/experimental/riscv/tests/data/rv/rv32ui-xori.qemu.elfnew file mode 100644index 0000000..e97bffbBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32ui-xori.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-div.elf b/experimental/riscv/tests/data/rv/rv32um-div.elfnew file mode 100644index 0000000..3bf040aBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-div.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-div.qemu.elf b/experimental/riscv/tests/data/rv/rv32um-div.qemu.elfnew file mode 100644index 0000000..6f90008Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-div.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-divu.elf b/experimental/riscv/tests/data/rv/rv32um-divu.elfnew file mode 100644index 0000000..91de8daBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-divu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-divu.qemu.elf b/experimental/riscv/tests/data/rv/rv32um-divu.qemu.elfnew file mode 100644index 0000000..2f9b68cBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-divu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-mul.elf b/experimental/riscv/tests/data/rv/rv32um-mul.elfnew file mode 100644index 0000000..0889aa6Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-mul.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-mul.qemu.elf b/experimental/riscv/tests/data/rv/rv32um-mul.qemu.elfnew file mode 100644index 0000000..5a02918Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-mul.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-mulh.elf b/experimental/riscv/tests/data/rv/rv32um-mulh.elfnew file mode 100644index 0000000..5f8ae97Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-mulh.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-mulh.qemu.elf b/experimental/riscv/tests/data/rv/rv32um-mulh.qemu.elfnew file mode 100644index 0000000..01baf81Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-mulh.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-mulhsu.elf b/experimental/riscv/tests/data/rv/rv32um-mulhsu.elfnew file mode 100644index 0000000..e0218a0Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-mulhsu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-mulhsu.qemu.elf b/experimental/riscv/tests/data/rv/rv32um-mulhsu.qemu.elfnew file mode 100644index 0000000..b4bfc8dBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-mulhsu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-mulhu.elf b/experimental/riscv/tests/data/rv/rv32um-mulhu.elfnew file mode 100644index 0000000..a90825dBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-mulhu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-mulhu.qemu.elf b/experimental/riscv/tests/data/rv/rv32um-mulhu.qemu.elfnew file mode 100644index 0000000..48f29b1Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-mulhu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-rem.elf b/experimental/riscv/tests/data/rv/rv32um-rem.elfnew file mode 100644index 0000000..466f3b1Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-rem.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-rem.qemu.elf b/experimental/riscv/tests/data/rv/rv32um-rem.qemu.elfnew file mode 100644index 0000000..56918eeBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-rem.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-remu.elf b/experimental/riscv/tests/data/rv/rv32um-remu.elfnew file mode 100644index 0000000..9cbf88aBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-remu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32um-remu.qemu.elf b/experimental/riscv/tests/data/rv/rv32um-remu.qemu.elfnew file mode 100644index 0000000..c55cc0bBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32um-remu.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32uzba-sh1add.elf b/experimental/riscv/tests/data/rv/rv32uzba-sh1add.elfnew file mode 100644index 0000000..28f0f1fBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32uzba-sh1add.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32uzba-sh1add.qemu.elf b/experimental/riscv/tests/data/rv/rv32uzba-sh1add.qemu.elfnew file mode 100644index 0000000..52699caBinary files /dev/null and b/experimental/riscv/tests/data/rv/rv32uzba-sh1add.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32uzba-sh2add.elf b/experimental/riscv/tests/data/rv/rv32uzba-sh2add.elfnew file mode 100644index 0000000..63791d6Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32uzba-sh2add.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32uzba-sh2add.qemu.elf b/experimental/riscv/tests/data/rv/rv32uzba-sh2add.qemu.elfnew file mode 100644index 0000000..7525aa6Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32uzba-sh2add.qemu.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32uzba-sh3add.elf b/experimental/riscv/tests/data/rv/rv32uzba-sh3add.elfnew file mode 100644index 0000000..6340c60Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32uzba-sh3add.elf differdiff --git a/experimental/riscv/tests/data/rv/rv32uzba-sh3add.qemu.elf b/experimental/riscv/tests/data/rv/rv32uzba-sh3add.qemu.elfnew file mode 100644index 0000000..8733516Binary files /dev/null and b/experimental/riscv/tests/data/rv/rv32uzba-sh3add.qemu.elf differdiff --git a/experimental/riscv/tests/test_rv_bridge.py b/experimental/riscv/tests/test_rv_bridge.pynew file mode 100644index 0000000..be8160c--- /dev/null+++ b/experimental/riscv/tests/test_rv_bridge.py@@ -0,0 +1,149 @@+"""The P3 bridge (rv/bridge.py) in the IR: a tool call's text is captured, </tool_call> starts a+run, putchar output is echoed as Qwen tokens; the circuit must predict every token it owns and+own exactly the positions the reference says."""++from pathlib import Path++import numpy as np+import pytest++from chipformer.q35 import ir+from chipformer.rv import bridge as BR+from chipformer.rv import core, elf, ref+from chipformer.rv import spec as S++CALC = Path(__file__).parent / "data" / "rv" / "calc-bridge.elf"++[email protected](scope="module")+def T():+ try:+ return BR.Text(BR.default_tokenizer())+ except FileNotFoundError:+ pytest.skip("no Qwen3.5 tokenizer in the HF cache")++[email protected](scope="module")+def circuit(T):+ return core.build(text=T)+++def call(T, expr: str) -> list[int]:+ return T.encode(f"<tool_call>\n<function=calc>\n<parameter=expression>\n{expr}\n"+ "</parameter>\n</function>\n</tool_call>")+++def session(T):+ m = ref.Machine.from_program(elf.program(str(CALC)))+ BR.park(m)+ return BR.Session(T, m), m+++def check(c, ses, m0, skip: set[int] = frozenset()):+ """IR == reference: owner at every position, the next token wherever the circuit owns it+ (positions in ``skip``: owned, but the stream is prefilled history)."""+ core.snapshot(c, m0)+ toks = ses.toks+ carry, mats = {}, []+ for a in range(0, len(toks), 2048):+ mats.append(ir.evaluate(c, toks[a:a + 2048], carry=carry))+ mat = np.concatenate(mats)+ pred = core.predictions(c, mat)+ own = np.rint(ir.ev(c.owner, mat)).astype(int)+ assert [i for i in range(len(toks)) if own[i] != int(ses.own[i])] == []+ bad = [i for i in range(len(toks) - 1)+ if ses.own[i] and i not in skip and pred[i] != toks[i + 1]]+ assert bad == [], [(i, toks[i], pred[i], toks[i + 1]) for i in bad[:5]]+++def fresh_machine():+ m = ref.Machine.from_program(elf.program(str(CALC)))+ BR.park(m)+ return m++[email protected]("expr,want", [("12*34", 408), ("2*(3+4)-50", -36),+ ("99999999999*99999999999", 99999999999**2)])+def test_calc_call(T, circuit, expr, want):+ m0 = fresh_machine()+ ses = BR.Session(T, fresh_machine())+ ses.qwen(T.encode("<|im_start|>assistant\n"))+ ses.qwen(call(T, expr))+ ses.qwen(T.encode("So the answer is"))+ out = T.decode_bytes(ses.runs[-1]).decode()+ assert out == f"\n<tool_response>\n{want}\n</tool_response>\n<think>\n\n</think>\n\n"+ check(circuit, ses, m0)+++def test_history_then_call(T, circuit):+ """A prefilled earlier call (its response as text) must not start a run, and the next real+ call still works (the buffer is cleared at <tool_call>)."""+ m0 = fresh_machine()+ ses = BR.Session(T, fresh_machine())+ hist = call(T, "1+1") + T.encode("\n<tool_response>\n2\n</tool_response>\nIt is 2. ")+ k = len(ses.toks)+ ses.history(hist)+ skip = {k + hist.index(T.TCC)}+ ses.qwen(call(T, "7*6"))+ assert T.decode_bytes(ses.runs[-1]).decode().split("\n")[2] == "42"+ check(circuit, ses, m0, skip)+++def test_two_calls_and_overflow(T, circuit):+ """Two runs in one stream; a call longer than the buffer is truncated (its later tokens+ all land in the last row), and the circuit still matches the reference."""+ m0 = fresh_machine()+ ses = BR.Session(T, fresh_machine())+ ses.qwen(call(T, "3*3"))+ ses.qwen(T.encode(" and "))+ long = call(T, "+".join(["1"] * 60))+ assert len(long) > S.CAP_MAX + 2+ ses.qwen(long)+ outs = [T.decode_bytes(r).decode().split("\n")[2] for r in ses.runs]+ assert outs[0] == "9" and outs[1] != "60"+ check(circuit, ses, m0)+++def test_decimal_call(T, circuit):+ """Decimal operands (+ - * exact, the point placed, trailing zeros dropped)."""+ m0 = fresh_machine()+ ses = BR.Session(T, fresh_machine())+ ses.qwen(call(T, "(1.2 - 3) * 4.5 - 0.75"))+ assert T.decode_bytes(ses.runs[-1]).decode().split("\n")[2] == "-8.85"+ check(circuit, ses, m0)+++def test_fraction_call(T, circuit):+ """Exact division: a reduced fraction (// stays floor)."""+ m0 = fresh_machine()+ ses = BR.Session(T, fresh_machine())+ ses.qwen(call(T, "-84/18"))+ ses.qwen(call(T, "-84//18"))+ assert [T.decode_bytes(r).decode().split("\n")[2] for r in ses.runs] == ["-14/3", "-5"]+ check(circuit, ses, m0)+++def test_decimal_reference():+ import random++ from chipformer.rv import calc++ rng = random.Random(3)++ def num():+ a = str(rng.randint(0, 10 ** rng.randint(1, 7)))+ if rng.random() < 0.7:+ a += "." + str(rng.randint(0, 10 ** rng.randint(1, 5))).rjust(rng.randint(1, 5), "0")+ return ("-" if rng.random() < 0.3 else "") + a++ exprs = ["0.1*0.2", "1.5-1.5", "100.0", "7.5/2"] + \+ [f"({num()}{rng.choice('+-*')}{num()}){rng.choice('+-*')}{num()}" for _ in range(40)]+ for e in exprs:+ assert calc.run(e)[0] == calc.expected(e), e+++def test_reference_buffer(T):+ ses, m = session(T)+ ses.qwen(call(T, "5"))+ rows = [r.rstrip(bytes(1)) for r in m.text[1:S.CAP_MAX + 1] if r.strip(bytes(1))]+ assert b"".join(rows).startswith(b"\n<function=calc>")diff --git a/experimental/riscv/tests/test_rv_core.py b/experimental/riscv/tests/test_rv_core.pynew file mode 100644index 0000000..6524c87--- /dev/null+++ b/experimental/riscv/tests/test_rv_core.py@@ -0,0 +1,282 @@+"""The RV32 core circuit (chipformer/rv/core.py) against the reference trace, teacher-forced."""++import json+import os+import random+from pathlib import Path++import pytest++from chipformer.q35 import ir+from chipformer.rv import core, elf, ref+from chipformer.rv import spec as S+from chipformer.rv.asm import assemble+from chipformer.rv.programs import ALL, program++DATA = Path(__file__).parent / "data" / "rv"+ELFS = sorted(json.loads((DATA / "qemu.json").read_text()))+SLOW = {"c-sieve", "c-sort", "c-crc32p", "c-gmpmul"} # 12k-44k tokens: RV_SLOW=1 runs them+# 0.13M-8M instructions: the executor runs these free-running (test_rv_fuzz.py), not the IR+FREE_RUN = {"c-fib", "c-sudoku", "c-qsort", "c-sieve4", "mpy-s1", "mpy-s5"}+EXIT = "\nli a7, 93\necall"++[email protected](scope="module")+def circ():+ return core.build()+++def run_check(c, prog, kernel=True):+ core.snapshot(c, ref.Machine.from_program(prog))+ toks = ref.Machine.from_program(prog).run()+ mat = ir.evaluate(c, toks, check=True, kernel=kernel)+ pred = core.predictions(c, mat)+ bad = [(i, S.name(toks[i + 1]), pred[i]) for i in range(len(toks) - 1) if pred[i] != toks[i + 1]]+ assert not bad, bad[:5]+ return toks++[email protected]("name", list(ALL))+def test_programs(circ, name):+ run_check(circ, program(name))++[email protected]("name", ELFS)+def test_compiled(circ, name):+ """riscv-tests rv32ui / rv32um / rv32uzba and gcc -O2 C programs, teacher-forced over their+ run."""+ if name in FREE_RUN:+ pytest.skip("checked free-running by the executor")+ if name in SLOW and not os.environ.get("RV_SLOW"):+ pytest.skip("long trace (RV_SLOW=1)")+ prog = elf.program(DATA / f"{name}.elf")+ n = len(ref.Machine.from_program(prog).run(10**7))+ toks = run_check(circ, prog, kernel=n < 1500)+ assert S.family(toks[-1])[0] == "HALT"+++def test_depth_and_caps(circ):+ st = circ.stats()+ assert st["layers"] <= 12, st+ assert st["max_heads"] <= circ.target.key_heads+ assert st["max_units"] <= circ.target.intermediate+++def random_alu_program(rng: random.Random, n: int) -> str:+ regs = ["a0", "a1", "a2", "a3", "a4", "a5", "t0", "t1", "t2", "s1"]+ lines = [f"li {r}, {rng.randint(-2**31, 2**31 - 1)}" for r in regs]+ ops3 = ["add", "sub", "and", "or", "xor", "sll", "srl", "sra", "slt", "sltu"]+ opsi = ["addi", "andi", "ori", "xori", "slti", "sltiu"]+ opss = ["slli", "srli", "srai"]+ for _ in range(n):+ k = rng.random()+ d, a, b = rng.choice(regs), rng.choice(regs), rng.choice(regs)+ if k < 0.5:+ lines.append(f"{rng.choice(ops3)} {d}, {a}, {b}")+ elif k < 0.8:+ lines.append(f"{rng.choice(opsi)} {d}, {a}, {rng.randint(-2048, 2047)}")+ elif k < 0.9:+ lines.append(f"{rng.choice(opss)} {d}, {a}, {rng.randint(0, 31)}")+ else:+ lines.append(f"lui {d}, {rng.randint(0, 2**20 - 1)}")+ return "_start: " + "\n".join(lines) + EXIT++[email protected]("seed", range(4))+def test_alu_fuzz(circ, seed):+ rng = random.Random(seed)+ run_check(circ, assemble(random_alu_program(rng, 120)), kernel=seed == 0)+++def test_branch_fuzz(circ):+ rng = random.Random(7)+ lines = ["_start: li s0, 40"]+ for i in range(12):+ a, b = rng.randint(-5, 5), rng.randint(-5, 5)+ br = rng.choice(["beq", "bne", "blt", "bge", "bltu", "bgeu"])+ lines += [f"li a0, {a}", f"li a1, {b}", f"{br} a0, a1, skip{i}", "addi s1, s1, 1",+ f"skip{i}: addi s0, s0, -1"]+ lines += ["loop: addi s0, s0, -1", "bgtz s0, loop"]+ run_check(circ, assemble("\n".join(lines) + EXIT))+++def test_refresh_after_63(circ):+ # 70 straight-line instructions: an R record is due before the 64th+ src = "_start: " + "\n".join(f"addi a0, a0, {i % 7}" for i in range(70)) + EXIT+ toks = run_check(circ, assemble(src))+ assert any(S.family(t)[0] == "R" for t in toks)++[email protected]("skip", [5, 40, 133])+def test_snapshot_mid_run(circ, skip):+ """A snapshot taken mid-run (n > 0, live registers and RAM) continues exactly."""+ prog = program("call")+ m = ref.Machine.from_program(prog)+ for _ in range(skip):+ m.record()+ assert 0 < m.n < S.N_MAX+ core.snapshot(circ, m)+ toks = m.run()+ mat = ir.evaluate(circ, toks, check=True, kernel=True)+ pred = core.predictions(circ, mat)+ assert all(pred[i] == toks[i + 1] for i in range(len(toks) - 1))+++FAULT_PROGRAMS = {+ "illegal": "_start: li a0, 7\n.word 0x0000000b",+ "illegal (csr write)": "_start: li a0, 1\ncsrw mscratch, a0",+ "ebreak": "_start: nop\nebreak",+ "misaligned load": "_start: li a0, 0x102\nlw a1, 0(a0)",+ "misaligned store": "_start: li a0, 0x101\nsh a0, 0(a0)",+ "load out of range": "_start: li a0, 0x8000\nlw a1, 0(a0)",+ "store out of range": "_start: li a0, -4\nsb a0, 0(a0)",+ "bad jump": "_start: li a0, 0x4000\njr a0",+ "bad jump (misaligned)": "_start: la a0, t\naddi a0, a0, 2\njr a0\nt: nop",+ "ecall": "_start: li a7, 5\necall",+}++[email protected]("name", list(FAULT_PROGRAMS))+def test_faults(circ, name):+ """Each fault halts with FLT(cause), the PC's 8 hex digits and HALT, exactly as the reference."""+ toks = run_check(circ, assemble(FAULT_PROGRAMS[name]))+ cause = name.split(" (")[0]+ fams = [S.family(t) for t in toks]+ assert fams[-10] == ("FLT", S.FAULTS.index(cause)) and fams[-1][0] == "HALT"+ assert all(f[0] == "HEX" for f in fams[-9:-1])+++def test_csr_stubs_and_putchar(circ):+ src = """_start: csrr a0, mhartid+ csrr a1, misa+ csrr a2, cycle+ csrr a3, instret+ li a7, 1+ li a0, 72+ ecall+ li a0, 105+ ecall""" + EXIT+ prog = assemble(src)+ toks = run_check(circ, prog)+ m = ref.Machine.from_program(prog)+ m.run()+ assert m.out == b"Hi" and m.x[11] == S.MISA and m.x[12] == m.x[13] == 0+ assert [S.family(t) for t in toks if S.family(t)[0] == "BYTE"] == [("BYTE", 72), ("BYTE", 105)]++[email protected]("seed", range(3))+def test_mul_fuzz(circ, seed):+ rng = random.Random(100 + seed)+ regs = ["a0", "a1", "a2", "a3", "t0", "t1"]+ lines = [f"li {r}, {rng.choice([rng.randint(-2**31, 2**31 - 1), rng.randint(-9, 9)])}"+ for r in regs]+ for _ in range(60):+ op = rng.choice(["mul", "mulh", "mulhsu", "mulhu", "add", "xor"])+ lines.append(f"{op} {rng.choice(regs)}, {rng.choice(regs)}, {rng.choice(regs)}")+ run_check(circ, assemble("_start: " + "\n".join(lines) + EXIT), kernel=seed == 0)+++DIV_EDGE = [0, 1, -1, 2, -2, 7, -7, 15, 16, 255, 256, 1000, 2**31 - 1, -2**31, -2**31 + 1, 0xFFFF,+ -0x10000, 10, 3]++[email protected]("seed", range(4))+def test_div_fuzz(circ, seed):+ """DIV / DIVU / REM / REMU on edge values and random operands of every size (digits skipped+ or not, signs, division by 0, overflow), rd = x0 and rd == rs1 included."""+ rng = random.Random(200 + seed)+ regs = ["a0", "a1", "a2", "a3", "t0", "t1", "s1"]++ def val():+ return rng.choice([rng.choice(DIV_EDGE), rng.randint(-2**31, 2**31 - 1),+ rng.randint(-2**(rng.randint(1, 31)), 2**rng.randint(1, 31))])++ lines = [f"li {r}, {val()}" for r in regs]+ for i in range(36):+ if i % 6 == 5:+ lines += [f"li {r}, {val()}" for r in rng.sample(regs, 3)]+ op = rng.choice(["div", "divu", "rem", "remu"])+ d = "zero" if rng.random() < 0.1 else rng.choice(regs)+ lines.append(f"{op} {d}, {rng.choice(regs)}, {rng.choice(regs)}")+ run_check(circ, assemble("_start: " + "\n".join(lines) + EXIT), kernel=seed == 0)+++def test_zba_and_rodata(circ):+ """sh1add / sh2add / sh3add, and loads of every width from the DROM (.rodata)."""+ src = """_start: li a0, 0x1234567+ li a1, -77+ sh1add a2, a0, a1+ sh2add a3, a1, a0+ sh3add a4, a0, a0+ la s0, tab+ lw t0, 0(s0)+ lw t1, 4(s0)+ lb t2, 5(s0)+ lbu t3, 7(s0)+ lh t4, 6(s0)+ lhu t5, 2(s0)+ sh2add t6, a0, s0""" + EXIT + """+ .rodata+ tab: .word 0x89abcdef, 0x80ff7f01"""+ prog = assemble(src)+ run_check(circ, prog)+ m = ref.Machine.from_program(prog)+ m.run()+ assert m.x[12] == (0x1234567 * 2 - 77) & 0xFFFFFFFF+ assert (m.x[5], m.x[7], m.x[28], m.x[29], m.x[30]) == (0x89abcdef, 0x7f, 0x80, 0xFFFF80FF,+ 0x89ab)+++def bn_ops(n=0, off=0, m=0, d=0, s=0, ovw=0):+ """A digit-file row op's operand word (rs2)."""+ return d | s << 3 | ovw << 6 | n << 7 | off << 13 | m << 20+++def test_digit_file(circ):+ """BSCAN / BLD / BMAC (add, scaled, overwrite) / BMSB (shifted, with a borrow) / BPRD / BGET."""+ src = f"""_start: la a0, txt+ bscan a1, a0+ add a2, a0, a1+ slli t2, a1, 7+ ori t0, t2, {bn_ops(d=1)}+ bld zero, a2, t0+ li a2, {bn_ops(n=6, m=1, d=2, s=1)}+ bmac a3, zero, a2+ bmac a3, zero, a2+ li t0, {bn_ops(n=5, m=7, d=3, s=1, ovw=1)}+ bmac a4, zero, t0+ li t0, {bn_ops(n=5, off=2, m=3, d=2, s=1)}+ bmsb a6, zero, t0+ bprd 2, 7+ bprd 2, 6+ bprd 2, 5+ bprd 2, 0+ li t1, {bn_ops(off=3, d=3)}+ bget a5, zero, t1+ addi a0, a0, 6+ bscan s1, a0""" + EXIT + """+ .data+ txt: .byte 0x31, 0x32, 0x33, 0x34, 0x39, 0x2b, 0x37, 0x41"""+ prog = assemble(src)+ run_check(circ, prog)+ m = ref.Machine.from_program(prog)+ m.run()+ assert m.bn[1][:5] == [9, 4, 3, 2, 1] and m.out == b"0638"+ assert (m.x[11], m.x[13], m.x[14], m.x[15], m.x[16], m.x[9]) == (5, 0, 0, 6, 1, 1)++[email protected]("expr", ["123456789*987654321", "-1000000007//37", "gcd(462,1071)",+ "(7-19)%5", "3**13", "12345678901234567890-12345678901234567891"])+def test_calc(circ, expr):+ """The calc firmware (env/calc, the digit-file ops), teacher-forced in the IR."""+ from chipformer.rv import calc++ m = calc.machine(expr)+ core.snapshot(circ, m)+ toks = calc.machine(expr).run(10**6)+ mat = ir.evaluate(circ, toks, check=True, kernel=len(toks) < 1500)+ pred = core.predictions(circ, mat)+ assert all(pred[i] == toks[i + 1] for i in range(len(toks) - 1))+ mm = calc.machine(expr)+ mm.run(10**6)+ assert mm.out.decode() == calc.expected(expr)diff --git a/experimental/riscv/tests/test_rv_fuzz.py b/experimental/riscv/tests/test_rv_fuzz.pynew file mode 100644index 0000000..5f81d90--- /dev/null+++ b/experimental/riscv/tests/test_rv_fuzz.py@@ -0,0 +1,34 @@+"""Random RV32IM programs (chipformer/rv/fuzz.py): fault-free in the reference, exact in the IR+(teacher-forced), and generated free-running by the executor (CPU here; the 10^6-instruction+runs are on a GPU: python -m chipformer.rv.fuzz)."""++import pytest++from chipformer.q35 import ir+from chipformer.rv import core, fuzz, ref+from chipformer.rv import spec as S+from chipformer.rv.asm import assemble++[email protected](scope="module")+def circ():+ return core.build()++[email protected]("seed", range(3))+def test_fuzz_ir(circ, seed):+ prog = assemble(fuzz.gen_program(1000 + seed, block=40, iters=3))+ m = ref.Machine.from_program(prog)+ core.snapshot(circ, m)+ toks = ref.Machine.from_program(prog).run()+ assert m.fault is None+ mat = ir.evaluate(circ, toks, check=True, kernel=seed == 0)+ pred = core.predictions(circ, mat)+ bad = [i for i in range(len(toks) - 1) if pred[i] != toks[i + 1]]+ assert not bad, (bad[0], S.name(toks[bad[0] + 1]))+++def test_fuzz_free_running_cpu(circ):+ progs = [assemble(fuzz.gen_program(2000 + s, block=30, iters=2)) for s in range(2)]+ res = fuzz.free_run(circ, progs, batch=2, device="cpu", compile_=False)+ assert res["instructions"] > 100diff --git a/experimental/riscv/tests/test_rv_hf.py b/experimental/riscv/tests/test_rv_hf.pynew file mode 100644index 0000000..252e9e6--- /dev/null+++ b/experimental/riscv/tests/test_rv_hf.py@@ -0,0 +1,30 @@+"""The RV32 core in stock Qwen3_5ForCausalLM modules (bf16, the torch reference kernels): with the+state snapshot injected into the HF cache, the residual stream equals the IR evaluator's on every+IR dim at every position, for a chunked prefill and for token-by-token decoding."""++import pytest++from chipformer.rv import core, hf, ref+from chipformer.rv.programs import program++[email protected](scope="module")+def circ():+ return core.build()++[email protected]("mode", ["prefill", "decode"])+def test_signext_in_hf(circ, mode):+ p = program("signext")+ core.snapshot(circ, ref.Machine.from_program(p))+ toks = ref.Machine.from_program(p).run()+ model, lay, ids = hf.build_model(circ, toks)+ assert hf.check(circ, model, lay, ids, toks, mode) == len(toks)+++def test_call_prefill_in_hf(circ):+ p = program("call")+ core.snapshot(circ, ref.Machine.from_program(p))+ toks = ref.Machine.from_program(p).run()+ model, lay, ids = hf.build_model(circ, toks)+ assert hf.check(circ, model, lay, ids, toks, "prefill") == len(toks)diff --git a/experimental/riscv/tests/test_rv_ref.py b/experimental/riscv/tests/test_rv_ref.pynew file mode 100644index 0000000..295ea95--- /dev/null+++ b/experimental/riscv/tests/test_rv_ref.py@@ -0,0 +1,110 @@+"""The RV32 reference: ISA emulator == qemu-riscv32, traced machine == ISA emulator, records."""++import json+import os+import random+from pathlib import Path++import pytest++from chipformer.rv import elf, ref+from chipformer.rv import spec as S+from chipformer.rv.programs import ALL, program++DATA = Path(__file__).parent / "data" / "rv"+QEMU = json.loads((DATA / "qemu.json").read_text())+LONG = {"c-sudoku", "c-fib"} # 8.4M / 2.1M instructions: RV_SLOW=1+++def run_rv32(path, sp):+ cpu = ref.Rv32()+ elf.load_into(cpu, path)+ cpu.x[2] = sp+ while not cpu.halted and cpu.steps < 2 * 10**7:+ cpu.step()+ return cpu+++def slow(name):+ if name in LONG and not os.environ.get("RV_SLOW"):+ pytest.skip("long run (RV_SLOW=1)")++[email protected]("name", sorted(QEMU))+def test_against_qemu(name):+ """Same code at qemu's addresses: exit code and output equal qemu's (built by+ chipformer/rv/env/build_tests.sh in WSL)."""+ slow(name)+ cpu = run_rv32(DATA / f"{name}.qemu.elf", 0x7FFF0000)+ assert cpu.exit_code & 0xFF == QEMU[name]["exit"]+ assert cpu.out.decode("latin-1") == QEMU[name]["stdout"]+ if name.startswith("rv32u"):+ assert QEMU[name]["exit"] == 0 # every compliance test passes++[email protected]("name", sorted(QEMU))+def test_machine_equals_isa(name):+ """The traced machine (records, implied PC, ROM/RAM map) ends in the ISA emulator's state and+ prints what qemu printed."""+ slow(name)+ m = ref.Machine.from_program(elf.program(DATA / f"{name}.elf"))+ toks = m.run(10**8)+ cpu = run_rv32(DATA / f"{name}.elf", S.SP_INIT)+ assert m.halted and m.fault is None and m.x == cpu.x+ assert ref.trace_stats(toks)["instructions"] == cpu.steps+ assert m.exit_code & 0xFF == QEMU[name]["exit"]+ assert m.out.decode("latin-1") == QEMU[name]["stdout"]++[email protected]("name", list(ALL))+def test_asm_programs(name):+ p = program(name)+ m = ref.Machine.from_program(p)+ m.run()+ cpu = ref.Rv32()+ for i, w in enumerate(p.text):+ cpu.store(S.ROM_BASE + 4 * i, w, 4)+ for i, b in enumerate(p.data):+ cpu.mem[p.data_base + i] = b+ cpu.x[2], cpu.pc = S.SP_INIT, p.entry+ while not cpu.halted:+ cpu.step()+ assert m.x == cpu.x+++def test_value_records_roundtrip():+ rng = random.Random(0)+ vals = [0, 1, -1, 2**14 - 1, -2**14, 2**14, -2**14 - 1, 2**29 - 1, -2**29, 2**29, 2**31 - 1,+ -2**31] + [rng.getrandbits(32) for _ in range(2000)]+ for v in vals:+ rec = S.value_record(v)+ assert S.decode_value(rec) == v & 0xFFFFFFFF+ assert [S.is_final(t) for t in rec] == [False] * (len(rec) - 1) + [True]+++def test_rom_words_cover_the_programs():+ """Every instruction of every program pre-decodes to a known uop (only CSR writes, which+ the machine does not have, become ILLEGAL)."""+ for name in sorted(QEMU):+ p = elf.program(DATA / f"{name}.elf")+ for i, w in enumerate(p.text):+ op = ref.decode(w).op+ if w and op != "ILLEGAL":+ rw = ref.rom_word(w, S.ROM_BASE + 4 * i)+ assert 0 < rw & 0x7F < len(ref.UOPS) or op.startswith("CSR"), (name, hex(w))+++def test_div_records():+ """DIV-class records: QS(k0), the digits k0..7 of |A| / |B|, then the value; the skipped+ digits are 0 and the result is the ISA's (div_parts asserts both)."""+ rng = random.Random(1)+ M = 0xFFFFFFFF+ edge = [0, 1, 2, 7, 15, 16, 255, 256, 0x7FFFFFFF, 0x80000000, 0x80000001, M, M - 1, 10, 1000]+ vals = edge + [rng.getrandbits(rng.choice([4, 8, 16, 24, 32])) for _ in range(500)]+ for op in ("DIV", "DIVU", "REM", "REMU"):+ for _ in range(5000):+ a, b = rng.choice(vals), rng.choice(vals)+ rec = ref.div_record(op, a, b)+ k0, digits, res = ref.div_parts(op, a, b)+ assert S.family(rec[0]) == ("QS", k0) and len(digits) == 8 - k0+ assert S.decode_value(rec[1 + len(digits):]) == res == ref.alu(op, a, b)diff --git a/experimental/riscv/uv.lock b/experimental/riscv/uv.locknew file mode 100644index 0000000..0f81384--- /dev/null+++ b/experimental/riscv/uv.lock@@ -0,0 +1,1388 @@+version = 1+revision = 3+requires-python = ">=3.11"+resolution-markers = [+ "python_full_version >= '3.12'",+ "python_full_version < '3.12'",+]++[[package]]+name = "annotated-doc"+version = "0.0.5"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/5a/8e/38aa427ed5402449e226975b649c5dc73ccadfefeb95e6aecb8f8ea4b6b6/annotated_doc-0.0.5.tar.gz", hash = "sha256:c7e58ce09192557605d8bbd92836d7e1d520ac9580096042c0bfd197efacf1bb", size = 10758, upload-time = "2026-07-28T13:50:58.129Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/3e/30/e900b21425a860e195f32e37657aa1f7c7f2b1bfb26f03ca209b90933c06/annotated_doc-0.0.5-py3-none-any.whl", hash = "sha256:117bac03a25ede5df5440e855b32d556049ca169ead221505badf432fed4b101", size = 5302, upload-time = "2026-07-28T13:50:57.239Z" },+]++[[package]]+name = "anyio"+version = "4.15.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "idna" },+ { name = "typing-extensions", marker = "python_full_version < '3.15'" },+]+sdist = { url = "https://files.pythonhosted.org/packages/a9/d2/f4d173e22df740bc37b1db102b386ba719b66e95b0f0d751f556b387e6d2/anyio-4.15.1.tar.gz", hash = "sha256:9f28306018cbd6d329e64a36d58256edff76dd996fe423bc957326e578b82a94", size = 276966, upload-time = "2026-09-05T10:42:39.44Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/12/b8/4bd346e22b28902df4d651910f5242c28d84e4a5c2435ca5c3f797ed7e2e/anyio-4.15.1-py3-none-any.whl", hash = "sha256:6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101", size = 132079, upload-time = "2026-09-05T10:42:37.923Z" },+]++[[package]]+name = "chipformer"+version = "0.1.0"+source = { editable = "." }+dependencies = [+ { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },+ { name = "numpy", version = "2.5.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },+ { name = "safetensors" },+]++[package.optional-dependencies]+hf = [+ { name = "torch" },+ { name = "transformers" },+]+play = [+ { name = "pygame" },+]++[package.dev-dependencies]+dev = [+ { name = "pillow" },+ { name = "pytest" },+ { name = "ruff" },+]++[package.metadata]+requires-dist = [+ { name = "numpy", specifier = ">=1.26" },+ { name = "pygame", marker = "extra == 'play'", specifier = ">=2.5" },+ { name = "safetensors", specifier = ">=0.8.0" },+ { name = "torch", marker = "extra == 'hf'", specifier = ">=2.14.1" },+ { name = "transformers", marker = "extra == 'hf'", specifier = ">=5.19.0" },+]+provides-extras = ["play", "hf"]++[package.metadata.requires-dev]+dev = [+ { name = "pillow", specifier = ">=12.3.0" },+ { name = "pytest", specifier = ">=8" },+ { name = "ruff", specifier = ">=0.6" },+]++[[package]]+name = "click"+version = "8.5.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/c7/0e/7fa0ef50764b67090eca4114772a2abf8b6148198475e54c660b97caeee6/click-8.5.0.tar.gz", hash = "sha256:ba0d2089de75ea0310e2dde03160e6ca10009947fb95a182f9b54021bb272e34", size = 382235, upload-time = "2026-08-26T13:33:14.56Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/58/50/6c0d534c5f134586a8e1ba4e330569e32f057e33372ae556463212fb4cd3/click-8.5.0-py3-none-any.whl", hash = "sha256:255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360", size = 125251, upload-time = "2026-08-26T13:33:12.928Z" },+]++[[package]]+name = "colorama"+version = "0.4.6"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },+]++[[package]]+name = "cuda-bindings"+version = "13.4.4"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "cuda-pathfinder" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/a1/15/0277326dd1a2d619ea8a85b790fc9e43cb67897d3a3377eb783f5bc2a33a/cuda_bindings-13.4.4-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b8b2e4235b552edb42cdbe39de7a9259ceec631e21c7344b76177db9cc6c58b4", size = 6494096, upload-time = "2026-10-14T12:35:11.314Z" },+ { url = "https://files.pythonhosted.org/packages/97/36/3bfb7b74817a0a78f80340fae742a27d166e8fcdffa459b79d5be66e077a/cuda_bindings-13.4.4-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e1cef1eaf5a08cc3bf3675ca9d3f97f3fc7085998556c848b4053e74ba4ba392", size = 7160763, upload-time = "2026-10-14T12:35:13.773Z" },+ { url = "https://files.pythonhosted.org/packages/d7/38/87909f62598d02d9e5c536326aed1755f6bfa190285507b962bb601e0005/cuda_bindings-13.4.4-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b346bfe1dda49537537c06cc4727b2210d949d8aee883cb591bca46581629d00", size = 6488222, upload-time = "2026-10-14T12:35:20.703Z" },+ { url = "https://files.pythonhosted.org/packages/c2/f8/332443cf969734551891cdbd93657eaeaf747ac2b7dd73feb3422701b02b/cuda_bindings-13.4.4-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:433efa31e33c2868621b5f6036bd3bf5de9c45c5546cb1361a5f6ff3415b8bb5", size = 7172184, upload-time = "2026-10-14T12:35:22.817Z" },+ { url = "https://files.pythonhosted.org/packages/a7/97/7d75105a92ee18b7764f5cf60672d6a966842348a2a049e663fe690902ff/cuda_bindings-13.4.4-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0c3f4cc887eac0d5edb5d7396cc78b336bee151a97e2ba7acf59e8b9380132d0", size = 6316532, upload-time = "2026-10-14T12:35:30.291Z" },+ { url = "https://files.pythonhosted.org/packages/ba/8f/aedec367fea64267594a26ede3704f241e1bee465962f3a92d6186c774b7/cuda_bindings-13.4.4-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5d6adc44f098472d8ac2d6211923eea6081830521852dc77f8c9237bf9320a59", size = 6929120, upload-time = "2026-10-14T12:35:32.508Z" },+ { url = "https://files.pythonhosted.org/packages/e6/f0/d71d1dd45caa358a39c6898dcc9f1bd22a9f22b92376b967835c047b50bc/cuda_bindings-13.4.4-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4a429132f06d0c3a853aa419aabe4c0a7bfc3f3886fb4c336d81729f63754d31", size = 6407946, upload-time = "2026-10-14T12:35:39.826Z" },+ { url = "https://files.pythonhosted.org/packages/15/65/0b8f2f2509e5f1a8e556a2e083ac0287fe4af79afdba3153454229306f39/cuda_bindings-13.4.4-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:680e70dcb8b61fdae16d3bb4876c41d55aadc9fe8b3825ea567f2f2c85606b0a", size = 6978954, upload-time = "2026-10-14T12:35:42.892Z" },+ { url = "https://files.pythonhosted.org/packages/f1/dd/81903887199ffd90f9ee29eee8ecd497b734007e78b80689e0dd2f904bd8/cuda_bindings-13.4.4-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:03a8fd2a25530dbae26ecd4462d418156028e35b3f3b081440ff6368506e10eb", size = 6314269, upload-time = "2026-10-14T12:35:50.607Z" },+ { url = "https://files.pythonhosted.org/packages/90/44/f6325a6061f1fc619e216dd6c7cbd9a0b475d3937bb32296c84cc27e6eeb/cuda_bindings-13.4.4-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0ce9208c7cf03a947ac11553b08ee2fca594ac63d34360639a9060980f546a62", size = 6865978, upload-time = "2026-10-14T12:35:52.874Z" },+ { url = "https://files.pythonhosted.org/packages/c2/26/00d628420ac318a6a6d559fe6602e08a0156e7f4b5bf9dc6ea20c63789bc/cuda_bindings-13.4.4-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:50bc70ef3b225f4794f5b374773629836bbfdf4a21fc975ac25e7daad67494b8", size = 6414169, upload-time = "2026-10-14T14:12:29.952Z" },+ { url = "https://files.pythonhosted.org/packages/63/88/5b50b13cef9ae8baadcca03a47cf0e9146c0b52ca6aeabaf50fc17e18445/cuda_bindings-13.4.4-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6781fe08ed5f4b5163a3c6577e079bc28955811382d35995c916bf052ee0994e", size = 7012626, upload-time = "2026-10-14T14:12:32.224Z" },+ { url = "https://files.pythonhosted.org/packages/63/cb/013d9fd501dc1b05c3f467f28ad3adeed9e4d39318a4668a7a75fc5b19da/cuda_bindings-13.4.4-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a4c97f8430d7f7c4a4d0f245510d26647dd81ea3fc65941f7a815ad395cbc30e", size = 6302110, upload-time = "2026-10-14T14:12:39.47Z" },+ { url = "https://files.pythonhosted.org/packages/c8/5d/ca0d5c1d9c09a9de15f474c1fa6abe801761e75cdc1be9a03431063f2af5/cuda_bindings-13.4.4-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:baa0f5f1f9a05032e7deb97051e85aa18b4e3c6db914f1c813bb2f02e5c72db4", size = 6878099, upload-time = "2026-10-14T14:12:41.504Z" },+]++[[package]]+name = "cuda-pathfinder"+version = "1.8.3"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b9/fb/f8e1890428f9f590b4beebd63b068aac1ce32a3331510c847b9f9a78f261/cuda_pathfinder-1.8.3-py3-none-any.whl", hash = "sha256:e29e59829c297a7a5233bd9cc71094fc5bddbd076951482670178f9eade39b1f", size = 62561, upload-time = "2026-10-02T03:20:23.712Z" },+]++[[package]]+name = "cuda-toolkit"+version = "13.0.3.0"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d1/c7/a79086a62c98befcdb8349656c6f114e2db3b8b2422f6e25c97a7f2a9a3c/cuda_toolkit-13.0.3.0-py2.py3-none-any.whl", hash = "sha256:d693caaa261214ddd7dbb60d68e71cbed884e68c2be7509778f3051da0b91c3f", size = 2512, upload-time = "2026-04-14T00:50:08.173Z" },+]++[package.optional-dependencies]+cublas = [+ { name = "nvidia-cublas", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-cuda-nvrtc", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cudart = [+ { name = "nvidia-cuda-runtime", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cufft = [+ { name = "nvidia-cufft", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-nvjitlink", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cufile = [+ { name = "nvidia-cufile", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cupti = [+ { name = "nvidia-cuda-cupti", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+curand = [+ { name = "nvidia-curand", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cusolver = [+ { name = "nvidia-cublas", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-cusolver", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-cusparse", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-nvjitlink", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cusparse = [+ { name = "nvidia-cusparse", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-nvjitlink", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+nvjitlink = [+ { name = "nvidia-nvjitlink", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+nvrtc = [+ { name = "nvidia-cuda-nvrtc", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+nvtx = [+ { name = "nvidia-nvtx", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]++[[package]]+name = "filelock"+version = "4.2.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/f4/a9/1af41b37c3279712b22cdc63aac78a52432202b6fe1f9666a2a3d2831fb4/filelock-4.2.0.tar.gz", hash = "sha256:7a60906c75227cf04d0c273afadc8219400f11aeb13cc69591d4f6cdc6c8036e", size = 569667, upload-time = "2026-10-14T20:57:13.11Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/8e/a3/9bc26acff301fe1aaea1cc3d82a1d57e0a34df3e1cadbfa91ac2dbcdde5c/filelock-4.2.0-py3-none-any.whl", hash = "sha256:2ff5690882e8cdb00ef31fb3d01a3094c29f30985426c59495afb1733f3b7238", size = 135032, upload-time = "2026-10-14T20:57:11.349Z" },+]++[[package]]+name = "fsspec"+version = "2026.9.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/77/cd/9be253869fc42e764de7f3dedd6969af7d44ff9c3375214a3442a6f3fc08/fsspec-2026.9.0.tar.gz", hash = "sha256:0f08147951c8cb31d844c3547d631053b127863b60be04cf06e121333ee0e2fe", size = 333545, upload-time = "2026-09-18T17:50:42.825Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/6c/c0/a98505f18594f1bce828bb159cec0fcf9860562f1a2c85913409fc8f3d9e/fsspec-2026.9.0-py3-none-any.whl", hash = "sha256:8dd6e646e99ea382bd85f97a45e6b526a442d79423a7dc673f1e2756d05fcb5f", size = 221738, upload-time = "2026-09-18T17:50:41.341Z" },+]++[[package]]+name = "h11"+version = "0.16.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },+]++[[package]]+name = "hf-xet"+version = "1.7.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/9e/27/06d899ea7bd721d272f84aac98bdb238de98af4cc767a69056d967d68c71/hf_xet-1.7.0.tar.gz", hash = "sha256:d406ec79053c0871817f700c2ac8c36ba0d87f9c34b7458b0f0063bb218b0466", size = 985689, upload-time = "2026-10-06T20:18:43.89Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/9f/7c/3e45174942e6793adde6cba4daa7fb037275cf02a944d9eadfcf9ff33b86/hf_xet-1.7.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:fa029678be1ba7f953c409b0b27bf15cc69cd1c9b3a674fbd78856ebefca1052", size = 3803919, upload-time = "2026-10-06T20:18:09.844Z" },+ { url = "https://files.pythonhosted.org/packages/ff/3a/5e8b363391adcbb002e191dbf924dab31464ea9c45adfeb73502afc36d35/hf_xet-1.7.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:57bc157b8b7fe3bee9dcb9af7f3da8de41801c3b31a9ef68a77a33c6a6be382f", size = 3553588, upload-time = "2026-10-06T20:18:13.376Z" },+ { url = "https://files.pythonhosted.org/packages/e5/c2/0d1eaa5da13bbf9c896badc7f380601c7d973a87a6ffb4d100267c4536c1/hf_xet-1.7.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:87dab080f8f7d32781c2586904e3603f4e60d09bfc727706c3ae419e0829beeb", size = 4201962, upload-time = "2026-10-06T20:18:16.11Z" },+ { url = "https://files.pythonhosted.org/packages/23/2d/225d5b11a9ca7d31b9470a57f2b2be1a5cef8b84325a2146aeb4589e226c/hf_xet-1.7.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:b01fe18dbbd151a2403d2c64ed30dc6547b00d6babab9a617d77c7acdb81ee66", size = 3982978, upload-time = "2026-10-06T20:18:18.092Z" },+ { url = "https://files.pythonhosted.org/packages/93/34/9d681f0e3dac0b5dae0d7dea748429266f24e52415446523f464fbaa828e/hf_xet-1.7.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:4ee5e05a627f5ab5bad7a86582277d645556ea1e199903aae19e033a392aa13a", size = 4181558, upload-time = "2026-10-06T20:18:20.082Z" },+ { url = "https://files.pythonhosted.org/packages/de/f0/277f039b7d72027bc2ed277f1b62a2f70f740a5aac2a3e7243e5b6854c5d/hf_xet-1.7.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:19c0e64f14175ccb6a1aff69e0d2ab9ec5269a560e6687abaf2b3fa4f73de7cd", size = 4411546, upload-time = "2026-10-06T20:18:21.999Z" },+ { url = "https://files.pythonhosted.org/packages/3d/7f/832d3ddb49326114175b7bcc50daea8565c09fd21ac03a02b211c09fefb7/hf_xet-1.7.0-cp314-cp314t-win_amd64.whl", hash = "sha256:757168feb5679647c0bb13ee5d0faebe799c4dff9051419885a566ebd79f949d", size = 3812809, upload-time = "2026-10-06T20:18:24.288Z" },+ { url = "https://files.pythonhosted.org/packages/3d/c4/310c3c29e5beae7c049e63947bd1923d597883b41c9ec4718589920812c4/hf_xet-1.7.0-cp314-cp314t-win_arm64.whl", hash = "sha256:b91569d5f1b61c34b043687da02c05dd3604f3d329e7868510bf3f7971599006", size = 3646174, upload-time = "2026-10-06T20:18:26.279Z" },+ { url = "https://files.pythonhosted.org/packages/9c/0b/b03be21ffaada749ba0d3197d8aefbf1aa698bac149580421c15239b299e/hf_xet-1.7.0-cp38-abi3-macosx_10_12_x86_64.whl", hash = "sha256:e3e88a7a75d7d95cbee1f37dc31341d6201124cf21c6c4b1dfab8ccba9b09e0f", size = 3796096, upload-time = "2026-10-06T20:18:28.43Z" },+ { url = "https://files.pythonhosted.org/packages/c3/47/a26ebdce7056a61e931f228439bc0ab08cbec239d1690f965e5e637cba79/hf_xet-1.7.0-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:59fba37039233c7fcbe196817d6cdcf1b40dfb17b410f229d85b0cf0a1848da4", size = 3560352, upload-time = "2026-10-06T20:18:30.365Z" },+ { url = "https://files.pythonhosted.org/packages/a3/4c/2bf3b66c215d409655f28de1622393dde04c9461280d48c7924bb3b2decd/hf_xet-1.7.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:2814a6e999d13464c4d679b788cc5d784eb5a4edfc638a31f10e9a11ab531ef8", size = 4212180, upload-time = "2026-10-06T20:18:32.292Z" },+ { url = "https://files.pythonhosted.org/packages/49/0c/a2f703a5a78267556e89e03316fa0805c86b72b50829bc67665746e8ebf0/hf_xet-1.7.0-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:fcfd6c22418e57dd5b3aea649e813b2e2cfb2aebf317b210d90f1fe4b3018b52", size = 3990011, upload-time = "2026-10-06T20:18:34.21Z" },+ { url = "https://files.pythonhosted.org/packages/a4/77/e52e4201b1cbf571530a61cc57f70182045a39a230089ee5f1df182a4de2/hf_xet-1.7.0-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:80f79dae613ce9e0ea1fd1ae15616ca9ac74aed4c770aabc199c4f03ebecc863", size = 4190628, upload-time = "2026-10-06T20:18:36.062Z" },+ { url = "https://files.pythonhosted.org/packages/6c/dc/03a21b89f118664a0926ff25b0f8e44a519bf22724a6a8fc7a9abbc188b6/hf_xet-1.7.0-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:0a9e802f33bf50c851abe45fc5380e61f959e2d369647d6742b79ad9d6c27cab", size = 4418814, upload-time = "2026-10-06T20:18:37.888Z" },+ { url = "https://files.pythonhosted.org/packages/4d/59/b35106dfa71b6eef605dc88bd038fe99c7f86fb132a15b60d0bf2f235b2c/hf_xet-1.7.0-cp38-abi3-win_amd64.whl", hash = "sha256:2b7bb5727889b0f2436dbaaad8fc4c3e66b8240d992716989e0c086b4278b1bc", size = 3822644, upload-time = "2026-10-06T20:18:40.052Z" },+ { url = "https://files.pythonhosted.org/packages/48/cd/072313585f74fe9d441e2eb5e0a4703c30586cd709810ea369675f61b74e/hf_xet-1.7.0-cp38-abi3-win_arm64.whl", hash = "sha256:acc3851cf2576a8fb2ae926da863f4efabe21303cf292e9a44332802ab0dcc6a", size = 3662436, upload-time = "2026-10-06T20:18:42.205Z" },+]++[[package]]+name = "httpcore2"+version = "2.13.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "h11" },+ { name = "truststore" },+]+sdist = { url = "https://files.pythonhosted.org/packages/cb/f3/1db7aa2bc2524062192bb0e0323969492d1883152a232fe36eea65f4e35c/httpcore2-2.13.1.tar.gz", hash = "sha256:e0aa977abe17e69a3b820a24542a6fa88702676d83880b8d194dcd18408e5103", size = 68071, upload-time = "2026-09-23T07:47:22.372Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/09/ba/a4568248771ce81957bfb7cc600264a40fbcda092391ee1c415c50be4bea/httpcore2-2.13.1-py3-none-any.whl", hash = "sha256:e1e05d4f25f7d7d496bfb96748f6f4b67657b03da069b3a68c36069f3db73d0a", size = 83423, upload-time = "2026-09-23T07:47:19.365Z" },+]++[[package]]+name = "httpx2"+version = "2.13.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "anyio", marker = "sys_platform != 'emscripten'" },+ { name = "httpcore2", marker = "sys_platform != 'emscripten'" },+ { name = "httpx2-jsfetch", marker = "python_full_version >= '3.12' and sys_platform == 'emscripten'" },+ { name = "idna" },+ { name = "truststore", marker = "sys_platform != 'emscripten'" },+ { name = "typing-extensions", marker = "python_full_version < '3.13'" },+]+sdist = { url = "https://files.pythonhosted.org/packages/d5/44/474bef2a0e9d90f1715d32cb98b0738695ca17ba324095fb2497ed7fbd59/httpx2-2.13.1.tar.gz", hash = "sha256:e48744a19e3af5ee48313d0ce5fe941d5422fae5705ea922a4aabf94d7800dfa", size = 100405, upload-time = "2026-09-23T07:47:23.052Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d8/9c/6fe8931fd9f381042a9e4c7d5a7b4cbf7016b252bec0c99a49fce42c3326/httpx2-2.13.1-py3-none-any.whl", hash = "sha256:6dff50fabc270ee5fd25d845d0b078ed20564579744d6d962850975996d2f9a4", size = 95597, upload-time = "2026-09-23T07:47:20.995Z" },+]++[[package]]+name = "httpx2-jsfetch"+version = "1.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/cd/c4/0e5636363151a2a1795e0a77617168b9ca438e1748ec05fc9b5687f93d64/httpx2_jsfetch-1.0.tar.gz", hash = "sha256:70a0e3eabfef7cce5ad9c629f7d01ca05e418f586646f4ddf14782e4c1454c60", size = 6872, upload-time = "2026-08-07T00:13:07.492Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/9b/43/832f631d32e4f1211caa2ba368317739fe71f0b8530e4c9d15dc454bac2a/httpx2_jsfetch-1.0-py3-none-any.whl", hash = "sha256:cb916b707601e69a07721aabc8f3f6659be3a6893bc1ff5c6f9e02241df2da32", size = 6382, upload-time = "2026-08-07T00:13:06.567Z" },+]++[[package]]+name = "huggingface-hub"+version = "2.2.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "click" },+ { name = "filelock" },+ { name = "fsspec" },+ { name = "hf-xet", marker = "platform_machine == 'AMD64' or platform_machine == 'ARM64' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'" },+ { name = "httpx2" },+ { name = "packaging" },+ { name = "pyyaml" },+ { name = "tqdm" },+ { name = "typing-extensions" },+]+sdist = { url = "https://files.pythonhosted.org/packages/12/47/6858d63643e66fb4f6585c3cfd4029c0b2bc1ae21688cee9b3335f20a10d/huggingface_hub-2.2.0.tar.gz", hash = "sha256:5d1b47537394e4215cb858aa12fd493d0f7ef7f58990f5dcd24bc173107b2871", size = 1041026, upload-time = "2026-10-08T15:30:59.971Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/eb/b0/0f7b430fd100b3a3b037fdbb314878200241082e607b3383c63d91a13a72/huggingface_hub-2.2.0-py3-none-any.whl", hash = "sha256:1667f145dc56dc210d60966069397df9ecfca9607a5d43db88b308c89dae56b3", size = 839884, upload-time = "2026-10-08T15:30:57.914Z" },+]++[[package]]+name = "idna"+version = "3.20"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/f5/08/8eea9d4b8302028f3abb2c0813953f7aec26d33b7a8960ed760e65ff29fa/idna-3.20.tar.gz", hash = "sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44", size = 216463, upload-time = "2026-09-17T14:11:04.752Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/58/a2/bb081bab032533a855d44de1d56f8e8426114ff1ba5d1f07a438a0a654f8/idna-3.20-py3-none-any.whl", hash = "sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c", size = 69583, upload-time = "2026-09-17T14:11:03.168Z" },+]++[[package]]+name = "iniconfig"+version = "2.3.1"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/01/e1/2069291243c926a2ff1cd706c7f3eeb9b62144bf60f77c9fb9ff2fb26bd3/iniconfig-2.3.1.tar.gz", hash = "sha256:67f4b9c50da0dedf52af349e7749a80a9057a5031199791b906c3bb3ae878960", size = 21209, upload-time = "2026-10-06T22:48:38.076Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/56/43/4ca9e49d27a1fcf6bece6f6aec0ea46bb9112489b93d4b688fb415457bdb/iniconfig-2.3.1-py3-none-any.whl", hash = "sha256:9121e2c1fdb355232495be3194c8dfe87ccc2d5dee45947b78e68f499790d7a7", size = 7552, upload-time = "2026-10-06T22:48:36.959Z" },+]++[[package]]+name = "jinja2"+version = "3.1.6"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "markupsafe" },+]+sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },+]++[[package]]+name = "markdown-it-py"+version = "4.2.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "mdurl" },+]+sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" },+]++[[package]]+name = "markupsafe"+version = "3.0.4"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/38/9b/e422a865e1d5d57d0e509b4e0bf1c1a70a7f6382c29a5aa428df994c8bc8/markupsafe-3.0.4.tar.gz", hash = "sha256:2e9ad7dd851bf45fab9f75cbff4cb493fee9979e8d8c7c9c3ee119022518edd6", size = 153777, upload-time = "2026-10-02T23:07:22.29Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/32/55/18dbb4778b30ada5ce071608503cc3edc9e14e13d868c17a6d178fc30f7a/markupsafe-3.0.4-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:9e25feb9e330b63edb0278a0acdf85e50d0cb0fbf49c3084abbe4e24ae195346", size = 11520, upload-time = "2026-10-02T23:04:36.299Z" },+ { url = "https://files.pythonhosted.org/packages/6c/14/0b05f79b4733e264a18d08fe08fa1df7347630ff32a6cb82180d9dccec55/markupsafe-3.0.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:7d3391b2188d18737cb2fa147028b1096236eaa7e156446c650a489fa2cadc91", size = 12063, upload-time = "2026-10-02T23:04:37.291Z" },+ { url = "https://files.pythonhosted.org/packages/ca/3a/63ba10b6c1463216b3e4df669a9f0e5a3b0c3071557d2e8229e3968c79fb/markupsafe-3.0.4-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:849dd2bb0e5e4ab2b71c7191726a4a8d5aa8a610daa584728cbee0b710ddc4ef", size = 24321, upload-time = "2026-10-02T23:04:38.262Z" },+ { url = "https://files.pythonhosted.org/packages/1a/2e/5f015261b76ad633d187ef6f388b413aedd64a8773c4df59e530a0be5525/markupsafe-3.0.4-cp311-cp311-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:befb4158af32106b9a93db8d6d1d1cbbd418c0d5aca0cabb7b1780abf0c89169", size = 28845, upload-time = "2026-10-02T23:04:39.295Z" },+ { url = "https://files.pythonhosted.org/packages/33/cf/26e594b26be40c2f1fec63ccf8a8b99d0335a5b2cbe84835c7d82a994375/markupsafe-3.0.4-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:71f88e749ea29f67f21f3b36433c1dc54c7729ed2a6d9e2da2e0d9e0d7b224eb", size = 24701, upload-time = "2026-10-02T23:04:40.348Z" },+ { url = "https://files.pythonhosted.org/packages/81/a5/a513b76c139a3915b43404324e55c0b7979ae4f0d39eb6f075b0282e90a8/markupsafe-3.0.4-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6da83a088f8ef93b2d483a8232a4dbf4d69d3d8496b568a03c56becac43e1808", size = 22961, upload-time = "2026-10-02T23:04:41.372Z" },+ { url = "https://files.pythonhosted.org/packages/46/cf/4c66192c100b4542bcbe392ae06696b670f66927be3ac38a213234778ff9/markupsafe-3.0.4-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:8f0fac8b13d14bb06c68195f849371924ae53dd7b1c00fed24650f704383b692", size = 21920, upload-time = "2026-10-02T23:04:42.429Z" },+ { url = "https://files.pythonhosted.org/packages/cb/17/ac3662678bfbad649893117ada2ba44dc30bf56884e84f13154a792b10f1/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:4a7cdc2a420ca01058182da4253329764d4bfa055564d1eced90e6ba1e8b1d3d", size = 23726, upload-time = "2026-10-02T23:04:43.531Z" },+ { url = "https://files.pythonhosted.org/packages/f7/af/fe47cee339180a69ebca3c57fb3483d0f5cbd1e8337d1871fb1d9c1aebee/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:83b3944fea42a8400edf92fd1770fb8d0d4f7de651353bd2d8525a92dba69a21", size = 22539, upload-time = "2026-10-02T23:04:44.618Z" },+ { url = "https://files.pythonhosted.org/packages/12/32/d55440ba140442800e02d799c9cb5ab597bf6ebdb1177b5ea39a11f797bd/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:8138eb83940ec7299024d92d4dee45f601b9e6c5ffde9d25f4e35e326203c707", size = 23963, upload-time = "2026-10-02T23:04:45.656Z" },+ { url = "https://files.pythonhosted.org/packages/50/9d/9c86042cb364c2ad4c971e6d1247929effd25f714cd7ee11b05e6316445b/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:811d02d5122171c1941357efd8f9bf4ffe907b7f0a1a4e729a880e4be3f46e3e", size = 21506, upload-time = "2026-10-02T23:04:46.671Z" },+ { url = "https://files.pythonhosted.org/packages/75/ef/5b824f03ba40c3b3652b6272d083d2fc4fcdd440de519a3a39ba2c3e7262/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:50b5bedc9ed8a94fc8857a42ef4f84a81ea88f8d4f05dc8705fb23ee6d8dcca7", size = 22955, upload-time = "2026-10-02T23:04:47.698Z" },+ { url = "https://files.pythonhosted.org/packages/1e/e8/44cfcb5ea40e5e43cec7793ef90704ed0c475280839076c4345757eb8e59/markupsafe-3.0.4-cp311-cp311-win32.whl", hash = "sha256:2e5a7cd7fdd14fcb1ae5d7d8bf23d24fbd1daefd1fbca2580132e1ea75f098b5", size = 14094, upload-time = "2026-10-02T23:04:48.788Z" },+ { url = "https://files.pythonhosted.org/packages/91/89/f2b509f7bf79352e40117824c1070dbeafd4df67031d3fa98165a3134228/markupsafe-3.0.4-cp311-cp311-win_amd64.whl", hash = "sha256:fdb4ca07ab75ffadab4a8b135ad59cdbb3156b99310f3d565370da74a15d6bd3", size = 14328, upload-time = "2026-10-02T23:04:49.801Z" },+ { url = "https://files.pythonhosted.org/packages/2b/5a/ccf22672a0f64dc682306e288f0dabcb06c7a201f3bb6e6cbf86d9e8ad03/markupsafe-3.0.4-cp311-cp311-win_arm64.whl", hash = "sha256:569d65055d367e3dcdf30c3f41119467b73d9ee9faf332bdf40402644f5ac08e", size = 14188, upload-time = "2026-10-02T23:04:50.847Z" },+ { url = "https://files.pythonhosted.org/packages/81/09/4c59d56b8461ae8eb0d8ba34bb25b7e618547044679d58a82ef9b2479fc1/markupsafe-3.0.4-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:61631e08084be9e21a8967ec3139c7616ed7c5e9368e05c86d1b39562c8a57b6", size = 11658, upload-time = "2026-10-02T23:04:51.876Z" },+ { url = "https://files.pythonhosted.org/packages/a2/f0/d6613774d86fbf6d145751d43c59875e47a6f9f17daee0aef173bd36d90e/markupsafe-3.0.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0930db9bdc62d22944e10b066448bb65dc9abe9112880c7cab8da54db4284d5f", size = 12050, upload-time = "2026-10-02T23:04:52.931Z" },+ { url = "https://files.pythonhosted.org/packages/0d/f2/8f18e0b806eb13c1f8d07d917a720831ead54253a6dec011fbc78098a6f8/markupsafe-3.0.4-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6a45c3d514f2436064db00d7fc8778d888f0236ebfed649b53d13a59e69ad51b", size = 24363, upload-time = "2026-10-02T23:04:53.895Z" },+ { url = "https://files.pythonhosted.org/packages/60/ce/fa07dbe8a5675558fa36dea033e19995bc783de2dec5f540ccb9030b06aa/markupsafe-3.0.4-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:1e1451fab512d1bcc3dc26988ec1edb0b82c2db909132872cd9356070a6b63df", size = 28525, upload-time = "2026-10-02T23:04:54.905Z" },+ { url = "https://files.pythonhosted.org/packages/85/40/be87c01f3868ec217f8a2015089d71c22c8c5a75324822e5ed1cdd87210d/markupsafe-3.0.4-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bd3ce56ae2cbae3ba82b683bc425cd7e48d2ed8b10f3e818186b6f5646d9271c", size = 24733, upload-time = "2026-10-02T23:04:56.229Z" },+ { url = "https://files.pythonhosted.org/packages/4f/a7/aeedb5140afa41fc74c225e9184ab96723a6e873b6ee1c9fede7283456d8/markupsafe-3.0.4-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8e124f974786f831d6043728e38296969d3579db8896fe004682f5758e613581", size = 22985, upload-time = "2026-10-02T23:04:57.521Z" },+ { url = "https://files.pythonhosted.org/packages/c3/fc/e91352bb08c6a59da3ef0909d457bf95a5f5908fbf151b30a06d9dbcfbb4/markupsafe-3.0.4-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c02e8f18bdedba082cef725942ac823b9b60656db07f7e265cb31618dfd00d77", size = 22001, upload-time = "2026-10-02T23:04:58.597Z" },+ { url = "https://files.pythonhosted.org/packages/5d/f8/bffee5e7d2a3deb59748a797650a48af7e672025cf641a79344a771ad106/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:9f098115c247e11d138ab83a28fa0323c77015007ea2df73ba5fd714dfefd67c", size = 23793, upload-time = "2026-10-02T23:04:59.686Z" },+ { url = "https://files.pythonhosted.org/packages/ed/59/b853d6628ecb4d658e1d637224846d5e9bb4adf4f8df97f3be9f29dce2ec/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:d5f93ebbeb8032d47e349328ec8662d973d9b05a70b3c35df1f91fe419b84749", size = 22640, upload-time = "2026-10-02T23:05:00.768Z" },+ { url = "https://files.pythonhosted.org/packages/09/b2/1506df394f0f075797c418d0301498f49e43be194e3ffcb49e6fe6ccf022/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:64511c54db4e4987aef4c41923235927428729e8174c5dba488429be70a998ed", size = 24074, upload-time = "2026-10-02T23:05:01.813Z" },+ { url = "https://files.pythonhosted.org/packages/c7/81/5ed69cda630ac69ef60d06c09ba5a7f84ff66a2e28cf986fd5614ab3c6e6/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:e1a622f13970d81f95d0c72f9dc090dce9085fccfa4c9f2174377ee32bd15786", size = 21563, upload-time = "2026-10-02T23:05:03.239Z" },+ { url = "https://files.pythonhosted.org/packages/0c/fe/fb1e79be0fea60aa32602ebefc9c35a82bb42b4df157285ab7dfec12341a/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:c9a7f43c0b202b334cc9184af09bb8f21d3a209e038efaf106936fb69e6b026e", size = 23048, upload-time = "2026-10-02T23:05:04.479Z" },+ { url = "https://files.pythonhosted.org/packages/c8/52/7632a53360671a9b750cdbabaf9cdd89f18b42248b8e4cb42c0b0296e459/markupsafe-3.0.4-cp312-cp312-win32.whl", hash = "sha256:f0ec3b750b59375eab5b0fb2b9254810c00a3375be6d789899f1055a1d556237", size = 14108, upload-time = "2026-10-02T23:05:05.513Z" },+ { url = "https://files.pythonhosted.org/packages/3f/bf/62495e180b7000aaf30000fff849e933f74264638057176cf46852500adc/markupsafe-3.0.4-cp312-cp312-win_amd64.whl", hash = "sha256:11935df9bf455ed0c04eb87bcd720f02b1fe5e02128a9430f23aed6f93336fc7", size = 14303, upload-time = "2026-10-02T23:05:06.538Z" },+ { url = "https://files.pythonhosted.org/packages/c5/8e/4c24208776a65878d656996945aacfbfe010d3720d1a98fc0eb8491fc03b/markupsafe-3.0.4-cp312-cp312-win_arm64.whl", hash = "sha256:a4bbd2d87dd233b9fc5812160c3d0ffbe42edc22a26ce0469f58479ede633fe9", size = 14174, upload-time = "2026-10-02T23:05:07.617Z" },+ { url = "https://files.pythonhosted.org/packages/6d/18/4bc5ba32499e87bb2b0ef5b3a9bb9c00a131fa961ddf0be548cb550f548b/markupsafe-3.0.4-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:de8b364c423ef0a4bad9069657d617f9a5d2b2062457a89b1fa16ee199c399c1", size = 12786, upload-time = "2026-10-02T23:05:08.709Z" },+ { url = "https://files.pythonhosted.org/packages/4e/6f/17f0c099bf25f3e31e63cc19244d9f6af861a9a4ab778c203997903cfdd0/markupsafe-3.0.4-cp313-cp313-android_24_x86_64.whl", hash = "sha256:34bdde374c5932765d7dc685c4a1d191a3207852d67e8e0a9eb6ea85156181f1", size = 12510, upload-time = "2026-10-02T23:05:09.93Z" },+ { url = "https://files.pythonhosted.org/packages/11/af/1a141081b905036ee904ec4bd945e1f70b4e1b32d33c4e59e8cf1d58b247/markupsafe-3.0.4-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:6bd9e1788e15bfcf6a9082de42e30387e7b85d211ab21e57a939bb8cfaaf8d96", size = 11583, upload-time = "2026-10-02T23:05:10.884Z" },+ { url = "https://files.pythonhosted.org/packages/e7/0a/a89385ae590232622a03e091805cff12f24fabe6c11e0e8bae096cece81c/markupsafe-3.0.4-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:5066b244f576f91afc8ee3ba029a89f99d39c79b1853fe9d39bea9f0afbec148", size = 11801, upload-time = "2026-10-02T23:05:11.913Z" },+ { url = "https://files.pythonhosted.org/packages/ed/85/ea548dc013962eb73653124bc595635fbf9e0fa41d1f181a967ccb784dfb/markupsafe-3.0.4-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:7a83aa6e4805df46fed18e989d3d16f86ef60cb50bbc8d9ce3a6be89165fbf6e", size = 11443, upload-time = "2026-10-02T23:05:12.887Z" },+ { url = "https://files.pythonhosted.org/packages/cc/72/15f2e5ec9cf2eb00d5cdfe968d94e4156a7bd7303832c3f3b2c403a36839/markupsafe-3.0.4-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:2d1b7d9308288661f56672b1b157d75fc536714d3638487bbea17b6318a78248", size = 11665, upload-time = "2026-10-02T23:05:13.829Z" },+ { url = "https://files.pythonhosted.org/packages/ca/e0/4030bea613677e333c8a2c901fd405055f657f9d06acba5b7357984b6ef7/markupsafe-3.0.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:73e77980c7207854f00fc4e71fb1626868d5740ab4012623d55c7a99ad122a72", size = 12061, upload-time = "2026-10-02T23:05:14.807Z" },+ { url = "https://files.pythonhosted.org/packages/f3/a5/28b76a7449eb702966b88bef599e2360b411fbb3afeee8fe560939be06ec/markupsafe-3.0.4-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7018d4af1cd272e847aa5917983ab5e83e4f6579f9dbfecd4a79c0ca80b144c2", size = 24395, upload-time = "2026-10-02T23:05:15.909Z" },+ { url = "https://files.pythonhosted.org/packages/07/6c/21232811afc3a063b5e934b1ae2efda52f46154ec382f585149c020e61fe/markupsafe-3.0.4-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:c90d5b3d4e944e065a301d741b3c1d784f6bd1f503aa68b4967e32b2ba313d85", size = 27183, upload-time = "2026-10-02T23:05:16.976Z" },+ { url = "https://files.pythonhosted.org/packages/14/38/6ccdfa5b59049cb36fb80cbc80aee9cf1fc9bb77d1335ad435f2070b08cf/markupsafe-3.0.4-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:18a801868a884f216e784d7d14db2a4077143ce7610440aee2ce8f734e7cfcde", size = 24772, upload-time = "2026-10-02T23:05:18.209Z" },+ { url = "https://files.pythonhosted.org/packages/63/e0/cec6865dfe88cb48fedd4b20aed6af5158e41092adcbf3e028bcc6ec2108/markupsafe-3.0.4-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:434139499bb20b502ed3baa1f169e618f924a97e7a777fea1a49446d80106cf6", size = 23017, upload-time = "2026-10-02T23:05:19.286Z" },+ { url = "https://files.pythonhosted.org/packages/ee/76/6ed4940bb7648a9aac457c14f870cfdd5105f139a0fb1f29cd61fafa47d1/markupsafe-3.0.4-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9e227f3dbe6bde7491cf0a9965d00b88c6b1a4a95d11480ddf88bb96d397c19f", size = 22036, upload-time = "2026-10-02T23:05:20.352Z" },+ { url = "https://files.pythonhosted.org/packages/a1/4f/ed476226d4fe46a09090a36025bf319296810028df55eb12f1253b540f3a/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:b8cd1f918b26fd7b1832ece557cc18f2d8747309ff8b3f0ef9d4250c5ad67a39", size = 23786, upload-time = "2026-10-02T23:05:21.576Z" },+ { url = "https://files.pythonhosted.org/packages/9a/35/66ff30450e35ef5fba9ebc930c9411747e537fd9447b65e44f5007e2b84d/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:a5fcffb37e602b0b3c1638a97746b9b96125caa9bcf6fa41d337a9261de231ee", size = 22672, upload-time = "2026-10-02T23:05:22.922Z" },+ { url = "https://files.pythonhosted.org/packages/32/0b/72f45ce4b4efcbca4b80cf1b06703eff0be8d37e82abb78f66c85a7ead1e/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:5989cb26b2e1efc6a42216a9f6b5ee495ce5ace2e5b352a9af489976b32d1ee2", size = 24107, upload-time = "2026-10-02T23:05:24.175Z" },+ { url = "https://files.pythonhosted.org/packages/d2/03/71776e5fdcba04614b384cc102e8a4198208579d896fd1394cb7cb9aa900/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:add96447a86d205ab616665d53b2950ee81083757f56e6ea833c8b2917646b46", size = 21615, upload-time = "2026-10-02T23:05:25.215Z" },+ { url = "https://files.pythonhosted.org/packages/ab/5f/801ce02a02e7aee0f784b1ec7843026178f6adeb9c93ac67eb1992a9a84d/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:2628d3a8cb648ecebb3c5d6b0a1052d400e4d8b7ac0fb786be8d285b50040d17", size = 23076, upload-time = "2026-10-02T23:05:26.423Z" },+ { url = "https://files.pythonhosted.org/packages/4a/85/c43776625428f3bb4a61e8633940400e3efe6409e3c6f5bff26de5e45618/markupsafe-3.0.4-cp313-cp313-win32.whl", hash = "sha256:672d207103e6b16ca098611b0f9efad6bc00afd47c03d6ef62186495ca677dc0", size = 14118, upload-time = "2026-10-02T23:05:27.716Z" },+ { url = "https://files.pythonhosted.org/packages/6f/36/163da64de88a13db79214ef75fa041be7fa13bdb42261cf5b7484de14bfb/markupsafe-3.0.4-cp313-cp313-win_amd64.whl", hash = "sha256:1f1f9477e174582b0a1b583d60b66e1f2cf5d3fe12cee985e4aedf44766600e5", size = 14313, upload-time = "2026-10-02T23:05:28.749Z" },+ { url = "https://files.pythonhosted.org/packages/9f/a8/9b662783ffaa1149221432a923cee562f78b9cbbb8baa3df9b3753e63e1e/markupsafe-3.0.4-cp313-cp313-win_arm64.whl", hash = "sha256:06de8ef6331f6e822c28d577dc8bf43fe398800477c49498f38fc38b67ff33fc", size = 14189, upload-time = "2026-10-02T23:05:29.917Z" },+ { url = "https://files.pythonhosted.org/packages/5c/c3/a944f3b0df22bd129e96915b9f4e98d2eeca6516687d7618304a966c3c74/markupsafe-3.0.4-cp314-cp314-android_24_arm64_v8a.whl", hash = "sha256:4ed644d75aa94a2baf7ec3a96eaa160ea58c742eb9d27c6506053c5c40fc84ed", size = 12787, upload-time = "2026-10-02T23:05:30.971Z" },+ { url = "https://files.pythonhosted.org/packages/d4/d6/a44863f69d88b6c7e27889108f70d47aed259edf89d5df3c5fca1eac87d6/markupsafe-3.0.4-cp314-cp314-android_24_x86_64.whl", hash = "sha256:6d2a9efe686f9de00d0d1ea32a4a5a86d558a2277501bd78d964214eab625e59", size = 12525, upload-time = "2026-10-02T23:05:32.263Z" },+ { url = "https://files.pythonhosted.org/packages/17/8f/168ba80e532dd6a93f96f8f706f1ad41d7990b6e1aeedc1cc0d211a33497/markupsafe-3.0.4-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:8781a792a070cf2bd1b86d3aa943894115faaba6e88122a7bf32d62072742453", size = 11589, upload-time = "2026-10-02T23:05:33.251Z" },+ { url = "https://files.pythonhosted.org/packages/32/b3/aa2c95a574d3af39403a469b295886eb9b6d448da568cbebb5a2cbfdc2e5/markupsafe-3.0.4-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:971a3bbb75d97ae4e2e8f7d4834236f86f85f0c85e04ab2e191db1123b04f80b", size = 11801, upload-time = "2026-10-02T23:05:34.315Z" },+ { url = "https://files.pythonhosted.org/packages/60/d0/34b810107d83840e768bf485de795893ebbae35b26ab061b487adfa0a692/markupsafe-3.0.4-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:8909c2f1c6dd65e054ac4b573a91c8384d1492281e55d82d159d653f7a13adf6", size = 11447, upload-time = "2026-10-02T23:05:35.302Z" },+ { url = "https://files.pythonhosted.org/packages/6c/ab/2f8488f0f817a39fca068d2b17daf446bf5cdb3eae28c3720af534d873b4/markupsafe-3.0.4-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:4cf3468d5ec187ffffcaca8e61929a37448f215dafc1386a12c750a72fe53634", size = 11722, upload-time = "2026-10-02T23:05:36.363Z" },+ { url = "https://files.pythonhosted.org/packages/ad/40/e2d117b048d47282ade906fbfd92814cbee5647afc13fda88a3406039372/markupsafe-3.0.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:52704c5d36eb6dda8866493decd61111fff86244c9b1ad225ca01b9e91e5970f", size = 12061, upload-time = "2026-10-02T23:05:37.397Z" },+ { url = "https://files.pythonhosted.org/packages/9a/a8/73a81135e85ba66217f5af7facb03bbb386807e1a729ab64532e4c802652/markupsafe-3.0.4-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1caa2fa5a6184fb233153b35f654e6687bd555476f6170f29d8ee9be1a8b0af9", size = 24434, upload-time = "2026-10-02T23:05:38.407Z" },+ { url = "https://files.pythonhosted.org/packages/ac/ca/fa9216dd01efee2dfdacafe7df32b4d0170fbac694b0c258a193d6e53999/markupsafe-3.0.4-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:387d8cd30e69b3f0a72877b9ae717033396404e19095b17fe89753a981fda44f", size = 27183, upload-time = "2026-10-02T23:05:39.581Z" },+ { url = "https://files.pythonhosted.org/packages/fa/4e/a469509e538d37af51103b17b073126973f2b1cbf197ff32c7ddf025cfe5/markupsafe-3.0.4-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:051417f74bcaaefa316276e0ff723f541616ca51043d070da00249d9bddd3e3c", size = 24813, upload-time = "2026-10-02T23:05:40.671Z" },+ { url = "https://files.pythonhosted.org/packages/8f/db/d7282caf7ab03af44d5d6fdbaa019b35c7d7f1c90588b839c07cba640d6a/markupsafe-3.0.4-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a8e9f292fcda89b324f2f5c91d13f1424a153e40fc2756f38ee23b15835ff300", size = 23049, upload-time = "2026-10-02T23:05:41.864Z" },+ { url = "https://files.pythonhosted.org/packages/30/f3/b6a425206e6964efda6acee544d0eb01d1501784d0b8e2dcc74986f33b17/markupsafe-3.0.4-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:df1ae86ff54725a01fa1a0510b914ca53a161b7050be74f6204e24aded5971d0", size = 22088, upload-time = "2026-10-02T23:05:43.014Z" },+ { url = "https://files.pythonhosted.org/packages/ea/8a/84d3582fc1f0d5bd466cdf2eebf175e172158a6e70701aacec1de1b35430/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:8965520ac587c94a4ac48b729be3d8b8de00af39699b17585dfb599babe77977", size = 23846, upload-time = "2026-10-02T23:05:44.098Z" },+ { url = "https://files.pythonhosted.org/packages/1c/65/db101cce51b7ba4864ac491a9859d297dd1adf0e55b103fee9db9c47c527/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:340cbb1957ba99929cbf19a75626d36ba1ae21d1730b287d1cf7f824a20c4fc7", size = 22674, upload-time = "2026-10-02T23:05:45.23Z" },+ { url = "https://files.pythonhosted.org/packages/e0/49/ddee9813d71db0c7a5c9d97c832125e6758a0c844777f1cf076569bb0e22/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:3a93d9616ddecfb393727a0041a562cf0b15a244e20f2bd25efc7949be4c4f17", size = 24104, upload-time = "2026-10-02T23:05:46.398Z" },+ { url = "https://files.pythonhosted.org/packages/aa/0e/7d8518d726726870a2399d69fd30d0fa36c5e57a2132c336b58d7c491073/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:d2e56fd3b00222722abfb3f5f0759ddbae4b90811b5ad4343c64030ad1bde70c", size = 21637, upload-time = "2026-10-02T23:05:47.48Z" },+ { url = "https://files.pythonhosted.org/packages/b4/b0/b505e8a361ba557dbf3b3aa7331ea39b00d2022a26e925ff8463b9714bb3/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:0d9c47709875fdb321452056622e930c52afbc07a7d780762fbb8b4d91ce6fa4", size = 23081, upload-time = "2026-10-02T23:05:48.611Z" },+ { url = "https://files.pythonhosted.org/packages/1c/ea/9cc3cea873f980c75cbdb6f4277ce30ee955de38be0b3d02f14c108e0698/markupsafe-3.0.4-cp314-cp314-win32.whl", hash = "sha256:38fc55594dab834470b6733dead2ee9e3f657fb0608c769dcafa0ba5ab52f45c", size = 14272, upload-time = "2026-10-02T23:05:49.707Z" },+ { url = "https://files.pythonhosted.org/packages/80/f0/5792ff768a410f93ee3f84fc19345295ffc352d2c936b424cb37e514714c/markupsafe-3.0.4-cp314-cp314-win_amd64.whl", hash = "sha256:c1bc67752d5f21013cfe430df4062441714eab79f65a6a05e01505957e9c35fe", size = 14501, upload-time = "2026-10-02T23:05:50.788Z" },+ { url = "https://files.pythonhosted.org/packages/5f/cf/3d074a8edffcc6899355232ff2543ae8d929733239596423b7db79698bc9/markupsafe-3.0.4-cp314-cp314-win_arm64.whl", hash = "sha256:7e1636da3d8dfc220b6dd10264db5f2b165e4888c4518594898fbe381049af8a", size = 14364, upload-time = "2026-10-02T23:05:51.857Z" },+ { url = "https://files.pythonhosted.org/packages/d9/31/87ce42159aae2163cf3bbbd0c44bc87780510eecab1ea3859099aed95dcb/markupsafe-3.0.4-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:805c8b84534fa10891890f0e4be39f3a99e94615d93e8836bf9fa1fdca2feeb2", size = 11759, upload-time = "2026-10-02T23:05:52.951Z" },+ { url = "https://files.pythonhosted.org/packages/5f/53/b047207eeb7752e960aca3eb1df5fb7eefa7dd4c62ac49bb156456c8a702/markupsafe-3.0.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:fa95848c929b6a75f6848d3c9793e59db365ee436776e57db835cdbfa79ba977", size = 12096, upload-time = "2026-10-02T23:05:54.066Z" },+ { url = "https://files.pythonhosted.org/packages/ee/51/4326c88a13c7b755657d44b4bb986f8c3d9843ecba7e22d98661d87f9a57/markupsafe-3.0.4-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e916035e3e9930cbdfdd10abf48861340221857f45509565898e012263f7b289", size = 25699, upload-time = "2026-10-02T23:05:55.15Z" },+ { url = "https://files.pythonhosted.org/packages/f2/bb/990581b7474bfcf2cf34bed6ba5ea23bd87adb9d671213d68e88620e7a6b/markupsafe-3.0.4-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:b4d12837e0203bbace818ff4a7461afdcd78bcd782351cea148139180d7bcffe", size = 27804, upload-time = "2026-10-02T23:05:56.29Z" },+ { url = "https://files.pythonhosted.org/packages/6b/89/89491878c28e8291f5aa2fffe2c2d57230d10ae366d55dd810b840513d78/markupsafe-3.0.4-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:5086f9975abb1ab531ee6afca1761e4b59a19b446f3f6522ed776963228cfe5a", size = 25912, upload-time = "2026-10-02T23:05:57.416Z" },+ { url = "https://files.pythonhosted.org/packages/30/77/680998b54efdea06fc114565cd739b6d059f826a0279219b218dfa750d29/markupsafe-3.0.4-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b4a635a0487774f841cb1fb62e907e7195cc95bc761e053184b8acc3ceb20733", size = 23623, upload-time = "2026-10-02T23:05:58.557Z" },+ { url = "https://files.pythonhosted.org/packages/ae/75/2709f5ac5de9467b40b10e2bb8f89cc63dfb74582e09aa734b1124a217de/markupsafe-3.0.4-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:cb96e6e088d6cf71c1ea977510948320234824cf226e32f6f6e044f7a9c82b34", size = 23301, upload-time = "2026-10-02T23:05:59.94Z" },+ { url = "https://files.pythonhosted.org/packages/a0/c8/39eadc6c5b14c9c7679bfb98f4d4c6a97863b5beb91839aca4d2d6e16e55/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8b5d563170ff8ba3181caa967c99a3c804d1dedb702c7cb93a6a7c32247da978", size = 24950, upload-time = "2026-10-02T23:06:01.289Z" },+ { url = "https://files.pythonhosted.org/packages/1a/5e/01037f8a43e8ccb0bffb4fbdc5212db05bf080fdd7286cd392332d58128a/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:396ec4e65cc889f69786b3b89478b471cee5a3bcf468b9d9bb03e1a30fb291fc", size = 23330, upload-time = "2026-10-02T23:06:02.441Z" },+ { url = "https://files.pythonhosted.org/packages/d4/f4/23e83ce0596bb0cbe670502d31df8f757bbd01a392aa486fa3b40d1ed399/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:15ba9e28640feef770374b116a6f019c21f52404aeabe516aa7f800587b98cfc", size = 25239, upload-time = "2026-10-02T23:06:03.579Z" },+ { url = "https://files.pythonhosted.org/packages/88/5b/3708897368073cc683d524750474f41a77d2986152c380dcc55b20fdf340/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:d920abdfa61279ba1a2ef9484aab07bf03331f8c08a10120fa332353d06e6932", size = 22767, upload-time = "2026-10-02T23:06:04.699Z" },+ { url = "https://files.pythonhosted.org/packages/c6/61/ebda1307864b409e6b3115757a3d4a09cca46cfb6cc65191b5de226b424b/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:a9f54054101545a9a9cccefddf54316aa6e4491611fcbef9e91b3b6bebec04f6", size = 23668, upload-time = "2026-10-02T23:06:05.9Z" },+ { url = "https://files.pythonhosted.org/packages/09/15/98075cceac3b5ba0dbb8e4762a847be967d2befc349a2cf2d0ac77f62c9d/markupsafe-3.0.4-cp314-cp314t-win32.whl", hash = "sha256:12a606a492de952afcb43b59a14aaaaad120e708d3663dd0fdf2d738d427a691", size = 14301, upload-time = "2026-10-02T23:06:07.109Z" },+ { url = "https://files.pythonhosted.org/packages/0b/a3/768b560fcc4156685cb563d922b217810cfa7bc135773367f62f1f9d2078/markupsafe-3.0.4-cp314-cp314t-win_amd64.whl", hash = "sha256:a18f38cafc329bac5e3c2b96c765b4c96d3d103421ed22ab7988c1e3fce27464", size = 14526, upload-time = "2026-10-02T23:06:08.276Z" },+ { url = "https://files.pythonhosted.org/packages/93/63/da554b4c97a6b0ea3229ca7fe8cbfb620be81613d517f482e85958550537/markupsafe-3.0.4-cp314-cp314t-win_arm64.whl", hash = "sha256:eba154571c16e032112afac0dc2dfe9e63c2ceb7aedd07bb7eecf2ce26d4dd4c", size = 14421, upload-time = "2026-10-02T23:06:09.402Z" },+ { url = "https://files.pythonhosted.org/packages/a9/30/54d11c8ca027114898cab97421fb39e4ffd9ddf47cdbc44df2ec76722da9/markupsafe-3.0.4-cp315-cp315-android_24_arm64_v8a.whl", hash = "sha256:737c9c3981998eba27f11786f84fddcbabc74068b72a4a1f454ea02094b57b65", size = 12791, upload-time = "2026-10-02T23:06:10.485Z" },+ { url = "https://files.pythonhosted.org/packages/10/6d/97c913e253a14bd3cd0e15a5c56d13203b823fa7ee32498342896a072dc4/markupsafe-3.0.4-cp315-cp315-android_24_x86_64.whl", hash = "sha256:489505b03f692c3f376394e49194fa7a7f9e8558d6e293a7056a0032b0c38163", size = 12527, upload-time = "2026-10-02T23:06:11.834Z" },+ { url = "https://files.pythonhosted.org/packages/26/f9/b86d032042a4d597d9e1997f0e5f63a3eedaf11258e0a05760b0a0a826ea/markupsafe-3.0.4-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:077293e425f28ec737dbcad442a71752e28f8ae27cde3d68acd1fb212091cd92", size = 11602, upload-time = "2026-10-02T23:06:13.122Z" },+ { url = "https://files.pythonhosted.org/packages/f2/dc/73c14c1eedf0ac5fa3292ba43435e6c49d2c2050f33cebde541f8f4807f1/markupsafe-3.0.4-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:9348cbb300d224fe3b89793262cb093504d4ae927004468463f745188a193e4a", size = 11817, upload-time = "2026-10-02T23:06:14.227Z" },+ { url = "https://files.pythonhosted.org/packages/8f/69/2c2fcaa5fcee22d72c7819c0d536fd181c74a688e6143845419579cd2863/markupsafe-3.0.4-cp315-cp315-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:b807e598953730f82e4eae3bd30f6a122cf6b31c398c6b504c0e04c13c170429", size = 11446, upload-time = "2026-10-02T23:06:15.574Z" },+ { url = "https://files.pythonhosted.org/packages/88/54/9e5ec76c62e6e2834d5a93623018c943e8b3bb41d663e3fd4c03303b9b85/markupsafe-3.0.4-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:799c39bdf5e2f1292fedd3009f7b3c9e760f10b2420cb9638d56920840ff6db8", size = 11719, upload-time = "2026-10-02T23:06:16.701Z" },+ { url = "https://files.pythonhosted.org/packages/96/24/3ec292b44064c16229e064d770b2625bd8ea941aa61f44905a9fa44942c0/markupsafe-3.0.4-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:ae9dcb8fbe244cb82f8a6458b455b927a03685e383d9bacf1ea5ce180b96dc97", size = 12075, upload-time = "2026-10-02T23:06:17.855Z" },+ { url = "https://files.pythonhosted.org/packages/aa/85/b64fdb1f304848518742136983c24e96d967bfb59a0ea160e92736901ab0/markupsafe-3.0.4-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4bced6e2a6dba6a28f7dd3c6ce14df1b2dd495923f16ea484cad03decd463b2b", size = 24656, upload-time = "2026-10-02T23:06:18.963Z" },+ { url = "https://files.pythonhosted.org/packages/9c/18/23997d4c65b355da6390d61cd56e0ab3befd6ba8dda25cb40c602bd0fa6b/markupsafe-3.0.4-cp315-cp315-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3882fb412298575bae3b9c46868251f15cc69307359f87bb1b382e53d6e5a2c9", size = 27737, upload-time = "2026-10-02T23:06:20.117Z" },+ { url = "https://files.pythonhosted.org/packages/d4/36/35998dead3c6af88c38265a56e58100211f036234ab88eb2283fd4cbce44/markupsafe-3.0.4-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:04e7902ba80ee4bac1d50a549606527a1dcf0476cd81403db41099d3b60ec653", size = 25003, upload-time = "2026-10-02T23:06:21.284Z" },+ { url = "https://files.pythonhosted.org/packages/82/96/ef49135ce260db4ca4a12b119ed468449cd248db6b1468e2112b546d7a2e/markupsafe-3.0.4-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:925f929d6b59a8b3f8b8c6ac363cd0af7eecc81efb3071770b3c6717c450a369", size = 23111, upload-time = "2026-10-02T23:06:22.524Z" },+ { url = "https://files.pythonhosted.org/packages/50/7d/83126e338bd88c17a220668235368ad719fd4638e426739858cbb8508f77/markupsafe-3.0.4-cp315-cp315-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f68edfc67aabac33708941f26f22a7b8e9f81429bc0cf249fcf7d66b23af8d19", size = 22403, upload-time = "2026-10-02T23:06:23.785Z" },+ { url = "https://files.pythonhosted.org/packages/83/dd/daf7e420de23c8206c365204e7b85e1251d8e19d34196a56336f316e5ed2/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:e5c802729725bd07e2bc3ab7b76dc7e0bbfc53129d8f1eb1c002c24cf774717e", size = 24083, upload-time = "2026-10-02T23:06:25.037Z" },+ { url = "https://files.pythonhosted.org/packages/19/3c/11eecdc06bc44ad5570350085b572ebf049e8f9a38d1ece6d76640b739cd/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_armv7l.whl", hash = "sha256:55ffd6ce583d97dc71dc92e930324c8c0d25aea7e3ade6ae54ef77cedb096811", size = 22931, upload-time = "2026-10-02T23:06:26.328Z" },+ { url = "https://files.pythonhosted.org/packages/0d/9e/ac0fd77f2a726e56ecc3ca0235d095feace1358d1b822406c2a2ef26a4dc/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:2cb3dd71fc6be918ad4264346a8ed69485f9b7ed7bf35495d8e22807cd6b8bea", size = 24299, upload-time = "2026-10-02T23:06:27.742Z" },+ { url = "https://files.pythonhosted.org/packages/d7/09/c6bd842ad58ff5b3bc76eeed7e9a42a6f11adc5d090ec697b72c9672731e/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:94f5407f7bc64fa6463906b896f9904beeeb7dd8dc116ee8e9056c8714ff9916", size = 21948, upload-time = "2026-10-02T23:06:29.274Z" },+ { url = "https://files.pythonhosted.org/packages/a3/46/82f586711fed61e86faa1ee1bc317d68cd45a10c8bdbe3f7d1fdf9026ad8/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:2dad610540cb2e6272855c178f08ae9a1c7ac258a7fb71660553a5f104b42741", size = 23150, upload-time = "2026-10-02T23:06:30.583Z" },+ { url = "https://files.pythonhosted.org/packages/19/2d/2dfdce99318abbfa26925195fbc17db188c46a1ec6457be121b6f9cfeb42/markupsafe-3.0.4-cp315-cp315-win32.whl", hash = "sha256:03470d1a8268e692ecf79ecd565593e59d44219377a7ead61f1f1b94c1f7ff6b", size = 14269, upload-time = "2026-10-02T23:06:31.949Z" },+ { url = "https://files.pythonhosted.org/packages/5b/ec/6000fd82e8791e58fcd0456ec20f098957e2b03d5ed02eb73241a577c0ba/markupsafe-3.0.4-cp315-cp315-win_amd64.whl", hash = "sha256:d882a373d8093c2941e01291b7ced96e9cbe4781da9a7751ca7e6c70385e5214", size = 14496, upload-time = "2026-10-02T23:06:33.258Z" },+ { url = "https://files.pythonhosted.org/packages/bc/66/e73bd5016421d5d6e2fb6de7dd609f9de020942ac8c626526bd8c6eeaf82/markupsafe-3.0.4-cp315-cp315-win_arm64.whl", hash = "sha256:353bd63081912ab8cfa6a0c7d185934cdf8426f04c618bba6bc4b394f2069b67", size = 14361, upload-time = "2026-10-02T23:06:34.539Z" },+ { url = "https://files.pythonhosted.org/packages/90/df/cb8c3dc98d313a951df2f8968f44e4cb5643df6d3cab749a530ce2f7d972/markupsafe-3.0.4-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:c61750fadcd119d0825bcb7d7d675dd264dcc89cc05292aab5be68ebdbb374ad", size = 11760, upload-time = "2026-10-02T23:06:35.807Z" },+ { url = "https://files.pythonhosted.org/packages/d6/bb/4af9b3ca0753d654ac75f9531d5bd741bb77ca6e696f36807c475ffc099a/markupsafe-3.0.4-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:1c0df495a977d10460a94941799c72d5b5ab03d3858d949b55b5a66c8f371c99", size = 12108, upload-time = "2026-10-02T23:06:37.089Z" },+ { url = "https://files.pythonhosted.org/packages/3f/d4/b56429313aee5fd59b079c3df5615299959e25e7113eb6d8caadbdd7d38a/markupsafe-3.0.4-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:02fa4acbc6a3fc5c693c34d4dd8c1130b7fe99cc915181b0ddd6f72aeb296002", size = 25762, upload-time = "2026-10-02T23:06:38.419Z" },+ { url = "https://files.pythonhosted.org/packages/65/f5/34c181e891aa4f7d59c918584672e0c5eb7fffe76c1387d1246008bf4081/markupsafe-3.0.4-cp315-cp315t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:05295589e619b9bed252a86b532b8e27350abc372d18ba89b59375325e91ec1e", size = 28400, upload-time = "2026-10-02T23:06:39.819Z" },+ { url = "https://files.pythonhosted.org/packages/ce/b5/ad14694fd0ac9a5ce30bc6498f2999378f418583dd1679cca5a1b512957e/markupsafe-3.0.4-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:be6cb0c799abb0e2ba3e618e6d28ddddf7e485f6c2ce938dfa237daf3905072c", size = 25946, upload-time = "2026-10-02T23:06:41.381Z" },+ { url = "https://files.pythonhosted.org/packages/d6/a8/26b606445387d0ceb1eb1f21840094b84e4e3c3c3983d80d10b89823b490/markupsafe-3.0.4-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:26e9867520db70d37f7fb421a7f0d8adb40171011fb84ce869afa1a83370dfa8", size = 23634, upload-time = "2026-10-02T23:06:42.748Z" },+ { url = "https://files.pythonhosted.org/packages/39/a2/b8814de672f1f0094d498bf646f2fec9d6356b503d28ef500b71c5095377/markupsafe-3.0.4-cp315-cp315t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f03460ff076f70ab595bb45a0205ccea1971443575b6920c52e755dec2b3fbfe", size = 23488, upload-time = "2026-10-02T23:06:44.176Z" },+ { url = "https://files.pythonhosted.org/packages/db/c7/287223376fb73335a3cc5d6eb22c6ab01358cf33945a9c39c06b9dac3f4b/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:436e3ffc6310d3c41878c601db29098102fe5d8a467c49da4a4125254e0980f2", size = 25037, upload-time = "2026-10-02T23:06:45.646Z" },+ { url = "https://files.pythonhosted.org/packages/f9/29/4df8355e313426d19e62ba33e0253c009ca12a0894ee77d67fa67255361c/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_armv7l.whl", hash = "sha256:4e2c4809c14559aa7ef426f27fb35afbb38104c349a903bf8f3600456764bb38", size = 23543, upload-time = "2026-10-02T23:06:47.264Z" },+ { url = "https://files.pythonhosted.org/packages/71/e5/8377731e8495668dcc768f645e717df18318c841edaf023a99395f6da9b4/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_ppc64le.whl", hash = "sha256:da2af0d7aebfc2074080d72efa6ab8317c62481ef1f896f65d9999c1c01f4494", size = 25282, upload-time = "2026-10-02T23:06:48.795Z" },+ { url = "https://files.pythonhosted.org/packages/ed/5f/373456e37ceb1478d657d6fe769cbe0a39f0a8dfc1548eeb19c471eefdd9/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_riscv64.whl", hash = "sha256:aa2c838cc024642cc04c6854232f32b43e5e22833dd11119c1766c7873b8370d", size = 22993, upload-time = "2026-10-02T23:06:50.31Z" },+ { url = "https://files.pythonhosted.org/packages/d7/93/2cbd5628435afb6f541bbaced4bce0c2edac4b09a142e6e928b8b0da9858/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:b91cc9d336957239ff200f30097e6fea2dc6d6fb3c81e853eaa09eac904fd894", size = 23712, upload-time = "2026-10-02T23:06:51.759Z" },+ { url = "https://files.pythonhosted.org/packages/81/99/157e10966b033b363aeda5263e82596ee232a0b1d082fdbf90aa417ff083/markupsafe-3.0.4-cp315-cp315t-win32.whl", hash = "sha256:e49fb0d1ce92cfa0cb198cc5b1b11cdf9d0638658e2a2db2687e39db7c87fc78", size = 14300, upload-time = "2026-10-02T23:06:53.241Z" },+ { url = "https://files.pythonhosted.org/packages/33/05/55884815414c9706a23deca150b72c25a62109e65b0b6ce232077802c719/markupsafe-3.0.4-cp315-cp315t-win_amd64.whl", hash = "sha256:4f6e0852a0283b1b1fd776eeb7b766a5f440b3e2bd31ab51af3b400585f3965c", size = 14522, upload-time = "2026-10-02T23:06:54.729Z" },+ { url = "https://files.pythonhosted.org/packages/92/f9/ecbde7149e95b8a0f18e16d5d747f7dc06049d5da2e4f77f6f5e4a1f46a8/markupsafe-3.0.4-cp315-cp315t-win_arm64.whl", hash = "sha256:39dbacefc411633db5b4378b066a9aca70a3d7e2922c9e578d825f844026eeba", size = 14417, upload-time = "2026-10-02T23:06:56.246Z" },+]++[[package]]+name = "mdurl"+version = "0.1.2"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" },+]++[[package]]+name = "mpmath"+version = "1.3.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/e0/47/dd32fa426cc72114383ac549964eecb20ecfd886d1e5ccf5340b55b02f57/mpmath-1.3.0.tar.gz", hash = "sha256:7a28eb2a9774d00c7bc92411c19a89209d5da7c4c9a9e227be8330a23a25b91f", size = 508106, upload-time = "2023-03-07T16:47:11.061Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl", hash = "sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c", size = 536198, upload-time = "2023-03-07T16:47:09.197Z" },+]++[[package]]+name = "networkx"+version = "3.6.1"+source = { registry = "https://pypi.org/simple" }+resolution-markers = [+ "python_full_version < '3.12'",+]+sdist = { url = "https://files.pythonhosted.org/packages/6a/51/63fe664f3908c97be9d2e4f1158eb633317598cfa6e1fc14af5383f17512/networkx-3.6.1.tar.gz", hash = "sha256:26b7c357accc0c8cde558ad486283728b65b6a95d85ee1cd66bafab4c8168509", size = 2517025, upload-time = "2025-12-08T17:02:39.908Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/9e/c9/b2622292ea83fbb4ec318f5b9ab867d0a28ab43c5717bb85b0a5f6b3b0a4/networkx-3.6.1-py3-none-any.whl", hash = "sha256:d47fbf302e7d9cbbb9e2555a0d267983d2aa476bac30e90dfbe5669bd57f3762", size = 2068504, upload-time = "2025-12-08T17:02:38.159Z" },+]++[[package]]+name = "networkx"+version = "3.7"+source = { registry = "https://pypi.org/simple" }+resolution-markers = [+ "python_full_version >= '3.12'",+]+sdist = { url = "https://files.pythonhosted.org/packages/dc/76/3af777226b63a5e64a6b36b1ec5855c14e2b94a37096d4760e595fc43511/networkx-3.7.tar.gz", hash = "sha256:fd77a511bd90f39f3d016351345b52cf5319b813bdca01de3f755d3cca62e96a", size = 1866482, upload-time = "2026-09-21T16:45:16.974Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/7e/cd/fe58041e9011f307c490e3e17dd48cc516448f7c698a3f2d9d9d65d7e6a8/networkx-3.7-py3-none-any.whl", hash = "sha256:e3fd2c13a7814cee3746340d8d7f8598a67f16a58bf47fb7f8793fab6efca1b0", size = 2142205, upload-time = "2026-09-21T16:45:14.609Z" },+]++[[package]]+name = "numpy"+version = "2.4.6"+source = { registry = "https://pypi.org/simple" }+resolution-markers = [+ "python_full_version < '3.12'",+]+sdist = { url = "https://files.pythonhosted.org/packages/d0/ad/fed0499ce6a338d2a03ebae59cd15093910c8875328855781952abf6c2fe/numpy-2.4.6.tar.gz", hash = "sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda", size = 20735807, upload-time = "2026-05-18T23:37:14.07Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b3/49/ec46835a70be8fa6446c495126ac84fdb28cb2558e1620ffb87a10c8b64c/numpy-2.4.6-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:0280e0356c0829a18d9de1cb7eee50ec22ca639878d7240307ca0943d73cd2c4", size = 16969194, upload-time = "2026-05-18T23:33:13.503Z" },+ { url = "https://files.pythonhosted.org/packages/0e/0d/f5957185c0ee2f3e12f78715aa9e3b353fd83633316c8532b38faa37e3f6/numpy-2.4.6-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:110f8b71aacb688ec69062bb7f6938a0f8acb01b7c1c4beb453c65b6d234584d", size = 14964111, upload-time = "2026-05-18T23:33:17.795Z" },+ { url = "https://files.pythonhosted.org/packages/ad/40/40a40ee0ddf7ceb782c49af278894b686e586d65d8c1889c8b5da01a3d7d/numpy-2.4.6-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:4cfe66903cc32a9921a6733d96b19bb6abf310397581bbad89c228f5abaf0ee8", size = 5469159, upload-time = "2026-05-18T23:33:20.654Z" },+ { url = "https://files.pythonhosted.org/packages/63/13/f9a8046535cb21deae82f8d03de9617e08882d274fad2539630761888228/numpy-2.4.6-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:8155154c7c691289fe18f510b5d4657c68c67989f293f0535a91360392ff6538", size = 6798936, upload-time = "2026-05-18T23:33:22.987Z" },+ { url = "https://files.pythonhosted.org/packages/33/a8/6fa8c1a345a8c85dbb21932c447bee07c30a2c2a3f31e369c0a84b300147/numpy-2.4.6-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ab0a9c4ffb1a6d95ef519fe4247dba8eb6b18ad93999f76b7f657039acabd47", size = 15966692, upload-time = "2026-05-18T23:33:26.62Z" },+ { url = "https://files.pythonhosted.org/packages/02/03/74fe2a4cb3817d94d86402f2506554130a2f01414e299b5a843e5a8a957f/numpy-2.4.6-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:89cd468399cfd2504718f0ba50e410dca55a170b61a02ad92bb18c8a65186e93", size = 16918164, upload-time = "2026-05-18T23:33:29.955Z" },+ { url = "https://files.pythonhosted.org/packages/c5/80/3615be3313f7e7696609bc194b9f0101da809df79e859bdb84e0cd043f46/numpy-2.4.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:c2d37ab77531417474168eb79d6d80b14f821a966818505d03013d0833edb7a8", size = 17322877, upload-time = "2026-05-18T23:33:34.724Z" },+ { url = "https://files.pythonhosted.org/packages/ca/ac/a691e0fe2675e370d0e08ff905adc49a1c8830e8cae03efe4477e92cd55d/numpy-2.4.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f407cb6b8e9d6d8c626bc73c945db1706035af8fd632295547bf1c9e46d092d6", size = 18651487, upload-time = "2026-05-18T23:33:38.217Z" },+ { url = "https://files.pythonhosted.org/packages/15/a7/9bc1cd626d7bf6869bfedf27b91b6ab5dd607758bf8e959d6fa80c6a59cb/numpy-2.4.6-cp311-cp311-win32.whl", hash = "sha256:ddea102b48f9e339f3948bf22040944184627a30fdf7f858667673b9c5f033c8", size = 6233945, upload-time = "2026-05-18T23:33:41.331Z" },+ { url = "https://files.pythonhosted.org/packages/c5/31/7fc6239c12bce7e931463251cca4426c465e1876ba3cc785402ef4dd8f4e/numpy-2.4.6-cp311-cp311-win_amd64.whl", hash = "sha256:1e254a00cdf42b1e4d5b3d68d33af63268d41340d8885df2ab6470f2e1500147", size = 12608406, upload-time = "2026-05-18T23:33:44.131Z" },+ { url = "https://files.pythonhosted.org/packages/27/83/140f85a466595a16382996a1bf06b2b54bcd597488921b0c9daaeeda72af/numpy-2.4.6-cp311-cp311-win_arm64.whl", hash = "sha256:ed9749eef4cbd126da3dc1d6bcb3a57f5eb7ac6a6484146bdbf743f552dfc577", size = 10479528, upload-time = "2026-05-18T23:33:50.725Z" },+ { url = "https://files.pythonhosted.org/packages/95/2a/3d7b5ac8aac24feaf9ad7ed58f45b0bbc06d37e4338ae84c9f2298b570f9/numpy-2.4.6-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:001fbb8e08d942dd57599e781f2472269ee7f2755fae407b4f67b2f0b17da3f1", size = 16689119, upload-time = "2026-05-18T23:33:54.065Z" },+ { url = "https://files.pythonhosted.org/packages/ea/12/92c4c131527599e8288d6918e888d88726f84d805d784b771f32408aeaef/numpy-2.4.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ebfb099f8dcf083deef3ac1ca4c1503f387cf76296fcb3816b66f5ecb5f54fdb", size = 14699246, upload-time = "2026-05-18T23:33:57.621Z" },+ { url = "https://files.pythonhosted.org/packages/ad/fe/c0a6b7b2ca128a8fb228575147073b660656734b8ebe4d76c8fd748dcc79/numpy-2.4.6-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:3213d622a0283a39a93d188f3cf72b26862df52fbb4ca3697f51705016523d41", size = 5204410, upload-time = "2026-05-18T23:34:00.302Z" },+ { url = "https://files.pythonhosted.org/packages/f3/d4/9770d14ba719432bb90a421bfd443872ed0f70f7264b64bec12ea363d5fd/numpy-2.4.6-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:357cc07a6d7b0b182ff02249616a03742827ebb1277546b5c7cd7f7620a45698", size = 6551240, upload-time = "2026-05-18T23:34:02.852Z" },+ { url = "https://files.pythonhosted.org/packages/c9/c6/50a46a6205feba2343f1d6d17438107c5dc491ed1c736e6ea68689fd906b/numpy-2.4.6-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f9fb9157b4ce2971008323afe46053787b526ef624fea915b261468a8421a0f", size = 15671012, upload-time = "2026-05-18T23:34:05.485Z" },+ { url = "https://files.pythonhosted.org/packages/99/60/14115e6364fa676c5397c2ad3004e527e9aa487abf5d0706ec81bbd08529/numpy-2.4.6-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:90f9849678c75fe7afa2d348ac842c168b0a4d3d61919687216dfc547976d853", size = 16645538, upload-time = "2026-05-18T23:34:09.265Z" },+ { url = "https://files.pythonhosted.org/packages/ae/c5/693cbe59e57db94d2231fa519ca3978dc9e19da5a8f088588f5c6e947ff2/numpy-2.4.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c1a2af6c6ef86344a6b0db6b97834208bf598db514f2b155042439b62605601a", size = 17020706, upload-time = "2026-05-18T23:34:13.053Z" },+ { url = "https://files.pythonhosted.org/packages/ef/fc/85b7c4eff9b4966ade25c2273cf7e7012e92366c032058653934b37de044/numpy-2.4.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e5805d5a22fd19c8ccff10a9561f9df94436b0545619ea579db2d3c35294bce2", size = 18368541, upload-time = "2026-05-18T23:34:17.024Z" },+ { url = "https://files.pythonhosted.org/packages/f6/81/e1b27545deedce7f4a0b348618c6b62d74e36a4dc9ccd42f3eb2f85eee32/numpy-2.4.6-cp312-cp312-win32.whl", hash = "sha256:e3eeb0aabd6bd5ce64faae67e9935203a6991b4bc2a485a767fbafb2c5125f45", size = 5962825, upload-time = "2026-05-18T23:34:20.3Z" },+ { url = "https://files.pythonhosted.org/packages/ab/ca/feab00bd44aa5fe1ad2c18f08b4d3bb92e26484b0b1d1443897809ed528c/numpy-2.4.6-cp312-cp312-win_amd64.whl", hash = "sha256:d8e8286dd7cea7895157318d1b91cdacac64c479f3cbc8dce548331728484751", size = 12321687, upload-time = "2026-05-18T23:34:23.095Z" },+ { url = "https://files.pythonhosted.org/packages/63/cf/5a6d34850a39d1093558564f77ee8e8e0bee5061151b8f05a55711001ec7/numpy-2.4.6-cp312-cp312-win_arm64.whl", hash = "sha256:4081eb135ac24158bd51cdfbef16f1c64df7063b1143f24731387137c092bec8", size = 10221482, upload-time = "2026-05-18T23:34:25.876Z" },+ { url = "https://files.pythonhosted.org/packages/fb/82/bdab26d7438c6791ca31b7c024ca37c1eab8b726ba236129005cd4a06e45/numpy-2.4.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0", size = 16684648, upload-time = "2026-05-18T23:34:29.41Z" },+ { url = "https://files.pythonhosted.org/packages/1b/30/a80189bcc7f5e4258b3fbc3968d909d1756f54d023299ecc39ad6fdb9ef8/numpy-2.4.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb", size = 14693902, upload-time = "2026-05-18T23:34:33.013Z" },+ { url = "https://files.pythonhosted.org/packages/97/12/70b5d0d7c15e1ebb8a6a84a8caa1d19e181d84fb58bb6d70aca29099dec1/numpy-2.4.6-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f", size = 5198992, upload-time = "2026-05-18T23:34:36.132Z" },+ { url = "https://files.pythonhosted.org/packages/ba/8c/ebd2a8f8a83541f8d38cc5667e8c2b69cecfd30da6e45693e8158857d44b/numpy-2.4.6-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3", size = 6546944, upload-time = "2026-05-18T23:34:38.484Z" },+ { url = "https://files.pythonhosted.org/packages/bb/c5/7b863a97a91671a0338f4253bd3b5a3d3852f0692dae91711c9f4a10e787/numpy-2.4.6-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b", size = 15669392, upload-time = "2026-05-18T23:34:41.257Z" },+ { url = "https://files.pythonhosted.org/packages/a5/9d/3584b9984ca4c047aea75214ce1a4c4c73d849bd71b604264b7f5653f8a8/numpy-2.4.6-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089", size = 16633220, upload-time = "2026-05-18T23:34:45.075Z" },+ { url = "https://files.pythonhosted.org/packages/05/ae/7c67fba23bd98caec7c99261f3a16072ade14813486b0282cb29846de832/numpy-2.4.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a", size = 17020800, upload-time = "2026-05-18T23:34:49.065Z" },+ { url = "https://files.pythonhosted.org/packages/d9/5d/3b6725cb31d983c5e66916f5d36f6d7e5521129e4c4404d64f918292a5b6/numpy-2.4.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605", size = 18357600, upload-time = "2026-05-18T23:34:52.709Z" },+ { url = "https://files.pythonhosted.org/packages/f7/da/2ccc6c2fe8898dee01d90c75c5f5f914a23daf99e3e0f59516a08760c8b5/numpy-2.4.6-cp313-cp313-win32.whl", hash = "sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91", size = 5961134, upload-time = "2026-05-18T23:34:55.618Z" },+ { url = "https://files.pythonhosted.org/packages/b5/cd/9cc4dc876fb065d5c220aae4d5e14826b2715331bb7618ce1fb07a679d99/numpy-2.4.6-cp313-cp313-win_amd64.whl", hash = "sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359", size = 12318598, upload-time = "2026-05-18T23:34:58.928Z" },+ { url = "https://files.pythonhosted.org/packages/39/1e/c0bcba1f8694116485fe28fd1be698c278fcda4141c5b0e53a2aed8b12a8/numpy-2.4.6-cp313-cp313-win_arm64.whl", hash = "sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778", size = 10222272, upload-time = "2026-05-18T23:35:02.167Z" },+ { url = "https://files.pythonhosted.org/packages/63/6d/cc5619247c8f4204e507f5883528372e4ac4bb189e579fb859a12e480b1f/numpy-2.4.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1", size = 14821197, upload-time = "2026-05-18T23:35:05.468Z" },+ { url = "https://files.pythonhosted.org/packages/00/58/f1c39161c87d9e9bed660f1ed4bafc0e403d5ec9650b6dd77aead07d489b/numpy-2.4.6-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe", size = 5326287, upload-time = "2026-05-18T23:35:08.693Z" },+ { url = "https://files.pythonhosted.org/packages/af/57/3917ab0fd97f271a8694513581b8a36c655f111c446852c302f04ccdb6fc/numpy-2.4.6-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997", size = 6646763, upload-time = "2026-05-18T23:35:11.459Z" },+ { url = "https://files.pythonhosted.org/packages/eb/0f/037e64c494b67581ae18193d770adef354c41f3f2c8ebf865602d949bf8f/numpy-2.4.6-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20", size = 15728070, upload-time = "2026-05-18T23:35:14.79Z" },+ { url = "https://files.pythonhosted.org/packages/21/a6/5d2bae9c9542eb4df16dc9c46dc79c186e9bad53805dfa5399a6023c6db0/numpy-2.4.6-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d", size = 16681752, upload-time = "2026-05-18T23:35:18.836Z" },+ { url = "https://files.pythonhosted.org/packages/92/14/23d1dfb410ae362cd59ce53e936b1513d545eb40db3949ced632e19a459e/numpy-2.4.6-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67", size = 17086024, upload-time = "2026-05-18T23:35:22.52Z" },+ { url = "https://files.pythonhosted.org/packages/4b/6e/23595a2c642cdf3bc567877064bdd7f91c8b0038a4453cf2daf7248eafe9/numpy-2.4.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd", size = 18403398, upload-time = "2026-05-18T23:35:26.398Z" },+ { url = "https://files.pythonhosted.org/packages/8a/90/0ac3bc947217e66dec77e7cbc6a1979d1af70b6461b82f620d3bccd5e4c8/numpy-2.4.6-cp313-cp313t-win32.whl", hash = "sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab", size = 6084971, upload-time = "2026-05-18T23:35:29.387Z" },+ { url = "https://files.pythonhosted.org/packages/77/71/5673e351671a1d2bd6063b91b44f70c0affea7d1516fa7a6572941ba4aa1/numpy-2.4.6-cp313-cp313t-win_amd64.whl", hash = "sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75", size = 12458532, upload-time = "2026-05-18T23:35:32.175Z" },+ { url = "https://files.pythonhosted.org/packages/3f/88/19d3503c5046e688f049274b27a3ef3d771152fa80d3ba3d01a3dff61abe/numpy-2.4.6-cp313-cp313t-win_arm64.whl", hash = "sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd", size = 10291881, upload-time = "2026-05-18T23:35:35.465Z" },+ { url = "https://files.pythonhosted.org/packages/f8/91/3ab2044d05fd16d343c5ac2e69b127f1b2854040dd20b193257c78028bd3/numpy-2.4.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079", size = 16683458, upload-time = "2026-05-18T23:35:38.353Z" },+ { url = "https://files.pythonhosted.org/packages/8e/62/764ce66fa4147ae6d73071a3abf804ffe606f174618697c571acdf26a7c9/numpy-2.4.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7", size = 14704559, upload-time = "2026-05-18T23:35:42.14Z" },+ { url = "https://files.pythonhosted.org/packages/60/61/23f27c172f022e04025b7dc2367f4d63c1a398120607ec896228649a6f48/numpy-2.4.6-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5", size = 5209716, upload-time = "2026-05-18T23:35:45.377Z" },+ { url = "https://files.pythonhosted.org/packages/03/71/21cf70dc6ea3e3acb95fc53a265b2fc248b981f0194ceb5b475271b8809d/numpy-2.4.6-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096", size = 6543947, upload-time = "2026-05-18T23:35:47.926Z" },+ { url = "https://files.pythonhosted.org/packages/d5/91/64288395ee1799bd2e0b04a305dce9666da90c961e1f3fe982a05ee1c036/numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b", size = 15685197, upload-time = "2026-05-18T23:35:50.863Z" },+ { url = "https://files.pythonhosted.org/packages/f3/eb/ebffaa97dc55502df69584a8f0dcf07f69a3e0b3e2323670a2722db9aa39/numpy-2.4.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8", size = 16638245, upload-time = "2026-05-18T23:35:54.752Z" },+ { url = "https://files.pythonhosted.org/packages/b8/0b/54f9da33128d7e350fab89c7455902eeae70349ee52bddb448dc4a576f45/numpy-2.4.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402", size = 17036587, upload-time = "2026-05-18T23:35:58.355Z" },+ { url = "https://files.pythonhosted.org/packages/b6/f0/fdebc1052db1cc37c64beb22072d67cd6d1c71adca1299f53dec2b5e20d3/numpy-2.4.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb", size = 18363226, upload-time = "2026-05-18T23:36:02.845Z" },+ { url = "https://files.pythonhosted.org/packages/aa/b4/298628d98c72b57e57f7165ae6a481a1deaf6f3c28262a6e4c739c275930/numpy-2.4.6-cp314-cp314-win32.whl", hash = "sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1", size = 6010196, upload-time = "2026-05-18T23:36:05.92Z" },+ { url = "https://files.pythonhosted.org/packages/df/ac/46de6dda46478f7942f839e094970be2d4a861e005c4b3bf07c92e291a09/numpy-2.4.6-cp314-cp314-win_amd64.whl", hash = "sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261", size = 12450334, upload-time = "2026-05-18T23:36:09.107Z" },+ { url = "https://files.pythonhosted.org/packages/78/92/b8b798ac784102c0da830d2257d59358e3d3d90d1e2b3f2575dad976c5cf/numpy-2.4.6-cp314-cp314-win_arm64.whl", hash = "sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6", size = 10495678, upload-time = "2026-05-18T23:36:12.766Z" },+ { url = "https://files.pythonhosted.org/packages/30/34/ec28d1aa8115971537c01469ab2011ee96827930f0a124de1000cc2a7ed7/numpy-2.4.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a", size = 14823672, upload-time = "2026-05-18T23:36:16.473Z" },+ { url = "https://files.pythonhosted.org/packages/16/bd/f6d1fede4e54e8042a7ff97bb495510f3c220f94bcd9e8b228e87c92cc0d/numpy-2.4.6-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e", size = 5328731, upload-time = "2026-05-18T23:36:19.767Z" },+ { url = "https://files.pythonhosted.org/packages/f4/f0/e105b9e2fd728a9910103884decd6951d9dd73896b914a98d9a231de02ee/numpy-2.4.6-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e", size = 6649805, upload-time = "2026-05-18T23:36:22.266Z" },+ { url = "https://files.pythonhosted.org/packages/82/dd/1206a7ca6ab15e3f02069707ca96222e202af681bb73756da7527f3cb837/numpy-2.4.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43", size = 15730496, upload-time = "2026-05-18T23:36:25.713Z" },+ { url = "https://files.pythonhosted.org/packages/51/e7/38d3ea825dcab85a591734decb2f6c67caa7c8367d374df1a1c3842f9b07/numpy-2.4.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e", size = 16679616, upload-time = "2026-05-18T23:36:29.652Z" },+ { url = "https://files.pythonhosted.org/packages/93/b7/caabfdf53edf663e0b4eb74d7d405d83baef09eb5e83bcd32d601d72b93e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895", size = 17085145, upload-time = "2026-05-18T23:36:33.449Z" },+ { url = "https://files.pythonhosted.org/packages/f9/45/68d7c33a6bcf3e5aa3bdbd57a367e6f615286dfd6482f97e8ffeb734306e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4", size = 18403813, upload-time = "2026-05-18T23:36:37.369Z" },+ { url = "https://files.pythonhosted.org/packages/9c/50/0753655aa844c99cd9e018aacf76f130f1bd81d881bb74bc0aef5d73a8ba/numpy-2.4.6-cp314-cp314t-win32.whl", hash = "sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063", size = 6156982, upload-time = "2026-05-18T23:36:40.817Z" },+ { url = "https://files.pythonhosted.org/packages/b2/d4/7c67becf668f973cb490cec3e98dfd799d866f9c989a54d355672cfa0db6/numpy-2.4.6-cp314-cp314t-win_amd64.whl", hash = "sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627", size = 12638908, upload-time = "2026-05-18T23:36:43.996Z" },+ { url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" },+ { url = "https://files.pythonhosted.org/packages/de/12/b422cc84439adc0d00de605bf4a308890ae5c26f2c71fbd73e5d08fbb0dd/numpy-2.4.6-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:55cced7c52e981362f708ad635198e97a752dfba412cc03c23bbf3bd8d5cd662", size = 16847511, upload-time = "2026-05-18T23:36:50.673Z" },+ { url = "https://files.pythonhosted.org/packages/44/53/f481bef68011740f8849418d82db07230e825013f31f4eef5ba5b805316a/numpy-2.4.6-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:d6da64deb6b8ed903e7560180a92f2d804ee1ba5eeb849ac2748b8c1aba1f6d7", size = 14889064, upload-time = "2026-05-18T23:36:53.879Z" },+ { url = "https://files.pythonhosted.org/packages/7f/57/42ed575c10ced8af951d426bc4e1f8aff16fd851db33f067036215a7f860/numpy-2.4.6-pp311-pypy311_pp73-macosx_14_0_arm64.whl", hash = "sha256:68a5124b13fa6cc2086764a20005d30bc0548146f7f5322f02fce212ca14317f", size = 5394157, upload-time = "2026-05-18T23:36:57.194Z" },+ { url = "https://files.pythonhosted.org/packages/6a/ef/f66cc724fcc36c1e364c67f51ae9146090b8b584f27d58b97fdae3edd737/numpy-2.4.6-pp311-pypy311_pp73-macosx_14_0_x86_64.whl", hash = "sha256:948424b06129ce883307e8cff868c31396d8dc7630a59c61d70d98dbe70f222c", size = 6708728, upload-time = "2026-05-18T23:36:59.575Z" },+ { url = "https://files.pythonhosted.org/packages/1a/9c/c531f2293b91265d8b48e9b329f54fdd7ffae73cb4134ea10cca4237e9cc/numpy-2.4.6-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5dbbdb29840ca3d91ee0fece42fc29278886d908280bfec0a5846c6f901a3eb0", size = 15798374, upload-time = "2026-05-18T23:37:02.674Z" },+ { url = "https://files.pythonhosted.org/packages/1a/b0/413077f6b1153ed3cba361401c6783bbad6114804a000cc22eb71c13e190/numpy-2.4.6-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8ad03c0965fb3c692200e74d458ca28c1dbb4ce96f9a479a8aa041ad5fabca02", size = 16747286, upload-time = "2026-05-18T23:37:06.327Z" },+ { url = "https://files.pythonhosted.org/packages/15/ce/e5ec180bc41812edcd8daeb8639d205622c0e8c02259d8ab25a0201b3c2a/numpy-2.4.6-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:2803abfebfc990042cd494d8ce2d5f82e9d847af6d35ec486923aa19dbad5e73", size = 12504263, upload-time = "2026-05-18T23:37:09.715Z" },+]++[[package]]+name = "numpy"+version = "2.5.4"+source = { registry = "https://pypi.org/simple" }+resolution-markers = [+ "python_full_version >= '3.12'",+]+sdist = { url = "https://files.pythonhosted.org/packages/95/b0/c7453d0b6e2073c3264468b106ee1563750cecc910965e67357e3698c83e/numpy-2.5.4.tar.gz", hash = "sha256:9a94cf751c9ad8ebaa835bcd3d40dacf8534ad086b88c38029b65123c7999d2a", size = 20866315, upload-time = "2026-10-10T20:05:31.422Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d0/97/ba2074e92b7befea137e77ea8471e768bbd87c339b7e8c9f5a931949f977/numpy-2.5.4-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c6342f54c67093cae5c0227eb0eb772fdb79f2a2c37a6eb278b9909ee06aa356", size = 17001609, upload-time = "2026-10-10T20:02:40.843Z" },+ { url = "https://files.pythonhosted.org/packages/ff/a9/bac826765e971d8e16e2064e9ac7525fd69b40ac17c905033a7f5442023f/numpy-2.5.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b11e8fda06a7d69f15ebf542660b74466c2e51094800c1fb794f47ad4faeef17", size = 12015718, upload-time = "2026-10-10T20:02:43.45Z" },+ { url = "https://files.pythonhosted.org/packages/31/2f/5ea3570fcb8ccd0882bea99436a513b2c85dad8f774a2057849130a8fb99/numpy-2.5.4-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:9cb18a327b49c5c337f972b03682f6a49855525faaf3c0d3e9c96cd0fd8880a8", size = 5451717, upload-time = "2026-10-10T20:02:46.169Z" },+ { url = "https://files.pythonhosted.org/packages/34/f2/b4fc1bafca03868220b5eaf729d2f21ebd7d7b151c0f9e144fe212bbca35/numpy-2.5.4-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:aec3fc4b32ff82421274f5d205c559c51c840c8df66a78efd7f3612dd005a26a", size = 6789926, upload-time = "2026-10-10T20:02:48.139Z" },+ { url = "https://files.pythonhosted.org/packages/dc/96/8319e2457ae4333c62c815c7006b869a4f60985c1e01024c2f8c6c040fe5/numpy-2.5.4-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fe4d21ab149f15e4e6043dfb0de87e6e5f34ac176cde83060e9802981fca2ac2", size = 15695312, upload-time = "2026-10-10T20:02:50.115Z" },+ { url = "https://files.pythonhosted.org/packages/43/a3/c799c62e19c337e6d3770b08e475887fb30ce8477d3c09efca6b2f0228a6/numpy-2.5.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fbde6962867ee75b48b0ee29b2b9372ec5d617799dbaf38e82dc0596f2f7738a", size = 16727283, upload-time = "2026-10-10T20:02:53.186Z" },+ { url = "https://files.pythonhosted.org/packages/39/6b/3604e53fb00314d0dc1b94ec9125a1484f649c0a17480b1f0f0c7a9d6250/numpy-2.5.4-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:381a7a3d2e65e64c0ec302795ab9dc12bb1e73f150904699c153716177eebdaf", size = 17047890, upload-time = "2026-10-10T20:02:56.038Z" },+ { url = "https://files.pythonhosted.org/packages/4a/7a/e8b58a5289a0d464c52885de47c35a935cdd70c03a4c3ab94a5126416dd0/numpy-2.5.4-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:b89d0aaae2fe498c648f4c4795c084db535af5bd98ef942b2a3681fb74ce8645", size = 18485839, upload-time = "2026-10-10T20:02:59.018Z" },+ { url = "https://files.pythonhosted.org/packages/6f/c9/47094f597015009f310b8c900def59065ef1ff5a6fe7b51fc65ec58ec2c6/numpy-2.5.4-cp312-cp312-win32.whl", hash = "sha256:9968ab7e49b93ac6e1c3b2239732183152c9150f16308d30b66a372cffe3483c", size = 6138936, upload-time = "2026-10-10T20:03:01.626Z" },+ { url = "https://files.pythonhosted.org/packages/12/33/fefe62073dc8acfd0f2b9ed7c003af2f50aa61555e113e6db02b8f79f145/numpy-2.5.4-cp312-cp312-win_amd64.whl", hash = "sha256:a7b1b6353e36a7e50de2973a38d705c88ee93adcf120673cee7f45a4a3fa223a", size = 12573091, upload-time = "2026-10-10T20:03:04.349Z" },+ { url = "https://files.pythonhosted.org/packages/1a/07/161270b0c2eec56e4c905f6d6d22e1b836887b2cb189d3f5820aa588e9dd/numpy-2.5.4-cp312-cp312-win_arm64.whl", hash = "sha256:aa1cce2ff3f8d953de38b76bf44602caeb69f101430208f64a10067f7cb4b1d3", size = 10521630, upload-time = "2026-10-10T20:03:06.767Z" },+ { url = "https://files.pythonhosted.org/packages/67/14/1c3ee0118a8fce08565a5d8482631608426a33af10a01077fada5dc7c119/numpy-2.5.4-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:2377da2dd3ba2c1200956acbab2a358c83b8e1f8531191672d1cd6ad83250d53", size = 16997729, upload-time = "2026-10-10T20:03:09.291Z" },+ { url = "https://files.pythonhosted.org/packages/83/8c/b0ea9477fb1f0d4484bbc5cba21678cc9969704d8d7f3f158d1db35f8e14/numpy-2.5.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:7415db95818b39ec475a5eea54d9e3b6bc83e3912158e46da3438cdce399804d", size = 12009826, upload-time = "2026-10-10T20:03:11.946Z" },+ { url = "https://files.pythonhosted.org/packages/e2/84/6a3d75b3ba3dfe84ac0053450753d1e6d250a8bf80f66474cc46d1fb643f/numpy-2.5.4-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:6d6a71b9d9a97c03633aa12565ef2825ffa036cc1d99cfd50dacf0f128af4fe2", size = 5445803, upload-time = "2026-10-10T20:03:14.329Z" },+ { url = "https://files.pythonhosted.org/packages/61/18/bb993f267ca20b376e07092a16793a5b31ed3138751e9ba480011a14d742/numpy-2.5.4-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:d8200f16437b289a5bb927c6e184eccc3e8389bc0070fea4cd5b9e13c1757959", size = 6786220, upload-time = "2026-10-10T20:03:16.602Z" },+ { url = "https://files.pythonhosted.org/packages/db/b6/135bb0953b61dc21c6cafa14b424ae666944e4899cf140e00c2b322a1a45/numpy-2.5.4-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1c2e71b04c6cad90026e544501bbe0ab9290fa8a4d845e7e8c0d124fb429c988", size = 15689178, upload-time = "2026-10-10T20:03:18.721Z" },+ { url = "https://files.pythonhosted.org/packages/da/24/3bd070f3269dc609d8f26b2643f62ef91bb415841c0b294805aaf7fe06da/numpy-2.5.4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6ffa07666f8da0eef81d149934a626d0d95fbd6838432a33e66245423a9062c0", size = 16718044, upload-time = "2026-10-10T20:03:21.386Z" },+ { url = "https://files.pythonhosted.org/packages/c7/8e/9d15bd356b0a019c965312b1a3c6a727cac4cae5bc40045fbc12ce4cff9c/numpy-2.5.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:2fa3328f784fc8277fc48026f6cad516f5c561c5d8e2e39b3c9e0c8f23223b34", size = 17048364, upload-time = "2026-10-10T20:03:24.468Z" },+ { url = "https://files.pythonhosted.org/packages/dc/fe/9d5b560db964f15871885f2250795d15945f8699e17ef90c0c2ff4c875b2/numpy-2.5.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b86966fbe4ad7de710422175572bcdc75fdedadfb54bc6fab7deabccddd7780b", size = 18474904, upload-time = "2026-10-10T20:03:27.895Z" },+ { url = "https://files.pythonhosted.org/packages/e9/98/d27552990f1bd611ef3e7466adadc78312ea2df63b83aad47fdc3d3ca8df/numpy-2.5.4-cp313-cp313-win32.whl", hash = "sha256:5258bc06526964be5face2fc6f756857a3f24f21ec3e72ca131337a75b165d6c", size = 6134537, upload-time = "2026-10-10T20:03:30.511Z" },+ { url = "https://files.pythonhosted.org/packages/90/8c/140a40398a66b4471211be1affdb6ed24c486d581bd28d07b7f2fcb69540/numpy-2.5.4-cp313-cp313-win_amd64.whl", hash = "sha256:8b4d2fd2d34e5f8c9235ee787de5631a37a28402b15cb80814df973d2be54129", size = 12566113, upload-time = "2026-10-10T20:03:32.612Z" },+ { url = "https://files.pythonhosted.org/packages/34/52/01d205e5e8ccb27b2b0b141e801f22b830198c979111b0fa44771438d9a9/numpy-2.5.4-cp313-cp313-win_arm64.whl", hash = "sha256:bc39ac66a7a9a3fbd6134fda43136b60ffde99c8f4501e64e0d2b24da137babf", size = 10519523, upload-time = "2026-10-10T20:03:35.163Z" },+ { url = "https://files.pythonhosted.org/packages/99/ba/005cb5edd580d2f84d7ca3206b92dc17d4388e56e6f87ffe8f2762f83139/numpy-2.5.4-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:c668b2f0d651605b58892644b0e302c7157f7159544227758c896982ef384b18", size = 17005499, upload-time = "2026-10-10T20:03:37.961Z" },+ { url = "https://files.pythonhosted.org/packages/f3/49/fee7587c33ee35f7977f9051d7f2023d4e7246d62710c80f20c2361ea232/numpy-2.5.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:ffa6ce09a1c6a08e9667dd9c97aa0b14184e8d18f2a14b78b2a2328c9147f076", size = 12019666, upload-time = "2026-10-10T20:03:40.606Z" },+ { url = "https://files.pythonhosted.org/packages/d5/b2/c6ce165acffceb15a82c07b9cc77d391f86b3f379ba62911908ae5d34b91/numpy-2.5.4-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:956555e0603a4d38019ae6925711cb9dc43195c076a928accf7ea5d50bddfe53", size = 5455617, upload-time = "2026-10-10T20:03:43.138Z" },+ { url = "https://files.pythonhosted.org/packages/77/7f/dd85ce260a669a89be06842cf355d7353a33e6cfbc590fb8ebb947d88dc9/numpy-2.5.4-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:2c2c4afffdeb7920e445028dd71eb932cac3e704792e964bc2a232426d4f1255", size = 6791932, upload-time = "2026-10-10T20:03:44.874Z" },+ { url = "https://files.pythonhosted.org/packages/63/d6/34b0a2b0741386a63025a65a2c09caaaaaad6d0ca95b66cd65c30dd7fcb5/numpy-2.5.4-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4054173604cd8658796053f1f3bc0befb68ec1c0762c57fdad61e199256a8617", size = 15710899, upload-time = "2026-10-10T20:03:46.839Z" },+ { url = "https://files.pythonhosted.org/packages/16/d5/928078d2b28f26829b138b4a6c3980045022fb409f570657a224ae60ef4e/numpy-2.5.4-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d549420b8858885cea8838a727842249218b9c1da24dd517e25c9c7a948310a3", size = 16721710, upload-time = "2026-10-10T20:03:49.489Z" },+ { url = "https://files.pythonhosted.org/packages/f9/cf/673fd1b8f4cd78eb6320e87ec4c90ac19c095644259e3749853a405c70f4/numpy-2.5.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:823874a507a84af050493b622affde94b6f7c3a0dc22cb2801381bc03b871c00", size = 17066182, upload-time = "2026-10-10T20:03:52.25Z" },+ { url = "https://files.pythonhosted.org/packages/f3/92/a77b5061b1b3e2643928c37976d79ee173e1b171ed158b7a3c61056b41bc/numpy-2.5.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:4e263278bfb5ee6409db8aedbc4cc32973b1b82bc1e8d3c668551d04d83a7e37", size = 18480315, upload-time = "2026-10-10T20:03:55.39Z" },+ { url = "https://files.pythonhosted.org/packages/bb/1d/1486ef3d3fb2279fd93c4c43c1bbbf1ca389a19816696684409f71babaab/numpy-2.5.4-cp314-cp314-win32.whl", hash = "sha256:cfd73180400042a7c532d30c5e287bdd03c59ff9ee1b4c0316af0539e29dfe23", size = 6185739, upload-time = "2026-10-10T20:03:58.186Z" },+ { url = "https://files.pythonhosted.org/packages/52/9a/e1e512ebc948d5b9dd33b08736760f0ebbed2848fd4eda1f553088a6dcee/numpy-2.5.4-cp314-cp314-win_amd64.whl", hash = "sha256:2ca144f15135b6212a5c47b1e2aeca6e412f102f95a2d5d88d8aec77eb255de3", size = 12703552, upload-time = "2026-10-10T20:04:00.28Z" },+ { url = "https://files.pythonhosted.org/packages/2c/05/de709a982d7bbcd688a3fad71f002e9ff80c2db39e03ee726609b610f1d1/numpy-2.5.4-cp314-cp314-win_arm64.whl", hash = "sha256:468397ba3c64427474706e5c9123fe266395496714dc684294eac75cd4930d1e", size = 10803901, upload-time = "2026-10-10T20:04:02.659Z" },+ { url = "https://files.pythonhosted.org/packages/13/34/083570ada3bb2a30fbe5d77c8c6fef9141144a15d33e6f793a67e9749ab8/numpy-2.5.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1ef3aa6d7e29bb13677323114280b05acc57607fa2300e66432d665d5418a162", size = 12138695, upload-time = "2026-10-10T20:04:05.012Z" },+ { url = "https://files.pythonhosted.org/packages/94/06/1f9c24db48eef0c2d1207e3b11fffb0478e39dfd8c1e1be7476936885eed/numpy-2.5.4-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:98b053943e5a0474ec0da309d2cb9d3f18ea57f8a2067c2ab7b5f763d1068380", size = 5574615, upload-time = "2026-10-10T20:04:07.316Z" },+ { url = "https://files.pythonhosted.org/packages/da/0f/593fba2e1560e949123bc7d2fc48b5893d56e58cd4bd5a273d2fbf60b220/numpy-2.5.4-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:b64a85f40e154983960a4167d4c1d57a50c7f109b3d3264a3a984154e90a8454", size = 6889383, upload-time = "2026-10-10T20:04:09.918Z" },+ { url = "https://files.pythonhosted.org/packages/eb/9f/b799dfdce4e05e80ed4bc815c71ff343a11533b2c0ffc221cae8538cda63/numpy-2.5.4-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a813ed7719bf45463c51779e6a98d0385fe905e48447526938a4b8337333d551", size = 15753763, upload-time = "2026-10-10T20:04:12.278Z" },+ { url = "https://files.pythonhosted.org/packages/34/88/16c5f12f86f5ad2817c4d103205131fc6c8acb3d1878af05a1a4f23ec859/numpy-2.5.4-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c9b80cdf5cedba0e90d93fa5f9a333c4d65bd545cd669b71bb97ce2b703c9d73", size = 16757212, upload-time = "2026-10-10T20:04:14.799Z" },+ { url = "https://files.pythonhosted.org/packages/ff/4f/a1fe40e18a898e6a5089f4f0d891f0a493eb0574d5b34458f0fbe5aa3e5c/numpy-2.5.4-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:2199ed071f460487c8db2c0e5c0b564494190edb4772fe80f9aad88b2604def5", size = 17116471, upload-time = "2026-10-10T20:04:17.58Z" },+ { url = "https://files.pythonhosted.org/packages/aa/46/e923a11c78e65c1722e7aaad817c06bd591324174b9d28ce5d31eee4d432/numpy-2.5.4-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:64f9c9878c1938476365e11ccfb6b770f3b9e5f045ccddc514235041e6959365", size = 18524063, upload-time = "2026-10-10T20:04:20.365Z" },+ { url = "https://files.pythonhosted.org/packages/5a/fa/84ab064514440c1f64a1b21088f2c82756defdd05e07c75ab233899565b2/numpy-2.5.4-cp314-cp314t-win32.whl", hash = "sha256:64d1c8ac28a4077cf987e0a71a7a0ef7e2df70722f07f0baa42dbb7eb6938647", size = 6340926, upload-time = "2026-10-10T20:04:22.865Z" },+ { url = "https://files.pythonhosted.org/packages/7e/7e/6cd886876f435b10685db9b9f7eeb70356f99e052116f4e5f11c5792c714/numpy-2.5.4-cp314-cp314t-win_amd64.whl", hash = "sha256:067374eb538c34c745436365cf7b0112595c1d326f21ce4ff340f61230239fbb", size = 12901584, upload-time = "2026-10-10T20:04:24.99Z" },+ { url = "https://files.pythonhosted.org/packages/38/1b/3c1684f6a06f7307f2335fca6e486cb162847fb97e91d65f8eb5cabad213/numpy-2.5.4-cp314-cp314t-win_arm64.whl", hash = "sha256:e94aef2c639da4a960ad0db8e06471208d8589974953d78b61d345b4eb99e394", size = 10891152, upload-time = "2026-10-10T20:04:27.52Z" },+ { url = "https://files.pythonhosted.org/packages/08/f4/3224deff3af2bef6bc0b175369698d8cb348f3d91d9bb0286cd5c9eae9e0/numpy-2.5.4-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:8dddfbee2e68d26d0d7d7d9cb247b1fd4409241cce32d815a11d97ec2cfde179", size = 17003231, upload-time = "2026-10-10T20:04:30.021Z" },+ { url = "https://files.pythonhosted.org/packages/be/75/fee0b8c6d94b44b2fdfae74f6a4ad5a138739589a8aebaec28ce4e713ed5/numpy-2.5.4-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:81e3420b27048b65eb14c3acf0c174a8cb0e023277716110347d2dcb26026dad", size = 12018300, upload-time = "2026-10-10T20:04:32.519Z" },+ { url = "https://files.pythonhosted.org/packages/47/c0/d0b335a499a04b65f532c3f034346ef390f81299060f928492dabc1e0272/numpy-2.5.4-cp315-cp315-macosx_14_0_arm64.whl", hash = "sha256:0b4724a19de67bea8cfc4970798efa78bcbbe2ac2613cfac16721a42d44de2a5", size = 5454250, upload-time = "2026-10-10T20:04:34.943Z" },+ { url = "https://files.pythonhosted.org/packages/5a/0e/461b3783c03d668052e6a21b01b673db6ffcb7831fd32d9aa5368c1cd426/numpy-2.5.4-cp315-cp315-macosx_14_0_x86_64.whl", hash = "sha256:2132418bf8dd124a427ca9e6a1daf9ee1a87185344c95119ceae868b99466da1", size = 6789644, upload-time = "2026-10-10T20:04:37.258Z" },+ { url = "https://files.pythonhosted.org/packages/b3/02/5dad269b02166965a7b4ca14adaddd75dbee0de42435bfecf561b84ba5a6/numpy-2.5.4-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:325518d4245b9e331387702aa58c2ce1dc4cdcbb41dfb4ccd5dcbc7e08db1266", size = 15704353, upload-time = "2026-10-10T20:04:39.616Z" },+ { url = "https://files.pythonhosted.org/packages/93/3a/01360c8036822ed9f7aa32189a77d1476567ec1e8e1383522389e4faac45/numpy-2.5.4-cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:56733449d2544178beaa4545cee357370440cf056c197f9c7bfb19dbfdd0e86d", size = 16718648, upload-time = "2026-10-10T20:04:42.383Z" },+ { url = "https://files.pythonhosted.org/packages/7d/5c/b863a2c093c4d6f21a597fcaf24ead0835c09ab16a8312d5a5a8868af683/numpy-2.5.4-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:5ec3753760c1a6d8bb91200666e545c3a9728e6269dfb5d6ce02340996698aa3", size = 17059053, upload-time = "2026-10-10T20:04:44.976Z" },+ { url = "https://files.pythonhosted.org/packages/0a/60/ced4f57f9a1258a0af74f17cb0b0c2700b5c67cd6678823c803b263e4df3/numpy-2.5.4-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:b1185012870173de7ae33d370bd45b1cf5baee747ea4b97036b65f4e93016877", size = 18477406, upload-time = "2026-10-10T20:04:47.863Z" },+ { url = "https://files.pythonhosted.org/packages/f9/bd/0ef22dafaafcc7d4bb3ca26b8d2afbd55dedad8eaba99a8c864e1997456f/numpy-2.5.4-cp315-cp315-win32.whl", hash = "sha256:298eca75243f2cbbfdb460560b9fb2a1792a33cf2ab4286efd43d92e8d3df508", size = 6185133, upload-time = "2026-10-10T20:04:50.467Z" },+ { url = "https://files.pythonhosted.org/packages/50/bc/d2651b155ecc608a77e6f4d15495c11f14f19bb98f8bf0c5b0d38f86dda1/numpy-2.5.4-cp315-cp315-win_amd64.whl", hash = "sha256:332f3378fe077dd850e677ec01bdcc4f22368fb5d50ef10b2c79230b1bf5a592", size = 12703085, upload-time = "2026-10-10T20:04:52.63Z" },+ { url = "https://files.pythonhosted.org/packages/dc/d2/45e404f8abb26fb9eda12b94012936873e827b1be76f2ee7890be128312e/numpy-2.5.4-cp315-cp315-win_arm64.whl", hash = "sha256:d4cccbbc78717966f764cd3af4fb70276fa01fc7a2688af11c78901fa5c04f05", size = 10801451, upload-time = "2026-10-10T20:04:55.677Z" },+ { url = "https://files.pythonhosted.org/packages/c6/c3/2ae14e09cfdb67dc187a342e15308a21c15bf4d2071f8079e6aee5fe56dc/numpy-2.5.4-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:950ea81d57ef070665581b6e1b5f6a029306423cd1739c5b95fe78aa30db6b9d", size = 17097121, upload-time = "2026-10-10T20:04:58.403Z" },+ { url = "https://files.pythonhosted.org/packages/f5/cf/305ae624ef8a039414317224abe9ec9c2fe7ea3c2e1cf204d43ff6b2ffb9/numpy-2.5.4-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:c05ede731b03fb1b7591faca9389ade3267d2bddf1ad8882bb3f2cc5e101694f", size = 12135439, upload-time = "2026-10-10T20:05:01.65Z" },+ { url = "https://files.pythonhosted.org/packages/a9/a8/f75c63813aef95827bb2c0d13b12803016853056e8792c280058cdbfe783/numpy-2.5.4-cp315-cp315t-macosx_14_0_arm64.whl", hash = "sha256:5fbf7141bbfd63aea22f435c9062a032b9ea0082fe9845dad7f021d3f1234e71", size = 5571451, upload-time = "2026-10-10T20:05:04.135Z" },+ { url = "https://files.pythonhosted.org/packages/6f/0f/f17763f983868b5c49b4101ebd7e00760bd1769478a6bb6a8de6e085bbac/numpy-2.5.4-cp315-cp315t-macosx_14_0_x86_64.whl", hash = "sha256:3573cd22564692a5b899ec344e5d5b9cc4576f2985b96f22af3564ed54f2710f", size = 6883356, upload-time = "2026-10-10T20:05:06.249Z" },+ { url = "https://files.pythonhosted.org/packages/67/a7/8af04c5a79e047996cfa38854dcfbececdd0343a7c933a46fdd03ef6f5da/numpy-2.5.4-cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6c109eac9cd439193678f69d70733c1108487546ca8eafc107b510ae10c1aecd", size = 15750991, upload-time = "2026-10-10T20:05:08.376Z" },+ { url = "https://files.pythonhosted.org/packages/57/7a/648254290d0c504faa8f2d07aa206660c728802c781a6f3fc68ab7cb5d71/numpy-2.5.4-cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:80d6ef6e8620eb2c2b4c4caad50b5935d6db3cde2d51581b55dcc79e14016d1d", size = 16757675, upload-time = "2026-10-10T20:05:11.393Z" },+ { url = "https://files.pythonhosted.org/packages/b8/fe/4a8c3cdb0c70400cfe4c5bec42d3099a5673802a95064614b33e07b82aa1/numpy-2.5.4-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:77045a4b175bbf5316ec08003880804336c78f92281a1b72222b274ea85ec5ac", size = 17113846, upload-time = "2026-10-10T20:05:14.49Z" },+ { url = "https://files.pythonhosted.org/packages/1b/7e/619692bb67778702c0e9eb2d468568a7573f4e269386ea61aed01ee4e557/numpy-2.5.4-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:0f02a46e49cfb6c73bdb7aea1c0d3461dbae9aba613542b65f657cd3d17b9fab", size = 18522915, upload-time = "2026-10-10T20:05:17.33Z" },+ { url = "https://files.pythonhosted.org/packages/b7/b5/4da41c328788f575838f97a098fe8ca691ebc6f6fd73ad4a262ee40b184d/numpy-2.5.4-cp315-cp315t-win32.whl", hash = "sha256:ad62a416ddcf863bf44bba76fbf6b53366ab0692e294f51cae4b5fbe0d246788", size = 6335804, upload-time = "2026-10-10T20:05:19.921Z" },+ { url = "https://files.pythonhosted.org/packages/98/94/6482ddfa3d312490cb9358f375bf2ad56427dbea8769187158e94d653753/numpy-2.5.4-cp315-cp315t-win_amd64.whl", hash = "sha256:38f47be9f74ab870d2633b5456ae519c43758a8d1fd05342f0ce4ecc034396ee", size = 12890095, upload-time = "2026-10-10T20:05:21.875Z" },+ { url = "https://files.pythonhosted.org/packages/48/7f/c2d1b436b6e7cfebac140c2579a298344b85f2991a2ce5c3615cefb29400/numpy-2.5.4-cp315-cp315t-win_arm64.whl", hash = "sha256:7a14a461d9340f1b46b8648578aed9cdb8b3b018a8fac6c1dde2c9192a01a87f", size = 10883718, upload-time = "2026-10-10T20:05:28.547Z" },+]++[[package]]+name = "nvidia-cublas"+version = "13.1.1.3"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-cuda-nvrtc" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/a7/a1/0bd24ee8c8d03adac032fd2909426a00c88f8c57961b1277ded97f91119f/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:b7a210458267ac818974c53038fbec2e969d5c99f305ab15c72522fa9f001dd5", size = 542848918, upload-time = "2026-04-08T18:46:22.985Z" },+ { url = "https://files.pythonhosted.org/packages/3b/cd/154ca20c38269e05eff77c1464e6c1da89f50a6390b565e9d82e06bc11e1/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:37936a16db8fe4ac1f065c2139360608a543a09275cb1a1af612e08cfa065436", size = 423138758, upload-time = "2026-04-08T18:46:58.655Z" },+]++[[package]]+name = "nvidia-cuda-cupti"+version = "13.0.85"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/2a/2a/80353b103fc20ce05ef51e928daed4b6015db4aaa9162ed0997090fe2250/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_aarch64.whl", hash = "sha256:796bd679890ee55fb14a94629b698b6db54bcfd833d391d5e94017dd9d7d3151", size = 10310827, upload-time = "2025-09-04T08:26:42.012Z" },+ { url = "https://files.pythonhosted.org/packages/33/6d/737d164b4837a9bbd202f5ae3078975f0525a55730fe871d8ed4e3b952b0/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl", hash = "sha256:4eb01c08e859bf924d222250d2e8f8b8ff6d3db4721288cf35d14252a4d933c8", size = 10715597, upload-time = "2025-09-04T08:26:51.312Z" },+]++[[package]]+name = "nvidia-cuda-nvrtc"+version = "13.0.88"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/c3/68/483a78f5e8f31b08fb1bb671559968c0ca3a065ac7acabfc7cee55214fd6/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:ad9b6d2ead2435f11cbb6868809d2adeeee302e9bb94bcf0539c7a40d80e8575", size = 90215200, upload-time = "2025-09-04T08:28:44.204Z" },+ { url = "https://files.pythonhosted.org/packages/b7/dc/6bb80850e0b7edd6588d560758f17e0550893a1feaf436807d64d2da040f/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d27f20a0ca67a4bb34268a5e951033496c5b74870b868bacd046b1b8e0c3267b", size = 43015449, upload-time = "2025-09-04T08:28:20.239Z" },+]++[[package]]+name = "nvidia-cuda-runtime"+version = "13.0.96"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/87/4f/17d7b9b8e285199c58ce28e31b5c5bbaa4d8271af06a89b6405258245de2/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:ef9bcbe90493a2b9d810e43d249adb3d02e98dd30200d86607d8d02687c43f55", size = 2261060, upload-time = "2025-10-09T08:55:15.78Z" },+ { url = "https://files.pythonhosted.org/packages/2e/24/d1558f3b68b1d26e706813b1d10aa1d785e4698c425af8db8edc3dced472/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7f82250d7782aa23b6cfe765ecc7db554bd3c2870c43f3d1821f1d18aebf0548", size = 2243632, upload-time = "2025-10-09T08:55:36.117Z" },+]++[[package]]+name = "nvidia-cudnn-cu13"+version = "9.24.0.43"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-cublas" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/ca/30/7c257e3d5cb4fecb147b93895c66e29c93f8e76d74b45bb418ff0587c4ec/nvidia_cudnn_cu13-9.24.0.43-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:a6812a554a1ff0413e9c52b84c26c050380649ab9615f9c16bded368ce9f421f", size = 650976863, upload-time = "2026-07-02T16:23:39.248Z" },+ { url = "https://files.pythonhosted.org/packages/5c/ba/791cffd048fe5b044e620df55267e3e95c0e6e07d50b41e377c03dfc910f/nvidia_cudnn_cu13-9.24.0.43-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:71f181cd810e90f9b6023b01186fe82d13d65f0ec098581ee201d39fad769e4b", size = 553099438, upload-time = "2026-07-02T16:27:42.58Z" },+]++[[package]]+name = "nvidia-cufft"+version = "12.0.0.61"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-nvjitlink" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/8b/ae/f417a75c0259e85c1d2f83ca4e960289a5f814ed0cea74d18c353d3e989d/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2708c852ef8cd89d1d2068bdbece0aa188813a0c934db3779b9b1faa8442e5f5", size = 214053554, upload-time = "2025-09-04T08:31:38.196Z" },+ { url = "https://files.pythonhosted.org/packages/a8/2f/7b57e29836ea8714f81e9898409196f47d772d5ddedddf1592eadb8ab743/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:6c44f692dce8fd5ffd3e3df134b6cdb9c2f72d99cf40b62c32dde45eea9ddad3", size = 214085489, upload-time = "2025-09-04T08:31:56.044Z" },+]++[[package]]+name = "nvidia-cufile"+version = "1.15.1.6"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/3f/70/4f193de89a48b71714e74602ee14d04e4019ad36a5a9f20c425776e72cd6/nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:08a3ecefae5a01c7f5117351c64f17c7c62efa5fffdbe24fc7d298da19cd0b44", size = 1223672, upload-time = "2025-09-04T08:32:22.779Z" },+ { url = "https://files.pythonhosted.org/packages/ab/73/cc4a14c9813a8a0d509417cf5f4bdaba76e924d58beb9864f5a7baceefbf/nvidia_cufile-1.15.1.6-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:bdc0deedc61f548bddf7733bdc216456c2fdb101d020e1ab4b88d232d5e2f6d1", size = 1136992, upload-time = "2025-09-04T08:32:14.119Z" },+]++[[package]]+name = "nvidia-curand"+version = "10.4.0.35"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/1e/72/7c2ae24fb6b63a32e6ae5d241cc65263ea18d08802aaae087d9f013335a2/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:133df5a7509c3e292aaa2b477afd0194f06ce4ea24d714d616ff36439cee349a", size = 61962106, upload-time = "2025-08-04T10:21:41.128Z" },+ { url = "https://files.pythonhosted.org/packages/a5/9f/be0a41ca4a4917abf5cb9ae0daff1a6060cc5de950aec0396de9f3b52bc5/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:1aee33a5da6e1db083fe2b90082def8915f30f3248d5896bcec36a579d941bfc", size = 59544258, upload-time = "2025-08-04T10:22:03.992Z" },+]++[[package]]+name = "nvidia-cusolver"+version = "12.0.4.66"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-cublas" },+ { name = "nvidia-cusparse" },+ { name = "nvidia-nvjitlink" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/c8/c3/b30c9e935fc01e3da443ec0116ed1b2a009bb867f5324d3f2d7e533e776b/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:02c2457eaa9e39de20f880f4bd8820e6a1cfb9f9a34f820eb12a155aa5bc92d2", size = 223467760, upload-time = "2025-09-04T08:33:04.222Z" },+ { url = "https://files.pythonhosted.org/packages/5f/67/cba3777620cdacb99102da4042883709c41c709f4b6323c10781a9c3aa34/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:0a759da5dea5c0ea10fd307de75cdeb59e7ea4fcb8add0924859b944babf1112", size = 200941980, upload-time = "2025-09-04T08:33:22.767Z" },+]++[[package]]+name = "nvidia-cusparse"+version = "12.6.3.3"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-nvjitlink" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/f8/94/5c26f33738ae35276672f12615a64bd008ed5be6d1ebcb23579285d960a9/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:80bcc4662f23f1054ee334a15c72b8940402975e0eab63178fc7e670aa59472c", size = 162155568, upload-time = "2025-09-04T08:33:42.864Z" },+ { url = "https://files.pythonhosted.org/packages/fa/18/623c77619c31d62efd55302939756966f3ecc8d724a14dab2b75f1508850/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:2b3c89c88d01ee0e477cb7f82ef60a11a4bcd57b6b87c33f789350b59759360b", size = 145942937, upload-time = "2025-09-04T08:33:58.029Z" },+]++[[package]]+name = "nvidia-cusparselt-cu13"+version = "0.8.1"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/46/e1/cdc1797eadf82d3a9a575a19b33fdc871a97edbec42c00b5b5e914f4aff4/nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_aarch64.whl", hash = "sha256:4dca476c50bf4780d46cd0bfbd82e2bc10a08e4fef7950917ce8d7578d22a23f", size = 221051344, upload-time = "2025-09-05T18:49:51.289Z" },+ { url = "https://files.pythonhosted.org/packages/34/7d/2661f2fb3ac4302f3a246f5fc030213ac60c1fe0bce84f9783dbd831dbb7/nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_x86_64.whl", hash = "sha256:786ce87568c303fadb5afcc7102d454cd3040d75f6f8626f5db460d1871f4dd0", size = 170148586, upload-time = "2025-09-05T18:50:50.248Z" },+]++[[package]]+name = "nvidia-nccl-cu13"+version = "2.30.7"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d1/21/a73174c6157101bdf1ffc22b517f76ff0082613989dd9bc8f43e8034caac/nvidia_nccl_cu13-2.30.7-py3-none-manylinux_2_18_aarch64.whl", hash = "sha256:ca786ffa5a647c75d4d1f5cc72a6c4f537947e2ba8823d7c8aaf768e7a7b9f77", size = 215983881, upload-time = "2026-06-09T03:23:15.633Z" },+ { url = "https://files.pythonhosted.org/packages/3f/34/c500f90c7ae641b8e0f98965b36b8a7ac79cc8b296e8d251fe3eb592ee54/nvidia_nccl_cu13-2.30.7-py3-none-manylinux_2_18_x86_64.whl", hash = "sha256:cefa7fdb9710efd0f39c5f1be1d61ff6fc9a996c451265bd7fbdcf9455ed4b50", size = 215965170, upload-time = "2026-06-09T03:23:39.73Z" },+]++[[package]]+name = "nvidia-nvjitlink"+version = "13.4.92"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/1d/6b/eef7a9e32872b8f41e145bf10cddc9af26e153c338852811fe9a9baddf9e/nvidia_nvjitlink-13.4.92-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:e0391f24ed94ec879b84e3da4d4ec320c879aff681f2c7a638462f7199284323", size = 42452378, upload-time = "2026-09-16T20:45:29.042Z" },+ { url = "https://files.pythonhosted.org/packages/1f/a8/1cbd4014898af8b419e69b0d7dbc63da2121ee92d92b47d59f4fe9075349/nvidia_nvjitlink-13.4.92-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:25f74fad0d654271c921ac4dca614bd6258bc21791242fc7b2289dad7ae9c099", size = 40420120, upload-time = "2026-09-16T20:45:19.163Z" },+]++[[package]]+name = "nvidia-nvshmem-cu13"+version = "3.4.5"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/dc/0f/05cc9c720236dcd2db9c1ab97fff629e96821be2e63103569da0c9b72f19/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:6dc2a197f38e5d0376ad52cd1a2a3617d3cdc150fd5966f4aee9bcebb1d68fe9", size = 60215947, upload-time = "2025-09-06T00:32:20.022Z" },+ { url = "https://files.pythonhosted.org/packages/3c/35/a9bf80a609e74e3b000fef598933235c908fcefcef9026042b8e6dfde2a9/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:290f0a2ee94c9f3687a02502f3b9299a9f9fe826e6d0287ee18482e78d495b80", size = 60412546, upload-time = "2025-09-06T00:32:41.564Z" },+]++[[package]]+name = "nvidia-nvtx"+version = "13.0.85"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/c2/f3/d86c845465a2723ad7e1e5c36dcd75ddb82898b3f53be47ebd429fb2fa5d/nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:4936d1d6780fbe68db454f5e72a42ff64d1fd6397df9f363ae786930fd5c1cd4", size = 148047, upload-time = "2025-09-04T08:29:01.761Z" },+ { url = "https://files.pythonhosted.org/packages/a8/64/3708a90d1ebe202ffdeb7185f878a3c84d15c2b2c31858da2ce0583e2def/nvidia_nvtx-13.0.85-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:cb7780edb6b14107373c835bf8b72e7a178bac7367e23da7acb108f973f157a6", size = 148878, upload-time = "2025-09-04T08:28:53.627Z" },+]++[[package]]+name = "packaging"+version = "26.3"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/7d/fa/3944b40b07da9ce895c0e6303a5ab7d53da063554f534556b134a54d6093/packaging-26.3.tar.gz", hash = "sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79", size = 313412, upload-time = "2026-08-04T18:15:28.737Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", size = 129956, upload-time = "2026-08-04T18:15:27.159Z" },+]++[[package]]+name = "pillow"+version = "12.3.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/1c/3d/bb7fca845737cf9d7dbde16ed1843984665ff2e0a518f5db43e77ec540b9/pillow-12.3.0.tar.gz", hash = "sha256:3b8182a766685eaa002637e28b4ec8d6b18819a0c71f579bf0dbaa5830297cce", size = 47025035, upload-time = "2026-07-01T11:56:38.965Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/fb/c8/0a78b0e02d7ac54bc03e5321c9220da52f0c2ea83b21f7c40e7f3169c502/pillow-12.3.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756", size = 5392415, upload-time = "2026-07-01T11:53:47.162Z" },+ { url = "https://files.pythonhosted.org/packages/b2/5b/a02d30018abd97ced9f5a6c63d28597694a00d066516b9c1c6de45859fc9/pillow-12.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6", size = 4785266, upload-time = "2026-07-01T11:53:49.079Z" },+ { url = "https://files.pythonhosted.org/packages/c8/98/766667a4be768150a202836acd9fad19c06824ca86c4286d3cf6b274964e/pillow-12.3.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd", size = 6263814, upload-time = "2026-07-01T11:53:51.32Z" },+ { url = "https://files.pythonhosted.org/packages/3b/2d/ede717bc1144f63886c21fd349bb95860b0d1a21149ff16f2bb362b612b6/pillow-12.3.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd", size = 6934408, upload-time = "2026-07-01T11:53:53.487Z" },+ { url = "https://files.pythonhosted.org/packages/a3/48/9c58b685e69d49c31af6c8eb9012055fab7e665785165c84796e2c73ce72/pillow-12.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c", size = 6337160, upload-time = "2026-07-01T11:53:55.457Z" },+ { url = "https://files.pythonhosted.org/packages/ff/fa/dc2a5c0ba6df93f67c31d34b808b7ce440b40cdbf96f0b81cde1d1e6fa93/pillow-12.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5", size = 7045172, upload-time = "2026-07-01T11:53:57.736Z" },+ { url = "https://files.pythonhosted.org/packages/86/a5/444817a4d4c4c2417df00513086ca196f388d8f9ef40c2e4ccd1ad1af54b/pillow-12.3.0-cp311-cp311-win32.whl", hash = "sha256:10e41f0fbf1eec8cfd234b8fe17a4caac7c9d0db4c204d3c173a8f9f6ef3232b", size = 6472232, upload-time = "2026-07-01T11:53:59.767Z" },+ { url = "https://files.pythonhosted.org/packages/63/c6/4bad1b18d132a50b27e1365e1ab163616f7a5bb56d330f66f9d1d9d4f9d4/pillow-12.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:8e95e1385e4998ae9694eeaa4730ba5457ff61185b3a55e2e7bea0880aef452a", size = 7233653, upload-time = "2026-07-01T11:54:02.066Z" },+ { url = "https://files.pythonhosted.org/packages/fd/16/00f91ab7760dc842f5aad55217e80fc4a7067a0604535249bc8a2d6d9870/pillow-12.3.0-cp311-cp311-win_arm64.whl", hash = "sha256:ebaea975e03d3141d9d3a507df75c9b3ec90fa9d2ffd07567b3a978d9d790b26", size = 2568195, upload-time = "2026-07-01T11:54:04.622Z" },+ { url = "https://files.pythonhosted.org/packages/37/bf/fb3ebff8ddcb76aac5a01389251bbbb9519922a9b520d8247c1ca864a25d/pillow-12.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965", size = 5345969, upload-time = "2026-07-01T11:54:06.397Z" },+ { url = "https://files.pythonhosted.org/packages/d8/66/9a386a92561f402389a4fc70c18838bf6d35eb5eb5c6850b4b2dc64f5048/pillow-12.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7", size = 4780323, upload-time = "2026-07-01T11:54:09.351Z" },+ { url = "https://files.pythonhosted.org/packages/25/27/ac8f99618ffd3dde21db0f4d4b1d2ab00c0880595bfd17df103f7f39fd0c/pillow-12.3.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9", size = 6266838, upload-time = "2026-07-01T11:54:11.71Z" },+ { url = "https://files.pythonhosted.org/packages/84/21/a35af28dcc61f37ed850a2d64c65c701321dfbf25085e469d5559360cbbf/pillow-12.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91", size = 6940830, upload-time = "2026-07-01T11:54:13.732Z" },+ { url = "https://files.pythonhosted.org/packages/eb/51/8b08617af3ad95e33ce6d7dd2c99ed6c8298f7fb131636303956be022e25/pillow-12.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c", size = 6344383, upload-time = "2026-07-01T11:54:15.756Z" },+ { url = "https://files.pythonhosted.org/packages/1d/72/cf78ac9780bb93c28328f408973845a309d4d145041665f734572ced1b52/pillow-12.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df", size = 7052934, upload-time = "2026-07-01T11:54:17.721Z" },+ { url = "https://files.pythonhosted.org/packages/20/20/25e0f4dc178a6bc0696793720055519a0de89e7661dae886992decbd2f81/pillow-12.3.0-cp312-cp312-win32.whl", hash = "sha256:dbce0b29841537a2fa4a214c2bbf14de3587c9680caa9b4e217568472490b28f", size = 6472684, upload-time = "2026-07-01T11:54:19.839Z" },+ { url = "https://files.pythonhosted.org/packages/45/89/da2f7971a317f83d807fdd4065c0af40208e59e692cc43d315a71a0e96d1/pillow-12.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:a2b55dd6b2a4c4b7d87ffa56bdb33fdc5fdb9a462173861a7bc097f17d91cb09", size = 7227137, upload-time = "2026-07-01T11:54:22.025Z" },+ { url = "https://files.pythonhosted.org/packages/de/47/4845a0a6c0dbf1db8456bd9fc791f13c5ced7ced20606d08a0aacfd25b49/pillow-12.3.0-cp312-cp312-win_arm64.whl", hash = "sha256:331b624368d4f1d069149002f25f44bc61c8919ce8ddb3c45bdad8f6e2d89510", size = 2568267, upload-time = "2026-07-01T11:54:24.051Z" },+ { url = "https://files.pythonhosted.org/packages/9d/ac/31fb64e1e7efb5a4b50cd3d92049ba89ac6e4d8d3bb6a74e15048ca3353e/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89", size = 4161684, upload-time = "2026-07-01T11:54:25.934Z" },+ { url = "https://files.pythonhosted.org/packages/87/b4/9805e23d2b4d77842b468513841fda254ee42f0289d25088340e4ff46e2d/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace", size = 4255487, upload-time = "2026-07-01T11:54:27.935Z" },+ { url = "https://files.pythonhosted.org/packages/df/39/ecf519435a200c693fe053a6ee4d835b41cf963a4dfc2551c4e637cb2a71/pillow-12.3.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec", size = 3696433, upload-time = "2026-07-01T11:54:29.813Z" },+ { url = "https://files.pythonhosted.org/packages/42/92/2fc3ffad878ae8dd5469ec1bc8eb83b71f48e13efdf68f02709003982a32/pillow-12.3.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66", size = 5345889, upload-time = "2026-07-01T11:54:31.97Z" },+ { url = "https://files.pythonhosted.org/packages/10/76/8803c13605b763d33d156c4678fc77f8443389c0c51c8aef707bb02015f4/pillow-12.3.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35", size = 4780109, upload-time = "2026-07-01T11:54:34.026Z" },+ { url = "https://files.pythonhosted.org/packages/1f/01/e18aff37cb0b4aac47ac90f016d347a49aca667ef97f190b06ac2aabc928/pillow-12.3.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65", size = 6263736, upload-time = "2026-07-01T11:54:36.131Z" },+ { url = "https://files.pythonhosted.org/packages/f7/62/de5bdd77d935331f4f802edc11e4d82950f642caad6cb2f949837b8560e2/pillow-12.3.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3", size = 6937129, upload-time = "2026-07-01T11:54:38.216Z" },+ { url = "https://files.pythonhosted.org/packages/70/4d/105627a13300c5e0df1d174230b32fd1273062c96f7745fd552b945d1e1d/pillow-12.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a", size = 6339562, upload-time = "2026-07-01T11:54:40.354Z" },+ { url = "https://files.pythonhosted.org/packages/6b/1d/f13de01a553988ab895ba1c722e06cf3144d4f57656fd5b81b6d881f1179/pillow-12.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e", size = 7049439, upload-time = "2026-07-01T11:54:42.489Z" },+ { url = "https://files.pythonhosted.org/packages/c9/f9/066794cca041b969964f779ee5fa66a9498bbf34248ac39c5d7954e4198f/pillow-12.3.0-cp313-cp313-win32.whl", hash = "sha256:a876864214e136f0eb367788dbd7df045f4806801518e2cfe9e13229cfe06d8f", size = 6473287, upload-time = "2026-07-01T11:54:44.9Z" },+ { url = "https://files.pythonhosted.org/packages/a6/9b/7a58e61d62be561da3a356fe2384d4059a6345fc130e23ef1c36a5b81d24/pillow-12.3.0-cp313-cp313-win_amd64.whl", hash = "sha256:1cca606cd25738df4ed873d5ad46bbdb3d83b5cbca291f6b4ff13a4df6b0bbe8", size = 7239691, upload-time = "2026-07-01T11:54:47.141Z" },+ { url = "https://files.pythonhosted.org/packages/aa/b0/c4ed4f0ef8f8fa5ee8351537db6650bb8189f7e118842978dd6589065692/pillow-12.3.0-cp313-cp313-win_arm64.whl", hash = "sha256:b629de27fda84b42cde7edef0d85f13b958b47f6e9bbcbba9b673c562a89bd8b", size = 2568185, upload-time = "2026-07-01T11:54:49.137Z" },+ { url = "https://files.pythonhosted.org/packages/dc/01/001f65b68192f0228cc1dbbc8d2530ab5d58b61037ba0587f946fea607cd/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330", size = 4161736, upload-time = "2026-07-01T11:54:51.156Z" },+ { url = "https://files.pythonhosted.org/packages/1a/d2/0219746d0fd16fc8a84498e79452375be3797d3ce4044596ce565164b84f/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217", size = 4255435, upload-time = "2026-07-01T11:54:53.414Z" },+ { url = "https://files.pythonhosted.org/packages/c8/02/8d0bc62ef0302318c46ff2a512822d2610e81c7aa46c9b3abe6cbaca5ad0/pillow-12.3.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930", size = 3696262, upload-time = "2026-07-01T11:54:55.739Z" },+ { url = "https://files.pythonhosted.org/packages/85/e2/73c77d218410b14f5f2d565e8a998d5317b7b9c75368d29985139f7a46f0/pillow-12.3.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8", size = 5350344, upload-time = "2026-07-01T11:54:57.657Z" },+ { url = "https://files.pythonhosted.org/packages/c7/da/32c752228ae345f489e3a42499d817b6c3996da7e8a3bc7a04fc806b243b/pillow-12.3.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0", size = 4780131, upload-time = "2026-07-01T11:54:59.713Z" },+ { url = "https://files.pythonhosted.org/packages/b1/9d/8b2c807dbef61a5197c047afe99823787eb66f63daf9fb2432f91d6f0462/pillow-12.3.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321", size = 6263757, upload-time = "2026-07-01T11:55:01.778Z" },+ { url = "https://files.pythonhosted.org/packages/5c/44/c85361f65dbe00eea8576ee467c768d25129989efb76e94f205e9ca9bb46/pillow-12.3.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b", size = 6936962, upload-time = "2026-07-01T11:55:03.93Z" },+ { url = "https://files.pythonhosted.org/packages/18/7e/e483414b35800b86b6f08dbbc7803fb5cd52c4d6f897f47d53ea2c7e6f65/pillow-12.3.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198", size = 6339171, upload-time = "2026-07-01T11:55:05.989Z" },+ { url = "https://files.pythonhosted.org/packages/f0/f4/68c491844841ede6bed70189546b3ee9731cf9f2cbad396faff5e1ccba45/pillow-12.3.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130", size = 7048116, upload-time = "2026-07-01T11:55:08.131Z" },+ { url = "https://files.pythonhosted.org/packages/a3/34/77f3f793fed8efc7d243f21b33c5a3f0d1c97ee70346d3db855587e155ff/pillow-12.3.0-cp314-cp314-win32.whl", hash = "sha256:af8d94b0db561cf68b88a267c5c44b49e134f525d0dc2cb7ed413a66bc23559a", size = 6467209, upload-time = "2026-07-01T11:55:10.408Z" },+ { url = "https://files.pythonhosted.org/packages/f1/e0/492879f69d94f91f60fc8cd05ba03650e9520afebb2fb7aa12777d7c7f38/pillow-12.3.0-cp314-cp314-win_amd64.whl", hash = "sha256:fdafc9cce40277e0f7a0feabce0ee50dd2fa1800f3b38015e51296b5e814048d", size = 7237707, upload-time = "2026-07-01T11:55:12.745Z" },+ { url = "https://files.pythonhosted.org/packages/c9/ac/6b11f2875f1c2ac040d84e1bbf9cf22a88038f901ca1037898b280b38365/pillow-12.3.0-cp314-cp314-win_arm64.whl", hash = "sha256:e91206ee562682b51b98ef4b26a6ef48fd84e15fd4c4bc5ec768eb641d206838", size = 2565995, upload-time = "2026-07-01T11:55:14.736Z" },+ { url = "https://files.pythonhosted.org/packages/52/69/c2208e56af9bfc1913afb24020297a691eb1d4ef688474c8a04913f65e04/pillow-12.3.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e", size = 5352503, upload-time = "2026-07-01T11:55:17.076Z" },+ { url = "https://files.pythonhosted.org/packages/07/70/e5686d753e898a45d778ff1718dba8516ead6ab6b95d85fc8c4b70650cf2/pillow-12.3.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17", size = 4782956, upload-time = "2026-07-01T11:55:19.448Z" },+ { url = "https://files.pythonhosted.org/packages/d5/37/25c6692f06927ee973ff18c8d9ee98ad0b4d84ee67a09610c2dd1447958e/pillow-12.3.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385", size = 6322855, upload-time = "2026-07-01T11:55:21.613Z" },+ { url = "https://files.pythonhosted.org/packages/cc/91/420637fcb8f1bc11029e403b4538e6694744428d8246118e45719f944556/pillow-12.3.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c", size = 6989642, upload-time = "2026-07-01T11:55:24.006Z" },+ { url = "https://files.pythonhosted.org/packages/10/08/b94d7811281ccf0d143a1cf768d1c49e1e54af63e7b708ab2ee3eb87face/pillow-12.3.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d", size = 6391281, upload-time = "2026-07-01T11:55:26.252Z" },+ { url = "https://files.pythonhosted.org/packages/d2/87/24233f785f55474dc02ce3e739c5528a77e3a862e9333d1dd7a25cc31f70/pillow-12.3.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931", size = 7096716, upload-time = "2026-07-01T11:55:28.318Z" },+ { url = "https://files.pythonhosted.org/packages/23/26/fcb2f6e37175b04f53570b59937867e2b80ee1685e744023153028fc14f9/pillow-12.3.0-cp314-cp314t-win32.whl", hash = "sha256:4b0a7fe987b14c31ebda6083f74f22b561fd3739bc0ac51e019622e3d72668c7", size = 6474125, upload-time = "2026-07-01T11:55:30.956Z" },+ { url = "https://files.pythonhosted.org/packages/90/de/3634abee5f1c9e13c56787b7d5517b0ba8d6de51700b95578cf338349c9f/pillow-12.3.0-cp314-cp314t-win_amd64.whl", hash = "sha256:962864dc93511324d51ddbb5b9f8731bf71675b93ca612a07441896f4688fb8c", size = 7242939, upload-time = "2026-07-01T11:55:34.044Z" },+ { url = "https://files.pythonhosted.org/packages/ce/2a/fd13f8eb24de5714a6eb444a3d67e2842c6c576e159a43793adf23051351/pillow-12.3.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0740a512dc522224c77d9aa5a8d70d8b7d73fb91f2c21125d8d025d3b8990e45", size = 2567506, upload-time = "2026-07-01T11:55:35.988Z" },+ { url = "https://files.pythonhosted.org/packages/5d/dc/8fdce34ec725a33c81c6ba122b904d6b9024e50ea9ac7bede62fab54506c/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139", size = 4162063, upload-time = "2026-07-01T11:55:37.941Z" },+ { url = "https://files.pythonhosted.org/packages/76/66/2044b9a63d3b84ff048228dfcb7cd9bf0df983e8470971bf7d4c57b693de/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402", size = 4255549, upload-time = "2026-07-01T11:55:40.022Z" },+ { url = "https://files.pythonhosted.org/packages/52/7e/1f67e6f4ece6b582ee4b539decbcc9f848dc245a93ed8cd7338bafef72f1/pillow-12.3.0-cp315-cp315-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c", size = 3696331, upload-time = "2026-07-01T11:55:41.98Z" },+ { url = "https://files.pythonhosted.org/packages/12/40/d306fc2c8e4d45d7f175c77edca7063be7b86fe7fe6e68f4353bf71d808c/pillow-12.3.0-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:dc624f6bc473dacdf7ef7eb8678d0d08edf15cd94fad6ae5c7d6cc67a4e4902f", size = 5350370, upload-time = "2026-07-01T11:55:44.028Z" },+ { url = "https://files.pythonhosted.org/packages/dd/44/668fb1437e8ce420f62d6106eb66e44a5971602a4d794615bdf79315d82d/pillow-12.3.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:71d6097b330eea8fd15097780c8e89cb1a8ce7838669f48c5bacd6f663dd4701", size = 4780147, upload-time = "2026-07-01T11:55:46.073Z" },+ { url = "https://files.pythonhosted.org/packages/0c/08/93fa2e70e30a2d81547e481b6ee2bb9522117221fb1e0ce4b5df70967677/pillow-12.3.0-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:28ce87c5ab450a9dd970b52e5aca5fe63ed432d18a2eaddd1979a00a1ba24ace", size = 6273659, upload-time = "2026-07-01T11:55:48.264Z" },+ { url = "https://files.pythonhosted.org/packages/f8/6d/043e96ff814fc31a33077e4cba86082167db520c93632afdf2042febbb0c/pillow-12.3.0-cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6b02afb9b97f65fbca5f31db6a2a3ba21aa93030225f150fa3f249717e938fb4", size = 6947439, upload-time = "2026-07-01T11:55:50.503Z" },+ { url = "https://files.pythonhosted.org/packages/af/92/ba71d2ee2ac0edf3fa33bd9d5ee9ee080da70b1766f3ca3934f9938ddac9/pillow-12.3.0-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:1182d52bc2d5e5d7d0949503aa7e36d12f42205dc287e4883f407b1988820d39", size = 6353577, upload-time = "2026-07-01T11:55:52.697Z" },+ { url = "https://files.pythonhosted.org/packages/0f/ce/e63064e2122923ff687c8ad792d0d736a7b3920a56a46982e81a7fdd25d6/pillow-12.3.0-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:e795b7eb908249c4e43c7c99fac7c2c75dab0c43566e37db472a355f63693d71", size = 7060394, upload-time = "2026-07-01T11:55:55.149Z" },+ { url = "https://files.pythonhosted.org/packages/54/76/a09cc3ccc8d773a7283d34c38bec1708f9e3cc932093cbc4c5e71ac4060b/pillow-12.3.0-cp315-cp315-win32.whl", hash = "sha256:57b3d78c95ba9059768b10e28b813002261d3f3dfc55cc48b0c988f625175827", size = 6467375, upload-time = "2026-07-01T11:55:57.769Z" },+ { url = "https://files.pythonhosted.org/packages/3e/03/1846c49ba3b1d5550392a4bbd06d6fb4578e1cd91a803198b5c90f5f7d53/pillow-12.3.0-cp315-cp315-win_amd64.whl", hash = "sha256:fa4ecea169a355be7a3ade2c783e2ed12f0e40d2c5621cda8b3297faf7fbb9f5", size = 7237048, upload-time = "2026-07-01T11:55:59.975Z" },+ { url = "https://files.pythonhosted.org/packages/fb/bb/89f35dcc79610423f9f195504d7def7f0d1416a711541b42867e25fe3412/pillow-12.3.0-cp315-cp315-win_arm64.whl", hash = "sha256:877c3f311ff35410f690861c4409e7ccbf0cd2f878e50628a28e5a0bb689e658", size = 2566006, upload-time = "2026-07-01T11:56:02.143Z" },+ { url = "https://files.pythonhosted.org/packages/30/88/707027ba09942dfa2c28759b5c222d769290a41c6d20ea60ec250801941f/pillow-12.3.0-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:e9871b1ffbfa9656b60aeee92ed5136a5742696006fa322b29ea3d8da0ecc9cf", size = 5352509, upload-time = "2026-07-01T11:56:04.2Z" },+ { url = "https://files.pythonhosted.org/packages/b0/6d/00352fa25332c2569cd387851f568cc5a4b75a9adbfb37ac4fbce4c02eec/pillow-12.3.0-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:53aa02d20d10c3d814d536aa4e5ac9b84ca0ff5a88377963b085ad6822f93e64", size = 4783167, upload-time = "2026-07-01T11:56:06.631Z" },+ { url = "https://files.pythonhosted.org/packages/13/4f/9e049dfa21af7c22427275720e2490267ba8138120add5c4c574deb69782/pillow-12.3.0-cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:446c34dcc4324b084a53b705127dc15717b22c5e140ae0a3c38349d4efec071e", size = 6329237, upload-time = "2026-07-01T11:56:08.868Z" },+ { url = "https://files.pythonhosted.org/packages/36/16/cf6eeaae8d0fce8dd390a33437cf68c5d5bd73834a2bc6e2f14efda0ab45/pillow-12.3.0-cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cf1845d02ad822a369a49f2bb9345b1614744267682e7a03527dc3bf6eea1777", size = 6997047, upload-time = "2026-07-01T11:56:11.379Z" },+ { url = "https://files.pythonhosted.org/packages/1e/69/dbf769bdd55f48bf5733cac28edc6364ffaa072ec9ba336266e4fe66be55/pillow-12.3.0-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:186941b6aef820ad110fb01fb06eb925374dc3a21b17e37ec9a53b250c6fe2d1", size = 6400440, upload-time = "2026-07-01T11:56:13.908Z" },+ { url = "https://files.pythonhosted.org/packages/a0/e1/ffc9cfc2eea0d178da8018e18e959301ad9d6bc9f3edb7181e748a474b97/pillow-12.3.0-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:f13c32a3abd6079a66d9526e18dad9b6d280384d49d7c54040cd57b6424041d9", size = 7105895, upload-time = "2026-07-01T11:56:16.575Z" },+ { url = "https://files.pythonhosted.org/packages/18/f0/a5595c1e8c3ae44b9828cb2f0fa8155e5095ef04d6327b8f61cf44a3df85/pillow-12.3.0-cp315-cp315t-win32.whl", hash = "sha256:1657923d2d45afb66526e5b933e5b3052e6bdea196c90d3abb2424e18c77dae8", size = 6474384, upload-time = "2026-07-01T11:56:18.855Z" },+ { url = "https://files.pythonhosted.org/packages/e4/04/62bcd9f844984c5938d3b05264a61d797a29d3e0812341a8204af70bbdee/pillow-12.3.0-cp315-cp315t-win_amd64.whl", hash = "sha256:8cd2f7bdda092d99c9fc2fb7391354f306d01443d22785d0cbfafa2e2c8bb418", size = 7243537, upload-time = "2026-07-01T11:56:21.214Z" },+ { url = "https://files.pythonhosted.org/packages/3d/68/1f3066acedf37673694a7141381d8f811ae97f30d34413d236abe7d489f1/pillow-12.3.0-cp315-cp315t-win_arm64.whl", hash = "sha256:06ff022112bc9cbf83b60f8e028d94ad87b60621706487e65f673de61610ab59", size = 2567491, upload-time = "2026-07-01T11:56:23.506Z" },+ { url = "https://files.pythonhosted.org/packages/75/18/2e8b40223153ccbc60df07f9e8928dc0c76202aa4e55ae9f53962b6510d6/pillow-12.3.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:b3c777e849237620b022f7f297dd67705f9f5cf1685f09f02e46f93e92725468", size = 5302510, upload-time = "2026-07-01T11:56:25.736Z" },+ { url = "https://files.pythonhosted.org/packages/46/3e/51fabf59d5ab801ceab709453d3ab6b180083496579549de4c45ced6528a/pillow-12.3.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:b343699e8308bdc51978310e1c959c584e7869cc8c40780058c87da7781a1e94", size = 4736058, upload-time = "2026-07-01T11:56:28.041Z" },+ { url = "https://files.pythonhosted.org/packages/bf/20/22fe9384b7949e25fb1293bcfc84fb82590ff4ea6b37c95b24d26d793d86/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbd139c8447d25dd750ab79ee274cc5e1fe80fc56340ab10b18a195e1b6eca3e", size = 5237776, upload-time = "2026-07-01T11:56:30.263Z" },+ { url = "https://files.pythonhosted.org/packages/08/14/f6ba68107680ffa74b39985f3f30884e41318fbc4250caa423c79b4788bb/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e7e480451b9fa137494bccd3a7d69adbe8ac65a87d97be61e11f1b1050a5bac3", size = 5860358, upload-time = "2026-07-01T11:56:32.68Z" },+ { url = "https://files.pythonhosted.org/packages/36/54/0169bc772ec491108b62f644f8ecf1fe5d8ae5ebafde2ee2142210166903/pillow-12.3.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:04f01d28a6aaff387bf842a13be313df23ba0597a44f1a976c9feb3c6ff4711a", size = 7231786, upload-time = "2026-07-01T11:56:35.046Z" },+]++[[package]]+name = "pluggy"+version = "1.6.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },+]++[[package]]+name = "pygame"+version = "2.6.1"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/49/cc/08bba60f00541f62aaa252ce0cfbd60aebd04616c0b9574f755b583e45ae/pygame-2.6.1.tar.gz", hash = "sha256:56fb02ead529cee00d415c3e007f75e0780c655909aaa8e8bf616ee09c9feb1f", size = 14808125, upload-time = "2024-09-29T13:41:34.698Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/c4/ca/8f367cb9fe734c4f6f6400e045593beea2635cd736158f9fabf58ee14e3c/pygame-2.6.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:20349195326a5e82a16e351ed93465a7845a7e2a9af55b7bc1b2110ea3e344e1", size = 13113753, upload-time = "2024-09-29T14:26:13.751Z" },+ { url = "https://files.pythonhosted.org/packages/83/47/6edf2f890139616b3219be9cfcc8f0cb8f42eb15efd59597927e390538cb/pygame-2.6.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:f3935459109da4bb0b3901da9904f0a3e52028a3332a355d298b1673a334cf21", size = 12378146, upload-time = "2024-09-29T14:26:22.456Z" },+ { url = "https://files.pythonhosted.org/packages/00/9e/0d8aa8cf93db2d2ee38ebaf1c7b61d0df36ded27eb726221719c150c673d/pygame-2.6.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c31dbdb5d0217f32764797d21c2752e258e5fb7e895326538d82b5f75a0cd856", size = 13611760, upload-time = "2024-09-29T11:10:47.317Z" },+ { url = "https://files.pythonhosted.org/packages/d7/9e/d06adaa5cc65876bcd7a24f59f67e07f7e4194e6298130024ed3fb22c456/pygame-2.6.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:173badf82fa198e6888017bea40f511cb28e69ecdd5a72b214e81e4dcd66c3b1", size = 14298054, upload-time = "2024-09-29T11:39:53.891Z" },+ { url = "https://files.pythonhosted.org/packages/7a/a1/9ae2852ebd3a7cc7d9ae7ff7919ab983e4a5c1b7a14e840732f23b2b48f6/pygame-2.6.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ce8cc108b92de9b149b344ad2e25eedbe773af0dc41dfb24d1f07f679b558c60", size = 13977107, upload-time = "2024-09-29T11:39:56.831Z" },+ { url = "https://files.pythonhosted.org/packages/31/df/6788fd2e9a864d0496a77670e44a7c012184b7a5382866ab0e60c55c0f28/pygame-2.6.1-cp311-cp311-win32.whl", hash = "sha256:811e7b925146d8149d79193652cbb83e0eca0aae66476b1cb310f0f4226b8b5c", size = 10250863, upload-time = "2024-09-29T11:44:48.199Z" },+ { url = "https://files.pythonhosted.org/packages/d2/55/ca3eb851aeef4f6f2e98a360c201f0d00bd1ba2eb98e2c7850d80aabc526/pygame-2.6.1-cp311-cp311-win_amd64.whl", hash = "sha256:91476902426facd4bb0dad4dc3b2573bc82c95c71b135e0daaea072ed528d299", size = 10622016, upload-time = "2024-09-29T12:17:01.545Z" },+ { url = "https://files.pythonhosted.org/packages/92/16/2c602c332f45ff9526d61f6bd764db5096ff9035433e2172e2d2cadae8db/pygame-2.6.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:4ee7f2771f588c966fa2fa8b829be26698c9b4836f82ede5e4edc1a68594942e", size = 13118279, upload-time = "2024-09-29T14:26:30.427Z" },+ { url = "https://files.pythonhosted.org/packages/cd/53/77ccbc384b251c6e34bfd2e734c638233922449a7844e3c7a11ef91cee39/pygame-2.6.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:c8040ea2ab18c6b255af706ec01355c8a6b08dc48d77fd4ee783f8fc46a843bf", size = 12384524, upload-time = "2024-09-29T14:26:49.996Z" },+ { url = "https://files.pythonhosted.org/packages/06/be/3ed337583f010696c3b3435e89a74fb29d0c74d0931e8f33c0a4246307a9/pygame-2.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c47a6938de93fa610accd4969e638c2aebcb29b2fca518a84c3a39d91ab47116", size = 13587123, upload-time = "2024-09-29T11:10:50.072Z" },+ { url = "https://files.pythonhosted.org/packages/fd/ca/b015586a450db59313535662991b34d24c1f0c0dc149cc5f496573900f4e/pygame-2.6.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:33006f784e1c7d7e466fcb61d5489da59cc5f7eb098712f792a225df1d4e229d", size = 14275532, upload-time = "2024-09-29T11:39:59.356Z" },+ { url = "https://files.pythonhosted.org/packages/b9/f2/d31e6ad42d657af07be2ffd779190353f759a07b51232b9e1d724f2cda46/pygame-2.6.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1206125f14cae22c44565c9d333607f1d9f59487b1f1432945dfc809aeaa3e88", size = 13952653, upload-time = "2024-09-29T11:40:01.781Z" },+ { url = "https://files.pythonhosted.org/packages/f3/42/8ea2a6979e6fa971702fece1747e862e2256d4a8558fe0da6364dd946c53/pygame-2.6.1-cp312-cp312-win32.whl", hash = "sha256:84fc4054e25262140d09d39e094f6880d730199710829902f0d8ceae0213379e", size = 10252421, upload-time = "2024-09-29T11:14:26.877Z" },+ { url = "https://files.pythonhosted.org/packages/5f/90/7d766d54bb95939725e9a9361f9c06b0cfbe3fe100aa35400f0a461a278a/pygame-2.6.1-cp312-cp312-win_amd64.whl", hash = "sha256:3a9e7396be0d9633831c3f8d5d82dd63ba373ad65599628294b7a4f8a5a01a65", size = 10624591, upload-time = "2024-09-29T11:52:54.489Z" },+ { url = "https://files.pythonhosted.org/packages/e1/91/718acf3e2a9d08a6ddcc96bd02a6f63c99ee7ba14afeaff2a51c987df0b9/pygame-2.6.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ae6039f3a55d800db80e8010f387557b528d34d534435e0871326804df2a62f2", size = 13090765, upload-time = "2024-09-29T14:27:02.377Z" },+ { url = "https://files.pythonhosted.org/packages/0e/c6/9cb315de851a7682d9c7568a41ea042ee98d668cb8deadc1dafcab6116f0/pygame-2.6.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2a3a1288e2e9b1e5834e425bedd5ba01a3cd4902b5c2bff8ed4a740ccfe98171", size = 12381704, upload-time = "2024-09-29T14:27:10.228Z" },+ { url = "https://files.pythonhosted.org/packages/9f/8f/617a1196e31ae3b46be6949fbaa95b8c93ce15e0544266198c2266cc1b4d/pygame-2.6.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:27eb17e3dc9640e4b4683074f1890e2e879827447770470c2aba9f125f74510b", size = 13581091, upload-time = "2024-09-29T11:30:27.653Z" },+ { url = "https://files.pythonhosted.org/packages/3b/87/2851a564e40a2dad353f1c6e143465d445dab18a95281f9ea458b94f3608/pygame-2.6.1-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4c1623180e70a03c4a734deb9bac50fc9c82942ae84a3a220779062128e75f3b", size = 14273844, upload-time = "2024-09-29T11:40:04.138Z" },+ { url = "https://files.pythonhosted.org/packages/85/b5/aa23aa2e70bcba42c989c02e7228273c30f3b44b9b264abb93eaeff43ad7/pygame-2.6.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ef07c0103d79492c21fced9ad68c11c32efa6801ca1920ebfd0f15fb46c78b1c", size = 13951197, upload-time = "2024-09-29T11:40:06.785Z" },+ { url = "https://files.pythonhosted.org/packages/a6/06/29e939b34d3f1354738c7d201c51c250ad7abefefaf6f8332d962ff67c4b/pygame-2.6.1-cp313-cp313-win32.whl", hash = "sha256:3acd8c009317190c2bfd81db681ecef47d5eb108c2151d09596d9c7ea9df5c0e", size = 10249309, upload-time = "2024-09-29T11:10:23.329Z" },+ { url = "https://files.pythonhosted.org/packages/7e/11/17f7f319ca91824b86557e9303e3b7a71991ef17fd45286bf47d7f0a38e6/pygame-2.6.1-cp313-cp313-win_amd64.whl", hash = "sha256:813af4fba5d0b2cb8e58f5d95f7910295c34067dcc290d34f1be59c48bd1ea6a", size = 10620084, upload-time = "2024-09-29T11:48:51.587Z" },+]++[[package]]+name = "pygments"+version = "2.21.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/49/2e/ced460408999b33da6b31b0021b0f37d329e202d4169aeb164493778f25b/pygments-2.21.0.tar.gz", hash = "sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c", size = 5005329, upload-time = "2026-08-17T08:02:48.824Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/71/46/17f022dd3e953bf20a04a028a21ec746d942f8d2af30fa0f124fa0e6a684/pygments-2.21.0-py3-none-any.whl", hash = "sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9", size = 1250147, upload-time = "2026-08-17T08:02:44.912Z" },+]++[[package]]+name = "pytest"+version = "9.1.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "colorama", marker = "sys_platform == 'win32'" },+ { name = "iniconfig" },+ { name = "packaging" },+ { name = "pluggy" },+ { name = "pygments" },+]+sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },+]++[[package]]+name = "pyyaml"+version = "6.0.3"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/6d/16/a95b6757765b7b031c9374925bb718d55e0a9ba8a1b6a12d25962ea44347/pyyaml-6.0.3-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e", size = 185826, upload-time = "2025-09-25T21:31:58.655Z" },+ { url = "https://files.pythonhosted.org/packages/16/19/13de8e4377ed53079ee996e1ab0a9c33ec2faf808a4647b7b4c0d46dd239/pyyaml-6.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824", size = 175577, upload-time = "2025-09-25T21:32:00.088Z" },+ { url = "https://files.pythonhosted.org/packages/0c/62/d2eb46264d4b157dae1275b573017abec435397aa59cbcdab6fc978a8af4/pyyaml-6.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c", size = 775556, upload-time = "2025-09-25T21:32:01.31Z" },+ { url = "https://files.pythonhosted.org/packages/10/cb/16c3f2cf3266edd25aaa00d6c4350381c8b012ed6f5276675b9eba8d9ff4/pyyaml-6.0.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00", size = 882114, upload-time = "2025-09-25T21:32:03.376Z" },+ { url = "https://files.pythonhosted.org/packages/71/60/917329f640924b18ff085ab889a11c763e0b573da888e8404ff486657602/pyyaml-6.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d", size = 806638, upload-time = "2025-09-25T21:32:04.553Z" },+ { url = "https://files.pythonhosted.org/packages/dd/6f/529b0f316a9fd167281a6c3826b5583e6192dba792dd55e3203d3f8e655a/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a", size = 767463, upload-time = "2025-09-25T21:32:06.152Z" },+ { url = "https://files.pythonhosted.org/packages/f2/6a/b627b4e0c1dd03718543519ffb2f1deea4a1e6d42fbab8021936a4d22589/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4", size = 794986, upload-time = "2025-09-25T21:32:07.367Z" },+ { url = "https://files.pythonhosted.org/packages/45/91/47a6e1c42d9ee337c4839208f30d9f09caa9f720ec7582917b264defc875/pyyaml-6.0.3-cp311-cp311-win32.whl", hash = "sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b", size = 142543, upload-time = "2025-09-25T21:32:08.95Z" },+ { url = "https://files.pythonhosted.org/packages/da/e3/ea007450a105ae919a72393cb06f122f288ef60bba2dc64b26e2646fa315/pyyaml-6.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf", size = 158763, upload-time = "2025-09-25T21:32:09.96Z" },+ { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" },+ { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" },+ { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" },+ { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" },+ { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" },+ { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" },+ { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" },+ { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" },+ { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" },+ { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" },+ { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" },+ { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" },+ { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" },+ { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" },+ { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" },+ { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" },+ { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" },+ { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" },+ { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" },+ { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" },+ { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" },+ { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" },+ { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" },+ { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" },+ { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" },+ { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" },+ { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" },+ { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" },+ { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" },+ { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" },+ { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" },+ { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" },+ { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" },+ { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" },+ { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" },+ { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" },+ { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" },+ { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" },+]++[[package]]+name = "regex"+version = "2026.9.29"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/fc/f2/af1da9d3ceed77bfcdce40427d49ba0be94e4fe84245e3bfef68c10e75b6/regex-2026.9.29.tar.gz", hash = "sha256:8b5fcc4771732191b2b7d1dd68d8f0353f47f8d90b6150f6dce58bf1112442cb", size = 419199, upload-time = "2026-09-29T00:49:58.298Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/e8/6b/6dea87689c3a06a6e79d254bf824e6f3e3d724b5ba027c6112559aa6cd2c/regex-2026.9.29-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:6abb75ab16bc3281714a5b99548a2225db70dba1f995f6d7f7419b76eb5a8fbe", size = 495413, upload-time = "2026-09-29T00:46:14.51Z" },+ { url = "https://files.pythonhosted.org/packages/3a/a5/0c791a0e83ad1013d262c13247c4c77e0f4a8d05bdc167df96aba6681c0d/regex-2026.9.29-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:b7b893976e7fe42053da64f2aa27239c24252fd2ec6df471e1be197c0addc3b1", size = 295145, upload-time = "2026-09-29T00:46:16.292Z" },+ { url = "https://files.pythonhosted.org/packages/b1/07/9bf3607d8d13a12e436ab9d63f9791e10706827d535695b23964ad79fd79/regex-2026.9.29-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:066d0e3dbfdd739bce2bf8c2a41dd16f73e3d8adc2eb06dd803a36a307f56075", size = 292123, upload-time = "2026-09-29T00:46:17.646Z" },+ { url = "https://files.pythonhosted.org/packages/64/6b/32c2e6fc617e1d3f247e250fea31a9a35b1265bd32f585968aa13b9999b9/regex-2026.9.29-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7020ed44df30b3aa492c00ee3b52d0548c1f30c2c6c5bb13ae897680900d3413", size = 799537, upload-time = "2026-09-29T00:46:18.976Z" },+ { url = "https://files.pythonhosted.org/packages/bf/72/f041177f3c7a4606f7c81a95fe7eea03e2a0c4e8bff9e439a01432cbc9f2/regex-2026.9.29-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ae4613d7d9dda60fcba95f846cc6f808017f1843f392cf9daad14a6534493d71", size = 873255, upload-time = "2026-09-29T00:46:20.684Z" },+ { url = "https://files.pythonhosted.org/packages/d0/4e/a78948e11dd715e0e46716c2e0f3404b3fe6a44e2a2e9abdc7d965cab2b3/regex-2026.9.29-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:bec37990e3d6121f29ecfb594bd8f1bf009e9f7926daba2e50e3b27d3892a783", size = 914570, upload-time = "2026-09-29T00:46:22.599Z" },+ { url = "https://files.pythonhosted.org/packages/8a/70/aa08d1d2b294894b365e5f8ba5380fe3f8546acdb81f10639dfd74209c37/regex-2026.9.29-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:612b709381c0355b70d89cdb51b7f670591ed5cbbc0e3b5337488019dc667b65", size = 804978, upload-time = "2026-09-29T00:46:23.981Z" },+ { url = "https://files.pythonhosted.org/packages/21/32/1b03534c4715aca3b564416d28d518083ed4dab3bc913267600d2256140d/regex-2026.9.29-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a760da040b47767b4b873adfb7c3b691e9ba2fc60f113f9d0b88f1a62f323e85", size = 779297, upload-time = "2026-09-29T00:46:25.318Z" },+ { url = "https://files.pythonhosted.org/packages/76/a7/378f6f558d9e4444af315a307c5953565a511d1e3666f1bb7bdc82012b6b/regex-2026.9.29-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:49ee178ca31c94621294bf9b8b676a92a2e6bba8af0529591753719e57edb621", size = 786961, upload-time = "2026-09-29T00:46:26.963Z" },+ { url = "https://files.pythonhosted.org/packages/59/13/79f0b1846f5f342f92ddbd4b27b18bcb86da96d902c1a0be26520bde98d7/regex-2026.9.29-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:5eeb8edc6110d9194a4d0d54610f64c37a31c605b5dbb7e407fc6ec7fa34a4a1", size = 863234, upload-time = "2026-09-29T00:46:28.58Z" },+ { url = "https://files.pythonhosted.org/packages/97/19/05af70dec9f2eed6ba34e08d2dcc6a48e7ae5e307659d5fe4201a5d7bbee/regex-2026.9.29-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:ccb64d887a9db1cd76dbc0f92051a1a478a2a67e7f56c62d915cb881d7734704", size = 766487, upload-time = "2026-09-29T00:46:29.941Z" },+ { url = "https://files.pythonhosted.org/packages/01/e1/9c7486d4afe8fdd1fe0ad60139f8aa91427381f409af6a29b609d8fdcb3a/regex-2026.9.29-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:9e4482589065c8ecd761cff522dcd85f2d39e62f551e37e025d1c7d54772def3", size = 854792, upload-time = "2026-09-29T00:46:31.358Z" },+ { url = "https://files.pythonhosted.org/packages/26/c7/49d008ff5f741d9a9799d7315556f3a12b983ff0fcd2cdfb62904bedafbf/regex-2026.9.29-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:d60030baaa7bfbb02d650c126cdcddcb6e33dbff14d819434c8fa2fdcaeeeba5", size = 793050, upload-time = "2026-09-29T00:46:32.775Z" },+ { url = "https://files.pythonhosted.org/packages/cb/a1/46ba549e65562ca04608b24179b8a7bb6f146ae0e7c6d7f5e70f3339c8ba/regex-2026.9.29-cp311-cp311-win32.whl", hash = "sha256:18ae8eed4526e35bdb754d61562b90bf5c00a67fdcf3cc1380dd59597486631b", size = 268940, upload-time = "2026-09-29T00:46:34.179Z" },+ { url = "https://files.pythonhosted.org/packages/4d/4a/aab232183c70fdcf77bcf0c51819da02ec522e393e6a0bf00bcf2142e21f/regex-2026.9.29-cp311-cp311-win_amd64.whl", hash = "sha256:1043aedf5917caa861bcb25a9c11460049656bdf0017a90a309fa8f255467725", size = 280641, upload-time = "2026-09-29T00:46:35.484Z" },+ { url = "https://files.pythonhosted.org/packages/33/b1/7c05954af0f51de376df2ba97f7f78a8b79334c7e5b3d2d9f2aead1f4d3d/regex-2026.9.29-cp311-cp311-win_arm64.whl", hash = "sha256:352cf115a810b357caa35193ab656ecf5ef41056855e82f292c99e8514f8d954", size = 279411, upload-time = "2026-09-29T00:46:37.193Z" },+ { url = "https://files.pythonhosted.org/packages/84/48/3fdcde9a0baa84d7d25571223265d6e434e114763b438601d54a8028bf3e/regex-2026.9.29-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:dc79d36d0618752265f0d575915bdc5c5130ecb9c9f6b3bcefeae32e4bdfafcf", size = 497903, upload-time = "2026-09-29T00:46:38.938Z" },+ { url = "https://files.pythonhosted.org/packages/2e/1c/4ee3e97c76f53940488dfe7a7e18705e78daac8cd7fb161d246b9e328449/regex-2026.9.29-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3a21a9509d0ee88e7a70e1ad228cd2f0e0fd1e187458db132e8a8d18c97daf9d", size = 296416, upload-time = "2026-09-29T00:46:40.406Z" },+ { url = "https://files.pythonhosted.org/packages/37/14/f3f0ba083d2094392d5eabf56db5ea6ba469fd6e927afd187042054ea68a/regex-2026.9.29-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f57dc6b8fef170f105d2cf5cdce254f47b137d7755086cf7050f47e16582abba", size = 293633, upload-time = "2026-09-29T00:46:41.959Z" },+ { url = "https://files.pythonhosted.org/packages/c9/72/67e7a8ce17f1aea49df215564048efb49cc8c2b31a0e0fc30f36838f8516/regex-2026.9.29-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f93bc1c3486ef3747e07c9d7c1d0a147b8fbaab975f80e348aed6f71309dfaca", size = 805885, upload-time = "2026-09-29T00:46:43.373Z" },+ { url = "https://files.pythonhosted.org/packages/f6/78/25436bcfd4d2260b4b4090094d55d7ab53ec8a1ab4865a0b8bcb33c7d5c0/regex-2026.9.29-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9e1d3a4cb7993b708f0ada8d0c84590efd853f169e7147d2202c9da503180242", size = 878344, upload-time = "2026-09-29T00:46:45.328Z" },+ { url = "https://files.pythonhosted.org/packages/97/e6/a09ec3a23ae41d6179880e67f0aace9284b2d95f2d7b326eff203f8eec5e/regex-2026.9.29-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:dabee8f4935e731fb46b2a3091bdda0d3d94b3bbfb907d2b4f12eefce4009619", size = 919181, upload-time = "2026-09-29T00:46:47.041Z" },+ { url = "https://files.pythonhosted.org/packages/26/83/d2fbd2e4e3afb1167daa825187d196f313cbaa1a4768f311fb041bb0e3d2/regex-2026.9.29-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:39ab5894d971f9ac68baa6eca5c50387db579cfcacf36ae8df3feceb1815e6d0", size = 807783, upload-time = "2026-09-29T00:46:48.894Z" },+ { url = "https://files.pythonhosted.org/packages/46/0b/eb429a7016610d44fc89a597163f8c9127505f0d7dc724dc9effbb6a3ac0/regex-2026.9.29-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c1a9a6651197fbed6f0212591418b9def774fc3f8324f78d1bf0e6a63e5f8aa1", size = 783465, upload-time = "2026-09-29T00:46:50.64Z" },+ { url = "https://files.pythonhosted.org/packages/1b/07/58a3c0153c7476898430f6a7cf3d9062a1d17fbea4f43399ecaf411c7b4c/regex-2026.9.29-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:87fb80cbe3557e27e7b28b995c2b2eedf689b8886f941ab93e0e288f0976518a", size = 793519, upload-time = "2026-09-29T00:46:52.396Z" },+ { url = "https://files.pythonhosted.org/packages/2a/e8/161b94d39164520e21a7befe0245569bf7fda4c7cf1fc4e2df2b5def49da/regex-2026.9.29-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:3c5c2ef13797466aa64170cbb66ad98a32351dd4127694cea7199f80f213750d", size = 869293, upload-time = "2026-09-29T00:46:54.128Z" },+ { url = "https://files.pythonhosted.org/packages/8f/07/3b02ed829aa2decdc1955d222bd1e2f99d1c8bb4873bbb9a66b2f0a36bff/regex-2026.9.29-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:59b49507f47479e299a9e1bc41b5cb83a7afda0540625f1dbae886615978acbf", size = 770239, upload-time = "2026-09-29T00:46:56.106Z" },+ { url = "https://files.pythonhosted.org/packages/42/5b/ba61f6fe062eb8562e742367d177bb75370434138ef6c9d2a27114f8d613/regex-2026.9.29-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:0dd8af32e9f7b56b7f95cc1fd79b23054c3bdc172392ae560acc24d57b7ffe71", size = 861973, upload-time = "2026-09-29T00:46:57.665Z" },+ { url = "https://files.pythonhosted.org/packages/cc/27/767259b20e8a842948990f5e99138d6c077248fd42f8b5468b1d9ca4b814/regex-2026.9.29-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:db5e82ba15c142425b8406690032df89e39cca4a2e8afbbb9a3d84edc2373ac3", size = 796470, upload-time = "2026-09-29T00:46:59.236Z" },+ { url = "https://files.pythonhosted.org/packages/a0/05/2566c4ba849b68a8ab81a6bf428fa79d20aae7ddee83979103c0381df254/regex-2026.9.29-cp312-cp312-win32.whl", hash = "sha256:d0c3082bf79bcd6a614d55916590ad4b8f93200e10b97f463ea5d9d07c9b5f23", size = 269327, upload-time = "2026-09-29T00:47:01.135Z" },+ { url = "https://files.pythonhosted.org/packages/93/19/489bc8db91196381c935752df01ba3f607140daece33b78d88573f028e64/regex-2026.9.29-cp312-cp312-win_amd64.whl", hash = "sha256:fdd88ed5e20b1bcdd234421e454962c971aa44b653bdb7f1ea9ef683e90fb649", size = 280334, upload-time = "2026-09-29T00:47:04.436Z" },+ { url = "https://files.pythonhosted.org/packages/0b/47/fb88ba779d0e5e7d4b0ec1aceeb13845948a2cb876bd572a2d1dfdba090b/regex-2026.9.29-cp312-cp312-win_arm64.whl", hash = "sha256:4fe97894d1b306c919b4e50def1e6f6c522f4d03a7283811f4d108f1ce5d3ac2", size = 279614, upload-time = "2026-09-29T00:47:06.541Z" },+ { url = "https://files.pythonhosted.org/packages/79/d5/6080f7d1a6e7e36aa720f806ac93c035ba39c209ae6cc510e8ef4c0279c6/regex-2026.9.29-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:f1a0d5117230dd46b399a30a38afa44f79c99f3168988fdc4f425c3f928b39df", size = 497622, upload-time = "2026-09-29T00:47:08.251Z" },+ { url = "https://files.pythonhosted.org/packages/00/71/c87fc7a2e21a42f9d57489db32951c37eef56d153840459a80d464f0321d/regex-2026.9.29-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:f0fe9834e5aeccaf19a0d8feb296d66a24be1a7c9922002f842a682cd5abb787", size = 296281, upload-time = "2026-09-29T00:47:09.764Z" },+ { url = "https://files.pythonhosted.org/packages/11/9e/aa0f4cde3bc4688c1d58b0cd8415edd708339bc0bc401a195b0b1e8c8f0c/regex-2026.9.29-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c90fcf7804ea0a54b896ce0f2b9565350220b8d4890fd0db461a476a4c687963", size = 293472, upload-time = "2026-09-29T00:47:11.723Z" },+ { url = "https://files.pythonhosted.org/packages/90/d4/e835c487850ed922a8d6074f953b888c8ea99775c76b9ed5f8a4d72eab92/regex-2026.9.29-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e11edba5bc344a32b029a7af9d4b3173982dd79eeafa0b9dbd787364414b0509", size = 805966, upload-time = "2026-09-29T00:47:13.235Z" },+ { url = "https://files.pythonhosted.org/packages/2c/57/ba8809847fbae8d2cbc71367c6ded510a7ec88bf52493c65efc1acf4effb/regex-2026.9.29-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bb90e7177944b6684738c1fc36aabd2dd00d1de3be7dbe09f91e196f1bc0dc81", size = 878346, upload-time = "2026-09-29T00:47:14.877Z" },+ { url = "https://files.pythonhosted.org/packages/1a/52/e3da19fc3cc15ef67ab67e121e87887c3bccfdb683a7a9ec557c460ca5b7/regex-2026.9.29-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d06fcdecc10fc7954d7c8f27a03c96055fe525274dc84a7b0dbdc3d6b9e03dab", size = 919250, upload-time = "2026-09-29T00:47:16.622Z" },+ { url = "https://files.pythonhosted.org/packages/9a/8e/c1ed81f55f992f6aa0b699a592a50c1ce9e6d44ff1aee2c14c0537dcef9c/regex-2026.9.29-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d49c18f1ea294cf4adde2e5ac256e98c82ea9d708462ce4bf799dffa7cfe8a2c", size = 807902, upload-time = "2026-09-29T00:47:18.268Z" },+ { url = "https://files.pythonhosted.org/packages/ad/bc/5a6886eb470e41040e21e05b75024a18b6ebfe7ea400b72094a60f949101/regex-2026.9.29-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3e778bfccd63075167709136afbc251c1f683758d5bf49c803c60ac3f894ce6b", size = 783514, upload-time = "2026-09-29T00:47:19.921Z" },+ { url = "https://files.pythonhosted.org/packages/cb/52/6d951d453b023c6edb880f1ba474291b53b8ce1cc438b96a9db6d791d991/regex-2026.9.29-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:686ac5350fceae63830bb98805fcb8039325bf4c06d9f6f048ff65229d5bffa5", size = 793466, upload-time = "2026-09-29T00:47:21.552Z" },+ { url = "https://files.pythonhosted.org/packages/99/b9/d5a41adc08360f5eee0dc4846c578f002366947211fc8af5a69a64ee7b9f/regex-2026.9.29-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:26ec4ccce55aa533fbd603d08911b01101a8fcfec987845ac3ae2c7087b2bde3", size = 869464, upload-time = "2026-09-29T00:47:23.276Z" },+ { url = "https://files.pythonhosted.org/packages/4b/32/d76c9d91f5d798e2e9e67f6f85ec4ae35445ac425f7454797311cecb80ca/regex-2026.9.29-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:a655d34b2a6943af32401f3d94f72e9d731f6ad16285815550bf2b4ee69d420a", size = 770278, upload-time = "2026-09-29T00:47:25.193Z" },+ { url = "https://files.pythonhosted.org/packages/24/00/aeebdb540c620a0f7317f6d6fad80a47729ecf0599a24b5c34ec155351f5/regex-2026.9.29-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:0c992c19cd45058a4b92f68f139c93db168b48fb1f322c9a7cd620806afb6b51", size = 861949, upload-time = "2026-09-29T00:47:27.005Z" },+ { url = "https://files.pythonhosted.org/packages/12/62/d0314bcedfd3586197e4596931fa220260eb2385bf53184e5b9ae67db24b/regex-2026.9.29-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ebb8912f565b8cdbbf27debfe00df04202c20e2f651b9e32767930c5eace3621", size = 796602, upload-time = "2026-09-29T00:47:29.233Z" },+ { url = "https://files.pythonhosted.org/packages/ae/c7/d5a8c13a613facb03e0fb55c1ebaaf7bb35d8e2c1abe8bef8dca809fc1d9/regex-2026.9.29-cp313-cp313-win32.whl", hash = "sha256:4d7d93613b01b0199961330e49cfc52d479b3d5776c56c691db31130c0a07d91", size = 269306, upload-time = "2026-09-29T00:47:31.14Z" },+ { url = "https://files.pythonhosted.org/packages/80/a7/bf93a3a6afa5f7bc16b7afb94ae581b01cae620b8ad56bd8f9572a985959/regex-2026.9.29-cp313-cp313-win_amd64.whl", hash = "sha256:61956f074ecd123f55adca68ee3eab46e6a07ad3f8e64e6db95dfacb444f55c4", size = 280307, upload-time = "2026-09-29T00:47:32.709Z" },+ { url = "https://files.pythonhosted.org/packages/b2/7d/388274e53605a86297f433a08102a7bbdcf9379d47683d307ccaefd88e2c/regex-2026.9.29-cp313-cp313-win_arm64.whl", hash = "sha256:bfc71e6d970419c1309b3640305298643e2a734cad3f7cfb6d2ddee4175ab53d", size = 279599, upload-time = "2026-09-29T00:47:34.674Z" },+ { url = "https://files.pythonhosted.org/packages/93/1f/d9dc6f02f569625faf67a4daec926cd5023472dcd69bb44286dccd5a5ab3/regex-2026.9.29-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:957bb708e8057ab1649ba566456429d691ec9b90d1c9ad1af1ba7ffbbeaf05f2", size = 497598, upload-time = "2026-09-29T00:47:36.541Z" },+ { url = "https://files.pythonhosted.org/packages/9c/83/9b693a3fd1451381e812031a8961ec5b3b8f0c8cc6871f14c5223642804d/regex-2026.9.29-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:c9b602fae1e00b7c035d661ce85575365719192a7b46784bd71cf64c68053aa0", size = 296250, upload-time = "2026-09-29T00:47:38.233Z" },+ { url = "https://files.pythonhosted.org/packages/dd/5f/52bc2abc3fef040cd9de76ab29c918d6a717a454ae2b9dd7938b0c95656d/regex-2026.9.29-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0166844493626c5015c6088ee15c9ca2fd060ca15b7641d1657da6a58432ae33", size = 293518, upload-time = "2026-09-29T00:47:39.957Z" },+ { url = "https://files.pythonhosted.org/packages/dc/fc/cf50671215ee0057046980b4571ef8646a005819bb67f0957e779ed107a5/regex-2026.9.29-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b97a38fb4c732b6832db6bf108963adbcd82ef1268ba2025dce390f45af75efa", size = 806037, upload-time = "2026-09-29T00:47:41.676Z" },+ { url = "https://files.pythonhosted.org/packages/14/4b/dddef8fc15c63e4347cc9efb138d0cd306f30e6c98acbcc81a8f780083b9/regex-2026.9.29-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a540abfab208e1b7ef2df231c40ef3b6cbb30a0aad6204e9b6a81c10a6794628", size = 878881, upload-time = "2026-09-29T00:47:43.755Z" },+ { url = "https://files.pythonhosted.org/packages/9f/cb/38daabed32d28f7e58a06e9344ce00dc67952e9996bc578ed6a29fe1240e/regex-2026.9.29-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ddfa987262763c3c22a8367d2a49c244b018a74c3a8e3ab1a864119ad45c5633", size = 918684, upload-time = "2026-09-29T00:47:45.594Z" },+ { url = "https://files.pythonhosted.org/packages/a9/4d/041d9458a645fee4fce4d642a89d27271a3cfcd91095104f6dde44da70bf/regex-2026.9.29-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2f7f7aa47b229f2b39a2ae2596d2ad5625d77b5eb9856fac2dab3eb506cdd0a0", size = 807176, upload-time = "2026-09-29T00:47:47.372Z" },+ { url = "https://files.pythonhosted.org/packages/bf/c4/4383eed7aa5aef67616cb1b3f3ad06b7c624c4e6cced48630cd5ce133d85/regex-2026.9.29-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d9b77b25b4f395f92de6099ab08e8ae2bc7e51dfe157f22900902243a5cc90c7", size = 784315, upload-time = "2026-09-29T00:47:49.518Z" },+ { url = "https://files.pythonhosted.org/packages/5c/a6/0086ad31cebb183c637d3198547075aa493afde308e1ff61fccccb29ba6e/regex-2026.9.29-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:34b6925af9853bf461950e6508910f179fd6e9b1a7ec8548e069606b7e51a26b", size = 793748, upload-time = "2026-09-29T00:47:51.279Z" },+ { url = "https://files.pythonhosted.org/packages/d5/a0/f9005cba3f629a859573fc5d1224ea4e1f97919ec8581d018e03a351a604/regex-2026.9.29-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:addd736a0547d553283adaf4e05d7104e7f2c7b0b092e9b4d28756825f14531f", size = 870302, upload-time = "2026-09-29T00:47:53.368Z" },+ { url = "https://files.pythonhosted.org/packages/01/4f/e1a3e46bb5315a4e18b01a990e7a28e2a16595609d50c442baf2815a3c65/regex-2026.9.29-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:fe3fa1dd453ed5c7f5ea23a26218329790ed7197a99b90e94330e313959a7f52", size = 770299, upload-time = "2026-09-29T00:47:55.606Z" },+ { url = "https://files.pythonhosted.org/packages/2c/fe/f303b4acfda44e1ff1379368748c1ef2dad04a6a8e9c0ecbc970b19d97ca/regex-2026.9.29-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:0cc63b5e47c12a48d90c7e9d7de6a035dd14f62868aaedbb4e0ff8ba2b8bfe7b", size = 861570, upload-time = "2026-09-29T00:47:57.617Z" },+ { url = "https://files.pythonhosted.org/packages/60/b6/b4f7e99249f596017c60ccad5faf9310fc8e3e59bb2244940a90a1b0bdff/regex-2026.9.29-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:724184b4aafed865e4f13ca313fdcb43024300c028ec67319cfa16847d84685e", size = 795967, upload-time = "2026-09-29T00:47:59.922Z" },+ { url = "https://files.pythonhosted.org/packages/fb/d3/fc865a4638d9f6762192b6bab5b7aa1f33a90e9e99578c2e111e2a63c8c3/regex-2026.9.29-cp314-cp314-win32.whl", hash = "sha256:c6c8fabf1dafc1f1ddcbb67896d3f93efb092e8c4b6322d7389b944e76a484e5", size = 274758, upload-time = "2026-09-29T00:48:01.8Z" },+ { url = "https://files.pythonhosted.org/packages/31/e2/c2b466924ccbeb874862968ca638051b15a8fd29d994a0e99004a5cbf78e/regex-2026.9.29-cp314-cp314-win_amd64.whl", hash = "sha256:1c2a0026062abcc321a53db4a185ceba0b59a66b5d37b0808917a88b55a5257f", size = 283817, upload-time = "2026-09-29T00:48:03.614Z" },+ { url = "https://files.pythonhosted.org/packages/c6/42/ea0f8dbaa924fa75c6338935eaee2f44dab369b27f02db1e03d74344b049/regex-2026.9.29-cp314-cp314-win_arm64.whl", hash = "sha256:121a76a0985db80ceae9e171c337f8c927868e37d01b54e3ce87bc87f9c6a208", size = 283663, upload-time = "2026-09-29T00:48:05.624Z" },+ { url = "https://files.pythonhosted.org/packages/44/48/d58e5081119f5c223bbb37d2340acde3d069e1df8e8cd166c37502eee4da/regex-2026.9.29-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:e31f72490b7c12f7790e1e25c3afffd20503ee1bfb43461d7838b871ff244b19", size = 501393, upload-time = "2026-09-29T00:48:07.833Z" },+ { url = "https://files.pythonhosted.org/packages/72/3c/c49945287d4f9efee7d41f98072f8ad880efb8f430595a612fbdea996a4e/regex-2026.9.29-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:80ea96f5c1a30bf09007d48466521d9c294bebe197c708c3359096e3e3691632", size = 298237, upload-time = "2026-09-29T00:48:09.684Z" },+ { url = "https://files.pythonhosted.org/packages/f9/1f/688cb61c3d4cf7bcc1ed444b5cc49399eba3e51c469ae285cf87fea3022e/regex-2026.9.29-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:554bffadcbcb6d5f4e5fb10a61cc52084b9a63d1dab5f10bcd2c4343972e8e2c", size = 295936, upload-time = "2026-09-29T00:48:11.454Z" },+ { url = "https://files.pythonhosted.org/packages/26/a3/de43ac6b877b7d09c19a3a426b1bd5acdd209eaaf68f406466f80439ccf6/regex-2026.9.29-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:864e9b87ac33c3fb9fb4ad48166d4fdb579c351d5c77deb0d34bccb36a775cd9", size = 816905, upload-time = "2026-09-29T00:48:13.321Z" },+ { url = "https://files.pythonhosted.org/packages/62/14/9940763201c51d537786304984c67d0fc3d2ed18837ffb6f09a869f6b6c9/regex-2026.9.29-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:044265d77d94f5e3cb2fd72c76723807c429cb8c533e9d4672d0334a6f14f588", size = 881527, upload-time = "2026-09-29T00:48:15.313Z" },+ { url = "https://files.pythonhosted.org/packages/d3/e1/c842d8df0b23245ebf202f8ab9c39fd48e2db39959454ec39a41c8c72082/regex-2026.9.29-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:2089fe39c406784d90101c726755ffa1497bb74638fd434300d2b88006186de8", size = 923115, upload-time = "2026-09-29T00:48:17.328Z" },+ { url = "https://files.pythonhosted.org/packages/d8/c1/98622479e3c354a446a75232e522d747d2b3df23092dcd8a5309380a2020/regex-2026.9.29-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0def9fb6abac55492d6d51cddb7225d07d6f279e774e0adc08569a54a5fc8d46", size = 820674, upload-time = "2026-09-29T00:48:19.32Z" },+ { url = "https://files.pythonhosted.org/packages/6c/d0/5808c95f9c79ed27b5eedaafc3df6239ec56a49f2e23ea8f831b18427c82/regex-2026.9.29-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:888d60953908dcf761aa320c3e390ab8556efbdb551ace63921de90f6ae0848d", size = 793306, upload-time = "2026-09-29T00:48:21.615Z" },+ { url = "https://files.pythonhosted.org/packages/bf/d3/021ca2638671ad20603bcd9b4d5bfa35d2610cd216a043ea7f0b44ea39f6/regex-2026.9.29-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:ed511a0708e2297e1d6431e7fb217e3402791e491e02da800658ace4973df1bb", size = 803103, upload-time = "2026-09-29T00:48:23.871Z" },+ { url = "https://files.pythonhosted.org/packages/6b/2d/755c6d13ef9c657378013676c391c7a402166b3f419a464a3e058dcbe533/regex-2026.9.29-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:e1172147d28d8fbcf8cb8d26c41506169f5ad8fe9ec969cb116835a19d4d8eca", size = 872175, upload-time = "2026-09-29T00:48:26.255Z" },+ { url = "https://files.pythonhosted.org/packages/6c/fc/e1cab183b9dafe8597f58c1c766da9bf96204d3b2f232bcf3eeb75ff7b6c/regex-2026.9.29-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:92f05c9c42bde5785dc48770bc2194d9f7442544156f951e19cd31b096cec562", size = 777362, upload-time = "2026-09-29T00:48:28.389Z" },+ { url = "https://files.pythonhosted.org/packages/06/7c/e10ea17fba31fb4a1f9d13ed53a2d2a9066a2aea58d7557e263f6d99e7b0/regex-2026.9.29-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:f37964e4a5e993d2fd45147741e9dff7f34a2d8c00ab94c4ea0514a4677f959e", size = 865606, upload-time = "2026-09-29T00:48:30.4Z" },+ { url = "https://files.pythonhosted.org/packages/8e/6e/69824d9aee1fd41c54ea7264654a47c8d9d84d8a228e11c2bcf4c201ed81/regex-2026.9.29-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:951733b1bbdb71e377cec567b409f1a7881b47cfcad84121aa74cb575fa425ea", size = 807945, upload-time = "2026-09-29T00:48:32.375Z" },+ { url = "https://files.pythonhosted.org/packages/89/22/857050a86e21ce60193e02a8ef662521f2e263a645c8b1b905fc136b61a7/regex-2026.9.29-cp314-cp314t-win32.whl", hash = "sha256:65b408d8fcb273e3499e7ef2ce796810da1becd208c7fb4373692a242d79d461", size = 276758, upload-time = "2026-09-29T00:48:34.72Z" },+ { url = "https://files.pythonhosted.org/packages/4d/96/56808fe029553d7d4c703414f2a527faad2ea2bfa9ca094a2e7f8762b530/regex-2026.9.29-cp314-cp314t-win_amd64.whl", hash = "sha256:bf48516e35cf848390ea68850aba53e7c333720d2945b4d2c25b69fc5171723f", size = 286527, upload-time = "2026-09-29T00:48:36.864Z" },+ { url = "https://files.pythonhosted.org/packages/01/aa/074e2cfb3d8101a6a764aba5f7c5d1e21de087483e35bdc0c4ce2eb60364/regex-2026.9.29-cp314-cp314t-win_arm64.whl", hash = "sha256:9173db3be74a35cb6731701094b98120f7ee4876a287882a59cdea1fa7da342f", size = 285954, upload-time = "2026-09-29T00:48:38.901Z" },+ { url = "https://files.pythonhosted.org/packages/a7/dc/d84990386c9dfdf8c377f00f371b241fdc9a2c8aea0e3d66941b2e51be0b/regex-2026.9.29-cp315-cp315-macosx_10_15_universal2.whl", hash = "sha256:c3589f40749acce747510bf5d589d54e376cb0930ea58b35effac97e5312b0c1", size = 497836, upload-time = "2026-09-29T00:48:40.858Z" },+ { url = "https://files.pythonhosted.org/packages/c2/ab/a569ebde875fa12ff8c6c9a30e07503620f195e4be4d54c3d3ee8eecc283/regex-2026.9.29-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:32ab11df9677ca80bcbb5fe4eb1da9109a5019239a054836efc6fa1c64e683cf", size = 296244, upload-time = "2026-09-29T00:48:42.952Z" },+ { url = "https://files.pythonhosted.org/packages/f3/3e/7d548e82a108e7c8b2d5246650e397a2f8db599f9b2e975466939c5b4e70/regex-2026.9.29-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:7c03031610e3e6ed1768a2b7a8fc84637c1257b50c5eacaf094c6e17a84fc563", size = 293748, upload-time = "2026-09-29T00:48:44.985Z" },+ { url = "https://files.pythonhosted.org/packages/40/34/a8e19a52f452bbb07b32a2bef70dcdf90c2737049749f74cc12d7486fb4f/regex-2026.9.29-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:42e82e578c904445d4c8a35b8f28052cf567593215fa5db06266fbc6f77aaa2e", size = 807840, upload-time = "2026-09-29T00:48:46.948Z" },+ { url = "https://files.pythonhosted.org/packages/88/7b/11fbd4640b3bb82b72822a63c20ade4013d562d291703a9debeedc24e682/regex-2026.9.29-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:0b65c72739f981377c9c22e0c5c3cd7f42da7bd8a3c9209330fac772c7d893ed", size = 879330, upload-time = "2026-09-29T00:48:49.168Z" },+ { url = "https://files.pythonhosted.org/packages/f3/55/de58c74f1f4e31586d83eb39c56872d686c4e0d0966d151884c833b94ced/regex-2026.9.29-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:4408b2b27a95ca8cc48b7411945753773353b5c93b307754781086c99d3a576f", size = 919251, upload-time = "2026-09-29T00:48:51.322Z" },+ { url = "https://files.pythonhosted.org/packages/81/42/a8c480f6dd5ac59fa28ddae79afd9d7ac7e596fdb61813adc65bb6e674b8/regex-2026.9.29-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a714befaacbd10092ffe4cea0d3c5f008fb9efe9bc322c715bcdfdee414b9a3d", size = 808808, upload-time = "2026-09-29T00:48:53.529Z" },+ { url = "https://files.pythonhosted.org/packages/68/60/0bc0d1ec8b37ad64be6fa30e035251f11de9667a0fac9e82ee74517d81be/regex-2026.9.29-cp315-cp315-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:33026515aebc0e70d1c89978e53e8d695d35d9e472f8d5b34465ba3c74028650", size = 789907, upload-time = "2026-09-29T00:48:56.036Z" },+ { url = "https://files.pythonhosted.org/packages/da/84/116a3ef19b3acfe81077f0bf2cbc7714a5e94bc8935b7243ab61cb0f1c3c/regex-2026.9.29-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:31b003f9a070335e2a8233ee9b14a3ca8e6d792012ae011f741bf0aaf11744c5", size = 795770, upload-time = "2026-09-29T00:48:58.284Z" },+ { url = "https://files.pythonhosted.org/packages/96/ba/e38c3f203e7e7e18c957d48e6cb6dbf96c11e95a44efa4a480522afc5d6d/regex-2026.9.29-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:c03c6eb6ece86dfdcbb34799efaa339b093132e1aceed491ba5e08fe06cdf699", size = 870671, upload-time = "2026-09-29T00:49:00.506Z" },+ { url = "https://files.pythonhosted.org/packages/2f/0f/9ee0b0cb76c55f63684bd7fff554978e8773b4fc86e2bcb2d50772dc1086/regex-2026.9.29-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:a5300757f8a68f5b6cc33f57338d72a0e3589c5cc9ad5f8504ea06f028be582a", size = 778631, upload-time = "2026-09-29T00:49:02.984Z" },+ { url = "https://files.pythonhosted.org/packages/b6/19/e6e3eeb226af5872c4958002f6edef4e4f40ea4cc5f5665023f2019eb045/regex-2026.9.29-cp315-cp315-musllinux_1_2_s390x.whl", hash = "sha256:80c7cadd3fd2bfde5df8aa0787e315812cad0c313a753095d02f4c2b6c01677b", size = 862187, upload-time = "2026-09-29T00:49:05.264Z" },+ { url = "https://files.pythonhosted.org/packages/5b/62/823c102e106bb2711d6b7dfe5981552fe4467b2969c46a20c5c383cf498c/regex-2026.9.29-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:3f1e6cb402a89457582cd696f982559217d13484a193202c394015297968c86d", size = 798423, upload-time = "2026-09-29T00:49:07.644Z" },+ { url = "https://files.pythonhosted.org/packages/37/e0/e927776258fa70b2f6feffc3be584ffc85ba4c1e20a320f0aee9a632fc7d/regex-2026.9.29-cp315-cp315-win32.whl", hash = "sha256:a64b85a4760337cfefdb27d42da6ed8b58e8cde3f2d57b6ef43e76ef6ea9ef47", size = 274757, upload-time = "2026-09-29T00:49:10.513Z" },+ { url = "https://files.pythonhosted.org/packages/77/04/358de85d1860238e1b4fa98fc2c80c990124a25d2e14739e28cc02c25562/regex-2026.9.29-cp315-cp315-win_amd64.whl", hash = "sha256:b3e445b66c80b4eb4234e855ce94d9adc183eedbd632816228d89930b91b2c5b", size = 283824, upload-time = "2026-09-29T00:49:12.849Z" },+ { url = "https://files.pythonhosted.org/packages/92/d3/d5c5b264784a5ab2b0f8cf620c1eeb4dbf3440d306761905e7d99345bef5/regex-2026.9.29-cp315-cp315-win_arm64.whl", hash = "sha256:8f39588af4731c8923c26810eb3b33f76f17633985e40f59c3cd45a33805a895", size = 283664, upload-time = "2026-09-29T00:49:15.331Z" },+ { url = "https://files.pythonhosted.org/packages/02/dc/f63ec2c201445ce1150fe780f5c56f16a10124d9a9da3a93161dbb0d8892/regex-2026.9.29-cp315-cp315t-macosx_10_15_universal2.whl", hash = "sha256:fb99cc9d45f48895d9d67f6a0b8a57f08d39c174d9f25ad97a313e0470267b1c", size = 501549, upload-time = "2026-09-29T00:49:17.705Z" },+ { url = "https://files.pythonhosted.org/packages/9a/5c/d2a698dc6bfc11fbce03f1cb0249c13284e93b79ed11f893edf6fac431c9/regex-2026.9.29-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:720537c7ea6f80dc61913184edb0ce2497a306b39ef19f28505b322553d52bdb", size = 298187, upload-time = "2026-09-29T00:49:20.171Z" },+ { url = "https://files.pythonhosted.org/packages/85/b7/88dcdb38cd3935d4ee9e9ce9b8e56cb3b3518d1f020acfa7dd62ad289bf8/regex-2026.9.29-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:0fd2c901cc307a745ad4bc87f20060d7a0825a3371d1e93488af22e7a387f78f", size = 296157, upload-time = "2026-09-29T00:49:22.342Z" },+ { url = "https://files.pythonhosted.org/packages/d3/8e/ba6c01dde33a69fc294b38b43f6677baaa5735a6248f39708031a738158a/regex-2026.9.29-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b11b589e00095ec69cf79841a76360f9b079e95b0368a25b5ebb951ab0c157ff", size = 818468, upload-time = "2026-09-29T00:49:24.612Z" },+ { url = "https://files.pythonhosted.org/packages/2a/f1/2586693e3a2d6b1247852593d37a6c17b42a92ee44f7cdcb9a0c1494e64a/regex-2026.9.29-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d7cab119d0df0b9413f106b4d7fc34f2872d3574ed3806fb48959c830b1537da", size = 882825, upload-time = "2026-09-29T00:49:26.996Z" },+ { url = "https://files.pythonhosted.org/packages/30/51/084f3e7bdcd0e9c33665c938cf5d134dc3548cbb4a75f0197ec7bfd754b1/regex-2026.9.29-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b89efc38431793d28b7cd91227e2f952ad7c48df19132b17f43a5fec3c14143b", size = 923314, upload-time = "2026-09-29T00:49:29.822Z" },+ { url = "https://files.pythonhosted.org/packages/5a/f1/066c6fc23b7dc229789c21c880b5ba5ad689fb95fed12e078266f55a1f9b/regex-2026.9.29-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:80a5ea3b4fd9d6a5b9a44f7976a9acaaab35aa3c1f6b29e5bd857dfabaded223", size = 822222, upload-time = "2026-09-29T00:49:32.404Z" },+ { url = "https://files.pythonhosted.org/packages/0a/56/592cd46fdb8f2f8682a1d7fd1310e4d0bcb93fbd0e6bbe4141ac28240227/regex-2026.9.29-cp315-cp315t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:19959129885356df0e97556856f77eb2888380dac18bed075a7c05c5128c618d", size = 794882, upload-time = "2026-09-29T00:49:35.076Z" },+ { url = "https://files.pythonhosted.org/packages/ee/4d/d65384bb071c864b01aa8314e3a6a687845ebd57588390976edc960c218b/regex-2026.9.29-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:6a1a824fbed817e0a891103886b68f063b1e83cc51bc97192a90a60195a9291f", size = 805887, upload-time = "2026-09-29T00:49:37.395Z" },+ { url = "https://files.pythonhosted.org/packages/65/b6/358de0d8f40d5178e4f7e7e121cfd5b961c812b77a055d11f5079e3f8fd7/regex-2026.9.29-cp315-cp315t-musllinux_1_2_ppc64le.whl", hash = "sha256:1ba8c6a416569ce0d37e83e28a254a61dc99a419084dfb6476cea02d997f74fa", size = 872901, upload-time = "2026-09-29T00:49:39.927Z" },+ { url = "https://files.pythonhosted.org/packages/00/06/6bfded72d043240c6b52bbb5e16f639d81affbf7484b4fe2ec45f3d4afc9/regex-2026.9.29-cp315-cp315t-musllinux_1_2_riscv64.whl", hash = "sha256:446654b29bfaa30500d80947eda42cef1449dc8a87f4e3cf061cc8485d3a1f0b", size = 782970, upload-time = "2026-09-29T00:49:42.581Z" },+ { url = "https://files.pythonhosted.org/packages/5a/20/9f418a50baa78b3ed8308fcb0cc49e472dd000b7ef935a7295af202ea744/regex-2026.9.29-cp315-cp315t-musllinux_1_2_s390x.whl", hash = "sha256:bf3c49863c23a1ad6da9c30351aed6cff8d5ddbeb63c5c8420ae54e98c7d0138", size = 865441, upload-time = "2026-09-29T00:49:45.238Z" },+ { url = "https://files.pythonhosted.org/packages/2c/29/817c7eacdeaf8463123e949bd394c39ad024eea1ec38ddf5ad141da2f3bd/regex-2026.9.29-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:01000ddf0e3ffef97f2413ceb514f6313040106b6d18a03ee00a4fe35c1eb1db", size = 809431, upload-time = "2026-09-29T00:49:47.878Z" },+ { url = "https://files.pythonhosted.org/packages/63/0b/83aab3b5b739947f744135a7a3a446e25433ebc92b05e01aae197ccbfdda/regex-2026.9.29-cp315-cp315t-win32.whl", hash = "sha256:c4e38dd8f39c43a91d2410ad2b85610701b0979342c3df1d69eaf8e838c757d8", size = 276964, upload-time = "2026-09-29T00:49:50.524Z" },+ { url = "https://files.pythonhosted.org/packages/72/f2/6314b5fc68789b5dcc38885bc6e3d6986b34fb3372b7231088ee5cecaa05/regex-2026.9.29-cp315-cp315t-win_amd64.whl", hash = "sha256:e2c89e9b762c57f59d5e99ee8b20202adb892e35f8d3485741340999ca55058e", size = 286487, upload-time = "2026-09-29T00:49:53.224Z" },+ { url = "https://files.pythonhosted.org/packages/56/bc/97b2245c8c7b2dd01f2db74f2bea003cd33c15009b4996a2447f46b5325c/regex-2026.9.29-cp315-cp315t-win_arm64.whl", hash = "sha256:e8c65ef3862a8ad6e86492b6ed9327805dd66904c012bd3649dc67d822ed6c34", size = 285955, upload-time = "2026-09-29T00:49:55.655Z" },+]++[[package]]+name = "rich"+version = "15.0.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "markdown-it-py" },+ { name = "pygments" },+]+sdist = { url = "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz", hash = "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36", size = 230680, upload-time = "2026-04-12T08:24:00.75Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" },+]++[[package]]+name = "ruff"+version = "0.17.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/e9/a7/70debb024dfacda67b8e560cc7511f52b34b9348a8cc8c1ec23036dcd51d/ruff-0.17.0.tar.gz", hash = "sha256:5cd03240d8208a557c2a9655a5cb07ebe36aa6bb35065f97d48c1f6adef5a322", size = 5320554, upload-time = "2026-10-09T19:47:29.248Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/bc/f8/ee5ab9da6089eae2a33e6008b01deb1eda19992c1c8e10661e98cee1640f/ruff-0.17.0-py3-none-linux_armv6l.whl", hash = "sha256:0e271826af9a20d18c6cfae8c51e82959167c24859686ddd3eb9a7f0842ce81e", size = 10148907, upload-time = "2026-10-09T19:46:38.695Z" },+ { url = "https://files.pythonhosted.org/packages/9f/d9/2f81fb5a9d580afbb11b1c8ff915233a11f2a1b27405d7991f183c5e1976/ruff-0.17.0-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:5f0ca4a40f81403689c04f12966e22f44e329ae362072d8f1587b7bda87f603b", size = 10224208, upload-time = "2026-10-09T19:46:41.711Z" },+ { url = "https://files.pythonhosted.org/packages/a7/20/643f3c8f75594f937b2bf74801241c56a2e2b8e139d24dff8b66b28cdd7f/ruff-0.17.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:cbf7149e0927dc3295d5d64679a4765576eef71b00782b2ae969ef82274d6bb9", size = 9273324, upload-time = "2026-10-09T19:46:44.323Z" },+ { url = "https://files.pythonhosted.org/packages/ec/91/627700b233d367736cb274f1bd0b47d1f2b12f68878192812bd875adadc3/ruff-0.17.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:13ee90156522998c3037059d8f66885c8adeeaf7643bdce2caceee196ecd23e0", size = 9953190, upload-time = "2026-10-09T19:46:47.155Z" },+ { url = "https://files.pythonhosted.org/packages/cd/92/91f7b5ed39490f89d6cbf56e1f543c383667a725efa8e2c2dee0f01f5591/ruff-0.17.0-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3d8e4a002a94cd9d0dc48b51dc69d807a172b5b9bf2b668e656424dc5b55ead1", size = 9961746, upload-time = "2026-10-09T19:46:50.098Z" },+ { url = "https://files.pythonhosted.org/packages/87/c5/7310f9fc63ce11ff6394edbd5e85433dfb0e14c9fbf6ccc97f1538491bc7/ruff-0.17.0-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c0b8a60c06a218c337e1161638d34757f83449243e2db161483ddf948e53ad14", size = 10625483, upload-time = "2026-10-09T19:46:53.379Z" },+ { url = "https://files.pythonhosted.org/packages/a5/8d/97443f0dca4a03a0bc7629fd396fd494a1cb6666121e38c5075acb217d8f/ruff-0.17.0-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a330178bdffc4205dbf3bda11d93e059e388fd6546f8cdd304501a9160363c0d", size = 11350129, upload-time = "2026-10-09T19:46:56.486Z" },+ { url = "https://files.pythonhosted.org/packages/9c/0a/c525efd9777be4b6b012e6969a3012648468e7e6c4b3e5b46af69f46e8eb/ruff-0.17.0-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7bb08489e234876fa2da67ae3ea938e9a2156da80293e0e4365abd6973d98329", size = 10906223, upload-time = "2026-10-09T19:47:00.263Z" },+ { url = "https://files.pythonhosted.org/packages/2c/3c/4a01195d93420cad1175bedad13a515dc8a56f95a6e39789b92e582682f5/ruff-0.17.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bc73e7c133e82d55b5f15897b2a442d72c0cb4a0c886c46801ce3c247150b60c", size = 10386314, upload-time = "2026-10-09T19:47:03.057Z" },+ { url = "https://files.pythonhosted.org/packages/c7/72/1a3951665485a921f6375f91e754a1854d5a645d41acc3642668064ff64d/ruff-0.17.0-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:db4f74c533403ab70fe4007873f6ae0c9f94a8b03158cf48d78788e47cdbe399", size = 10565464, upload-time = "2026-10-09T19:47:05.831Z" },+ { url = "https://files.pythonhosted.org/packages/90/8c/539b4d8c082f57e18db8ae2be85a460d77861c79dcd5798e32b536a6a06f/ruff-0.17.0-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:3d8cc360e666d1914e47b0777c6906d70cf18891a55532bd0a16844195d70859", size = 10079468, upload-time = "2026-10-09T19:47:08.617Z" },+ { url = "https://files.pythonhosted.org/packages/e0/b8/84286966db79434e8c26b585b0a0f6897cb3ab1c51a4aa4df10c28488b62/ruff-0.17.0-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:d66de796b726c4801e05fa99a2a8d7a780e107be222486c304ab61765561e866", size = 9958556, upload-time = "2026-10-09T19:47:11.324Z" },+ { url = "https://files.pythonhosted.org/packages/69/50/27b6eed27b83fcdd5bfa0d52b83231e29094754374698da404d094487ae3/ruff-0.17.0-py3-none-musllinux_1_2_i686.whl", hash = "sha256:c3f268baf004aea944f040623327119527ea231af15f7fb7890e82cea0679589", size = 10352173, upload-time = "2026-10-09T19:47:14.188Z" },+ { url = "https://files.pythonhosted.org/packages/2a/fa/955399fd13044cd827862044117d784a59e3196f6cce7424908ac9a7f914/ruff-0.17.0-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:864b6c1acb6b0bccf94b5a3938a1531fd09aaca5e5659a2e7bf0f3cf2a685540", size = 10791761, upload-time = "2026-10-09T19:47:16.931Z" },+ { url = "https://files.pythonhosted.org/packages/ae/bf/024e01e1f6aec87768696725e648ed5b438941341eea8f8100beb681961f/ruff-0.17.0-py3-none-win32.whl", hash = "sha256:5e50aa5b84decd9fe5b0bb0e6f71c3b592f1767ed09faa4b7207d933961e35cd", size = 9904112, upload-time = "2026-10-09T19:47:19.75Z" },+ { url = "https://files.pythonhosted.org/packages/cc/77/1ee73df41dcc8d1cdb686ee4bc46ea29704ea175feb6b95c78420f631ab8/ruff-0.17.0-py3-none-win_amd64.whl", hash = "sha256:8ab76bcda86dfd28e13776cb5de3c7bcdcf1ae3d37ed761113d1a5a415dc134c", size = 10115585, upload-time = "2026-10-09T19:47:22.698Z" },+ { url = "https://files.pythonhosted.org/packages/fd/71/eb4f0ccc844aece56963e8578df9c95d4c00f580547d52329d4035d3af18/ruff-0.17.0-py3-none-win_arm64.whl", hash = "sha256:c154c73ff43f9854395e24cac507af13078962e53d2b511605058d22af1fdb88", size = 9872035, upload-time = "2026-10-09T19:47:26.306Z" },+]++[[package]]+name = "safetensors"+version = "0.8.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/45/06/f955dbbb1859e3bd23c8ac6141af5106e7ad5fedec4a3a6e3d60f94b7001/safetensors-0.8.0.tar.gz", hash = "sha256:fabaf3e0f18a6618d9b36560682562157f77c2b71fcffc7b432be2baed9d753d", size = 325846, upload-time = "2026-06-09T07:52:25.563Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/39/a0/f718cda65b05407d228f97602cf60dca269c979867aa5beb25410de26cd3/safetensors-0.8.0-cp310-abi3-macosx_10_12_x86_64.whl", hash = "sha256:c554f85858e05226d3c2828e32395e677434685d6d94594a41643361c5e837f0", size = 473568, upload-time = "2026-06-09T07:52:18.829Z" },+ { url = "https://files.pythonhosted.org/packages/f5/b1/fa7c600e7dceae12e9606c7578cbc9ff1e1ed55844883ee5c92205e86226/safetensors-0.8.0-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:c80201d22cbf405b80647a60ada77bba06c8fba2da2743ba1e89cdcc39a81f25", size = 484562, upload-time = "2026-06-09T07:52:17.518Z" },+ { url = "https://files.pythonhosted.org/packages/09/7d/65a7de0af421317bb36a067241e4235fff194eed60b961ed6d3f59a3fc60/safetensors-0.8.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7a46e5ff292c356d6991e60942ba7f79817682d3a2cef0702136448cb9c4d235", size = 502844, upload-time = "2026-06-09T07:52:07.624Z" },+ { url = "https://files.pythonhosted.org/packages/91/4f/3175c9d75634e0e0dda0082794193521035edd7c70a6f212bf33ca06ddf4/safetensors-0.8.0-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:4124502b78f03534117c848f87a39b8f31e577b15eff423bf8bfb95f2a8c30d0", size = 511823, upload-time = "2026-06-09T07:52:09.565Z" },+ { url = "https://files.pythonhosted.org/packages/20/87/846c289e7aa2299eff406335717cf43ce8777194ece8aad75772e0411615/safetensors-0.8.0-cp310-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7bc0a787ba8a35be368ee3574edfa2b1ad389eebd0a72e482ae275490e3f6c98", size = 633461, upload-time = "2026-06-09T07:52:11.128Z" },+ { url = "https://files.pythonhosted.org/packages/76/22/8d64d9df2c45d5ded401df889d0ad90882804ca172d79ec4f0df8f727fe0/safetensors-0.8.0-cp310-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:040070828e36dc8e122178bbbd5830ff9e97920affb84cbe0f46442497bed358", size = 545148, upload-time = "2026-06-09T07:52:13.603Z" },+ { url = "https://files.pythonhosted.org/packages/28/50/f203ff3a3ddfe19308efc83c5a3a29ed02bf786732ec35e68bf9162f3365/safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fd6f3f93c9a0a7cc2788ee63fb763353d4bd2e89b0751bc78fcf7dda00bea774", size = 516040, upload-time = "2026-06-09T07:52:16.29Z" },+ { url = "https://files.pythonhosted.org/packages/46/fb/cdaed17ceb2948784fd9c36b6fd3e951b608547cea81a48e8ee6f8cfdfcb/safetensors-0.8.0-cp310-abi3-manylinux_2_31_riscv64.whl", hash = "sha256:fcdd41ec4628fee5799f807c73c353629130fbd942aa23d83c623dd6c9d52d78", size = 513832, upload-time = "2026-06-09T07:52:12.37Z" },+ { url = "https://files.pythonhosted.org/packages/0d/49/1e15de264dcc3b77943d2d0c56a95809956883b1c2d6d585c792523f180b/safetensors-0.8.0-cp310-abi3-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:8e9f537aa183a38ace122d27303dcd986b26bd2a7591f9181d7f0c396f4677ca", size = 559930, upload-time = "2026-06-09T07:52:14.743Z" },+ { url = "https://files.pythonhosted.org/packages/2a/43/bf38443278eab4b1be1fce2931e2b012ad9cb7df52ada751d0aab8f7659a/safetensors-0.8.0-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:87eec7ffed2b809f05a398a8becb7d013f19f7837cd15d9748580d6cf30dbaf4", size = 678670, upload-time = "2026-06-09T07:52:20.032Z" },+ { url = "https://files.pythonhosted.org/packages/72/e3/68cd3fa5b48488e84add63e04cb12f3bc28ae4638c06d4508c6e88823d0e/safetensors-0.8.0-cp310-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:4a95ae2b05d7726d751da4ebf626a2ca782b706e101bd894c95bc2450b1cffcc", size = 786679, upload-time = "2026-06-09T07:52:21.322Z" },+ { url = "https://files.pythonhosted.org/packages/29/4b/1c19c509d56e01f4fbb3d0a2e597450f6cc04d1d56cf52defb0a62dfd715/safetensors-0.8.0-cp310-abi3-musllinux_1_2_i686.whl", hash = "sha256:3ae091f16662658bdc019a4ff6cb4c085bb7d725eb5978b183ffd265863b6d2d", size = 765683, upload-time = "2026-06-09T07:52:22.594Z" },+ { url = "https://files.pythonhosted.org/packages/27/43/41c1621732edd934d868a00d1b891584c892a7b62a9aab82ea5a0a5623ee/safetensors-0.8.0-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:8e080062fcde23be189565e1c3305d16751a218ecf9412c8601e64204eb6f846", size = 722361, upload-time = "2026-06-09T07:52:23.924Z" },+ { url = "https://files.pythonhosted.org/packages/8e/3f/73ccf82579412b4a71c4ca673f10b5f1f888d7cf5af7fe24f27d30307be4/safetensors-0.8.0-cp310-abi3-win32.whl", hash = "sha256:2ddf52eac562eda224f99acfa7889d02968c1fd59a5b011ae7d8137c37e9c02d", size = 342401, upload-time = "2026-06-09T07:52:28.895Z" },+ { url = "https://files.pythonhosted.org/packages/1b/6d/3fba214c1e5e0f69991677ec3bc17023f0421776975e1de0c682dca475e2/safetensors-0.8.0-cp310-abi3-win_amd64.whl", hash = "sha256:096ec1a98435df7beb08853bb5aa9081a84f23d0adc67ed1a0a10550f608373f", size = 355540, upload-time = "2026-06-09T07:52:27.832Z" },+ { url = "https://files.pythonhosted.org/packages/8d/fc/7eedc3510d97878876e32774eebbeb61c43f148a96e915c84229a3e967aa/safetensors-0.8.0-cp310-abi3-win_arm64.whl", hash = "sha256:f7838e5135a406ad3e02efdcb8cf2e5397d368b0154537c4fec682dbc544d452", size = 340500, upload-time = "2026-06-09T07:52:26.745Z" },+]++[[package]]+name = "setuptools"+version = "84.0.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/6d/44/f5da03a8ef95d369145c5bb53050e7877c9f3d312e128605fd9504829143/setuptools-84.0.0.tar.gz", hash = "sha256:f4695c21257f0d9b537ec2692c941d02ee143b7cc1276941349a546573b2ef73", size = 1168449, upload-time = "2026-08-08T18:27:58.365Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/95/9c/c510029fc6ef33a6275cd2c5d3cecd6613dfd6aa401d57c54f1c18852ccf/setuptools-84.0.0-py3-none-any.whl", hash = "sha256:51a52592b3b99e102b609654876bd65f19f999935166d1352678931132b0c670", size = 818216, upload-time = "2026-08-08T18:27:56.719Z" },+]++[[package]]+name = "shellingham"+version = "1.5.4"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/58/15/8b3609fd3830ef7b27b655beb4b4e9c62313a4e8da8c676e142cc210d58e/shellingham-1.5.4.tar.gz", hash = "sha256:8dbca0739d487e5bd35ab3ca4b36e11c4078f3a234bfce294b0a0291363404de", size = 10310, upload-time = "2023-10-24T04:13:40.426Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl", hash = "sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", size = 9755, upload-time = "2023-10-24T04:13:38.866Z" },+]++[[package]]+name = "sympy"+version = "1.14.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "mpmath" },+]+sdist = { url = "https://files.pythonhosted.org/packages/83/d3/803453b36afefb7c2bb238361cd4ae6125a569b4db67cd9e79846ba2d68c/sympy-1.14.0.tar.gz", hash = "sha256:d3d3fe8df1e5a0b42f0e7bdf50541697dbe7d23746e894990c030e2b05e72517", size = 7793921, upload-time = "2025-04-27T18:05:01.611Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353, upload-time = "2025-04-27T18:04:59.103Z" },+]++[[package]]+name = "tokenizers"+version = "0.23.3"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "huggingface-hub" },+]+sdist = { url = "https://files.pythonhosted.org/packages/e0/7c/2cabb2174e772636683008f2c5621949b645da7d303c596589e84516a184/tokenizers-0.23.3.tar.gz", hash = "sha256:cded33237c77caeef62944d32aa9a7ef42bdce2b3497e18d137e072a8c4be438", size = 385286, upload-time = "2026-10-09T10:16:55.759Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/aa/2e/4ce5b9716f26e526eff6b0502ebed4ea8d7161f03b3c77617c9f25528e97/tokenizers-0.23.3-cp310-abi3-macosx_10_12_x86_64.whl", hash = "sha256:9d2b5c97daf61688c2ad1803ca851800feaba50fb68d5821779e9ea5880d968c", size = 3148800, upload-time = "2026-10-09T10:00:51.457Z" },+ { url = "https://files.pythonhosted.org/packages/b2/72/01e49f032bb346e5aaf06c10c74fe8aeec847173adbadd66eb7c53054bf2/tokenizers-0.23.3-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:68649e97d5b43c44c031d8d848874a6eecae8f8fe40ea989aa777a5a83aca716", size = 3101381, upload-time = "2026-10-09T10:00:54.063Z" },+ { url = "https://files.pythonhosted.org/packages/15/fc/ae987741829b1cd547668c4c94be732ae3eefd1d74344e64c3d2ca714acd/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ec82e80e65a862275b97c3d90b7a523df8d9519ee48aeb4e9625b2cc909274e0", size = 3519944, upload-time = "2026-10-09T10:00:55.885Z" },+ { url = "https://files.pythonhosted.org/packages/1c/da/cc8f6c030afaf05fbddc608158fbb761dca46913cbeba6b112e59fc82e2a/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:c64a0713180ff16829d4e7f39a658b77ea11443af4e1aa46523692943c9b1414", size = 3397695, upload-time = "2026-10-09T10:00:57.444Z" },+ { url = "https://files.pythonhosted.org/packages/ec/f1/256f78d1365fa2cd3ea6db716883d74667c8cbb6a21f15fa5b89a773cdc2/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ddedfd4b3b4be6be24ff6ca645c4a37fddfd305f6f3e354c54cf10b715c48215", size = 3753125, upload-time = "2026-10-09T10:01:00.165Z" },+ { url = "https://files.pythonhosted.org/packages/60/93/eee007ac2fcbf4ecfce7fbc354826cf3611f56bdb886f3e91b1f7dd06b8f/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:2a89614730d7b80940a5d2ed9320e1ec8add5a745c6151d8d05071b7215505b6", size = 4018598, upload-time = "2026-10-09T10:01:02.05Z" },+ { url = "https://files.pythonhosted.org/packages/bf/f9/0c96c4739461fce9d8d865b416728081bf6230022d7163bd6244f35f4b31/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e88646b8580c5ad7f4361477f1298e9cc01771a1ee9aecfe32c47b8ff614cc38", size = 3602442, upload-time = "2026-10-09T10:01:03.77Z" },+ { url = "https://files.pythonhosted.org/packages/3a/40/6706b82693715581457c6d5423eaa7faae576bb0526c5738a57085eb4449/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:376851d22bcf9d650a5c3090bb83e6cf9e895fbf0595369fa4cd43c1f69b5f87", size = 3396193, upload-time = "2026-10-09T10:01:05.48Z" },+ { url = "https://files.pythonhosted.org/packages/fe/0c/85946de40e25b7364b8f1bcf56def129069acd5bb364b7c86a32919e1a23/tokenizers-0.23.3-cp310-abi3-manylinux_2_31_riscv64.whl", hash = "sha256:bf501c40b72d2d5c8623620210430e9cac1ce47a46e45b34107b70a1557d46b0", size = 3553483, upload-time = "2026-10-09T10:01:07.387Z" },+ { url = "https://files.pythonhosted.org/packages/f1/6b/8d615d92cad1d511ca5ab188d1c7c167f0b3d295cc0d96207f9f82d486d8/tokenizers-0.23.3-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:114e2b55ed177179d59f4ab98200a4471e11e78f9e4b5a922d146740f96fcf52", size = 9972248, upload-time = "2026-10-09T10:01:09.437Z" },+ { url = "https://files.pythonhosted.org/packages/c9/7d/a922e37ddd58d1b463bbc2ad08120c8f59c60b814cd353519a116b24f8ba/tokenizers-0.23.3-cp310-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:d3407fb7b9c4d75dd68850ffd7180bc0a5d2dbaf0762d888e612f31fec3f9c6b", size = 9802957, upload-time = "2026-10-09T10:01:11.869Z" },+ { url = "https://files.pythonhosted.org/packages/4b/06/5d3f506a86ae0699a0e4ea05c05978f9aee169ef2c1d844e68c971cf8194/tokenizers-0.23.3-cp310-abi3-musllinux_1_2_i686.whl", hash = "sha256:84513ef0aeb8bf8f4ea11a2e8a7ac163ec5288aa115e649a59b470ac5c3107df", size = 10145487, upload-time = "2026-10-09T10:01:14.268Z" },+ { url = "https://files.pythonhosted.org/packages/26/e5/065625317690ea3548d834dad81f48ea1fd32e4964610e658e195d7fe28e/tokenizers-0.23.3-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:e05ab7baf7f47b406a95fea6f3b0a484b2ddcd9e1d14b68844c457eb755085a3", size = 10266026, upload-time = "2026-10-09T10:16:33.054Z" },+ { url = "https://files.pythonhosted.org/packages/77/4e/babede85d0d19f5e3deeef0063e01848141329934d3d77c31b5cab5ac2b4/tokenizers-0.23.3-cp310-abi3-win32.whl", hash = "sha256:1ebf28794e7e4954e20a7f70fbea410b2d1f0418f7dbbca97ca384fcfef38c25", size = 2588086, upload-time = "2026-10-09T10:16:35.686Z" },+ { url = "https://files.pythonhosted.org/packages/d1/6c/24f074c9a0efb98e61b20aafe6b2641922d5db24e447d5d6daffd9e17555/tokenizers-0.23.3-cp310-abi3-win_amd64.whl", hash = "sha256:1f0823bb00c5fdc98e487354d54dd55a03848d61a1a0bf29a68c77f24f3b26c3", size = 2872101, upload-time = "2026-10-09T10:16:37.533Z" },+ { url = "https://files.pythonhosted.org/packages/53/77/a476b6f73a661c11d113a342d2326b91506cf2285f0995d1212a6bb2022d/tokenizers-0.23.3-cp310-abi3-win_arm64.whl", hash = "sha256:7e48734d2de9260d86f03ab056d2cfeeff3869f61dbd49aaa15a2793b5f3458b", size = 2742580, upload-time = "2026-10-09T10:16:39.244Z" },+ { url = "https://files.pythonhosted.org/packages/65/46/f66baaedd42414a3f583c47379dc350e3e1f858a690d2574fd85ae70681b/tokenizers-0.23.3-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:efa3d7318406b4d115dce61ad5061953f1f44b128e79c020ce4615d763e23b6e", size = 3154274, upload-time = "2026-10-09T10:16:40.876Z" },+ { url = "https://files.pythonhosted.org/packages/c6/41/8de8c63b2d935eee5a0f42011fb7b786ffafeab0b8eb6d17acb8af2293b7/tokenizers-0.23.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:a4fbb3662f9f59d199d61338e54b4bcc11d07ebbb1aeb3540dacb2be9c521cb7", size = 3077805, upload-time = "2026-10-09T10:16:42.856Z" },+ { url = "https://files.pythonhosted.org/packages/e3/08/b1cbae8dc8fc7c91f992ac2d87a086e9b3f25a28814047ca16a82fe8c87b/tokenizers-0.23.3-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:de536665495cb4b409d25bade41963f801aff4225c19a6b804b048f7d14e34c7", size = 3491678, upload-time = "2026-10-09T10:16:45.093Z" },+ { url = "https://files.pythonhosted.org/packages/3e/0d/aac0cb2f3a1fdbef514145b4c5f2df4d05deeb1ee8f73ae641a1b4a62a85/tokenizers-0.23.3-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5cc24bb457dd4a8af89c8fcb40074d570129ec473df2a866c276ee55db4749d7", size = 3367420, upload-time = "2026-10-09T10:16:47.112Z" },+ { url = "https://files.pythonhosted.org/packages/1e/1d/41a697d0c193a320b243fbd68b2057b6eb2f01ecf80899e1a16e646ff699/tokenizers-0.23.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:acd5c57b4bd3e56e246e2731a3a3a6825a7a7d89b7e3b761ba80bc521710f04b", size = 9945973, upload-time = "2026-10-09T10:16:49.326Z" },+ { url = "https://files.pythonhosted.org/packages/37/e9/b56e619fcd583000a2b1254bb46af8dc6a174d3ba3329f454ad5a95a2be2/tokenizers-0.23.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:82eb480f6f1c21cea3349dec32cf1a6384c6c1e775f00f83b0d51197bc013687", size = 10237491, upload-time = "2026-10-09T10:16:51.943Z" },+ { url = "https://files.pythonhosted.org/packages/6f/68/f58b3beb95f3b62816e91e5e768e684cd63e58f9cbece22036dae3b1c971/tokenizers-0.23.3-cp314-cp314t-win_amd64.whl", hash = "sha256:1554a6eed34d9d6a78d23360f4e06df8dffab1ae08c7e8488e0b3e3b36cc266f", size = 2847654, upload-time = "2026-10-09T10:16:54.166Z" },+]++[[package]]+name = "torch"+version = "2.14.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "cuda-bindings", marker = "python_full_version < '3.15' and sys_platform == 'linux'" },+ { name = "cuda-toolkit", extra = ["cublas", "cudart", "cufft", "cufile", "cupti", "curand", "cusolver", "cusparse", "nvjitlink", "nvrtc", "nvtx"], marker = "sys_platform == 'linux'" },+ { name = "filelock" },+ { name = "fsspec" },+ { name = "jinja2" },+ { name = "networkx", version = "3.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },+ { name = "networkx", version = "3.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },+ { name = "nvidia-cudnn-cu13", marker = "sys_platform == 'linux'" },+ { name = "nvidia-cusparselt-cu13", marker = "sys_platform == 'linux'" },+ { name = "nvidia-nccl-cu13", marker = "sys_platform == 'linux'" },+ { name = "nvidia-nvshmem-cu13", marker = "sys_platform == 'linux'" },+ { name = "setuptools" },+ { name = "sympy" },+ { name = "triton", marker = "python_full_version < '3.15' and sys_platform == 'linux'" },+ { name = "typing-extensions" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/5d/9b/fd1cd143a7bda5abc1fc352ed7b431f6f52aa3c454293211a33656db2cb5/torch-2.14.1-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:b6074b130fd26bc50f5d171f07dfed70db22dd0c354ab34767729249f9e0f042", size = 127281154, upload-time = "2026-09-30T17:43:58.328Z" },+ { url = "https://files.pythonhosted.org/packages/8d/0e/21281ff4015779255229f487f4481c5bfba53b8c83e01eaa4a8a8f1180c9/torch-2.14.1-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:ef1276fd07adb8d463ac402a24ea7482e45c9a42df8ebdac211aa757779200ba", size = 453982557, upload-time = "2026-09-30T17:44:52.656Z" },+ { url = "https://files.pythonhosted.org/packages/aa/4f/7af6fbada912b3023cc79b6ab30f89234b258a9a373880ea52ed914a2ae9/torch-2.14.1-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:305a61f61f35f128579f299c5bd33d475f6a01c6307336139632e30856c4854d", size = 554581699, upload-time = "2026-09-30T17:49:10.777Z" },+ { url = "https://files.pythonhosted.org/packages/fe/54/4e0a71376b2fb15aaf65ed9e6a4c8043d36f554338c6daabc63059a5e157/torch-2.14.1-cp311-cp311-win_amd64.whl", hash = "sha256:0bd2fb0c4098856bc5af59fe5988bf38bdbcdb6550b44273c12ae425cb8ec17e", size = 124096268, upload-time = "2026-09-30T17:44:04.362Z" },+ { url = "https://files.pythonhosted.org/packages/4b/3c/379349820e9a6e713826de3d0b88f07728541bdcabe2a3afd128871cdc4b/torch-2.14.1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:420dbf314c180ee4b86e9bc00aee5746a7d6e5bacdd7df925af671ed393f0b2e", size = 127307099, upload-time = "2026-09-30T17:44:10.421Z" },+ { url = "https://files.pythonhosted.org/packages/a4/3e/474930ab971e306957ec763b7df695d659179e15ccab9e7073d47ae14ce8/torch-2.14.1-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:0b13acde294b401509830d85a8906cbf2fc97425a12a66cce1d9fa13c5106a2a", size = 453998918, upload-time = "2026-09-30T17:52:45.134Z" },+ { url = "https://files.pythonhosted.org/packages/ba/fd/c470857d0dfc5d4c94430658b3eed60fd5292b0bf596a8aae7b9fd66d674/torch-2.14.1-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:23011fe29a99b591eabb3a31a25080c62e2c2f0690a3c798744e5489dae50451", size = 554618695, upload-time = "2026-09-30T17:53:06.8Z" },+ { url = "https://files.pythonhosted.org/packages/ad/1a/b9648b0d228e513f69f8453ec3f90a8d436d78ff6d85a2773e539f0c07cf/torch-2.14.1-cp312-cp312-win_amd64.whl", hash = "sha256:38bee9f2a2ccfc6898172a5075e4ba52522fbe143eeb099674fc67ab390e858d", size = 124113906, upload-time = "2026-09-30T17:52:21.377Z" },+ { url = "https://files.pythonhosted.org/packages/7d/11/faaca4f8541c45127b7e0d6bb141221fe944c8466d89986b8466c6c195f9/torch-2.14.1-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:dbe359d705f4d67236743794c296c6dee93a922fd8117eff8c3e880d7d0fb2b9", size = 127315286, upload-time = "2026-09-30T17:52:27.43Z" },+ { url = "https://files.pythonhosted.org/packages/60/1f/0330275c705b846882531c64e80d32b13fb572eeb9b995e87e84154b0712/torch-2.14.1-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:6d530bd11479fb574567af3a1f384af9a77bf5bb9550cde8080b356ca7220d5a", size = 453999200, upload-time = "2026-09-30T17:53:21.716Z" },+ { url = "https://files.pythonhosted.org/packages/7c/cc/bb579ac0c80e077c58204e43b37a254b74dea5873635100bea25789a55d4/torch-2.14.1-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:c8f71aabc67bcbfc9373dc131537a5968d04edce73e88add21354a7cd0a76985", size = 554618164, upload-time = "2026-09-30T17:53:42.73Z" },+ { url = "https://files.pythonhosted.org/packages/7b/4b/32c00cacbe682a4c08d9c6c3e7a7116c912184cc7155f06c64f545db97d1/torch-2.14.1-cp313-cp313-win_amd64.whl", hash = "sha256:711713391d26a1ce5e9fbc6c996d954a8c12e8825374cc77b809a6af29539b8c", size = 124113754, upload-time = "2026-09-30T17:52:33.474Z" },+ { url = "https://files.pythonhosted.org/packages/43/19/23a1aed488423a5055727256b25406e4b93bd2bcf1352bef582b9951c10c/torch-2.14.1-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:cee091caf2a6229e248daf41d18ceebf590ba02d9179205610062c64ee5fef03", size = 127325330, upload-time = "2026-09-30T17:52:54.752Z" },+ { url = "https://files.pythonhosted.org/packages/93/f4/94219ada13edd62fda1f976163292b7b8595f1fd2f4ef74af24b3baad375/torch-2.14.1-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:f68f5476e2bc0e8f60b74f7ca21acda885c4477af6392f8977e4f0d1ea1aa162", size = 454016297, upload-time = "2026-09-30T17:54:03.078Z" },+ { url = "https://files.pythonhosted.org/packages/fd/df/23c69e9b9fd19fe6563422f1bc59f89601bda47234a89a1cf432cfde3aee/torch-2.14.1-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:1d4df07be7338bbcc4d54085adee21363c91314702b9bd4d78ef72ffac9465ba", size = 554621744, upload-time = "2026-09-30T17:54:22.951Z" },+ { url = "https://files.pythonhosted.org/packages/d9/dc/a36a4431ab5e3ad168a75f341e24677a4bbbf0c5dda59c97a7079262bb94/torch-2.14.1-cp314-cp314-win_amd64.whl", hash = "sha256:d02a4c48a2ca5fb7654e36e71f710f74494d83f1f10aaff8e059dd554adca956", size = 124110781, upload-time = "2026-09-30T17:53:31.141Z" },+ { url = "https://files.pythonhosted.org/packages/2d/bc/1afbd1a22f6023eafcdb9295ac1d97137a7816f54091d693701072da2218/torch-2.14.1-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:06c3ec25f3b497f9a73dc0c36f293f295cfe8d5584fd446238d9a501b30a66d5", size = 127657364, upload-time = "2026-09-30T17:53:53.837Z" },+ { url = "https://files.pythonhosted.org/packages/e4/94/b97e863c9ceef2bc6e082e967cbfb9065b5e232f2a60c5af0f5115a1bc31/torch-2.14.1-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:9113f94f70429f9f302bf55b090b5411269083e8a72a5d015ffcc5a7f83f2c69", size = 453998592, upload-time = "2026-09-30T17:54:38.268Z" },+ { url = "https://files.pythonhosted.org/packages/03/d8/8272157c438cc26a199a8fa0caf87985477b8a1ed8a09084eb2ae40609d8/torch-2.14.1-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:e65d5fe136e533b23c2d134377f7f126721c7c78dd75b3dbb735af082a8aeb85", size = 554578730, upload-time = "2026-09-30T17:54:54.838Z" },+ { url = "https://files.pythonhosted.org/packages/45/05/451a69a4287033d8f106f5c65f92c2c0c37229d81ea101d4b047caf758cc/torch-2.14.1-cp314-cp314t-win_amd64.whl", hash = "sha256:e07306caa1de2a4ac1467e11ecfc92fc44f523dd6a521145039aef46d913963c", size = 124400129, upload-time = "2026-09-30T17:54:12.306Z" },+]++[[package]]+name = "tqdm"+version = "4.70.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "colorama", marker = "sys_platform == 'win32'" },+]+sdist = { url = "https://files.pythonhosted.org/packages/0d/ea/b2a5bd54b28a324dae8211928b2d730b6547500342c7e6c6dea08bd0a485/tqdm-4.70.1.tar.gz", hash = "sha256:cefd0eca11b2a37a3aee776544d4f4ae913f02688135b5556b8788dfa474afc4", size = 171846, upload-time = "2026-09-11T07:25:16.601Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/a7/03/921a3d3c75785aca9ebfbfcabfbc3a1be12e2ab5265deb026d55a5a3f83e/tqdm-4.70.1-py3-none-any.whl", hash = "sha256:c293e525e6fef9c20e8728fd4612df02a0aa31bb5fe91ecd93e123b1b7bffa73", size = 80199, upload-time = "2026-09-11T07:25:14.599Z" },+]++[[package]]+name = "transformers"+version = "5.19.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "huggingface-hub" },+ { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },+ { name = "numpy", version = "2.5.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },+ { name = "packaging" },+ { name = "pyyaml" },+ { name = "regex" },+ { name = "safetensors" },+ { name = "tokenizers" },+ { name = "tqdm" },+ { name = "typer" },+]+sdist = { url = "https://files.pythonhosted.org/packages/65/4c/70012ed0795235fb6bd7806fafd4a1842f03e49ebbfa3bb9580e451948ce/transformers-5.19.0.tar.gz", hash = "sha256:87f38dd25e4521151b97e94520ac457f44a0ae8a8358a5b112daff6c64a822d6", size = 10032022, upload-time = "2026-10-06T16:39:00.229Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b9/59/1e7f1212c215c73690ae6cab4678efe1555ff985953e145d88cc212859b0/transformers-5.19.0-py3-none-any.whl", hash = "sha256:afcd2dd5f603ed28c1e1fcb00a338ccbb4ef5f878ed289635df8b58187afb518", size = 12599123, upload-time = "2026-10-06T16:38:56.794Z" },+]++[[package]]+name = "triton"+version = "3.8.0"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/a3/cf/d21c9b1a4d1df9ba3aed218069f24f762a4b9565c5a362bef6f110c08e72/triton-3.8.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:372285307d4c44ee74cee32de0b4f04bd157e071427e38c6f4ee3e3beb2194f4", size = 226467015, upload-time = "2026-08-28T16:08:05.871Z" },+ { url = "https://files.pythonhosted.org/packages/6f/9a/2c3d3823726d5ad5359f7065b02b984925158b74611c1e3987f6a37461fb/triton-3.8.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:68988ac85d5e7086baeda0ddc175af9667db7529b3c5e11a5c0601b8bef2200a", size = 247945226, upload-time = "2026-08-28T15:55:43.795Z" },+ { url = "https://files.pythonhosted.org/packages/87/07/0f8cd8e8db0472334253efdaaab3d0819fea27aa99bf0e7f1aeea4ceb5ae/triton-3.8.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a9c404c69ed4a39e8ec632eaf6b9fe058a060bf98979c177f6ef666f06bb8d50", size = 226474486, upload-time = "2026-08-28T16:08:18.29Z" },+ { url = "https://files.pythonhosted.org/packages/c1/09/b7012e5bfae67640f268aa584caa80fe1674f6b0da949046b679972c33e3/triton-3.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e91ffa46d095b252248297292dd22bcbacd53a125a0c2eefbbbf74925a320bc3", size = 247972921, upload-time = "2026-08-28T15:55:53.157Z" },+ { url = "https://files.pythonhosted.org/packages/87/4d/4c564374bcdadb166fccbf3e45aee0d4a473f88d341761bd2fefe3b8e8c1/triton-3.8.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b7004666652f500ed854a86988e4b3d69d247188b5d2092b5df1e44f4a954099", size = 226476793, upload-time = "2026-08-28T16:08:30.956Z" },+ { url = "https://files.pythonhosted.org/packages/b0/b6/3394d5548404c1cabd1dadadd28d0b3f9478db1dff8180da53bb3f0a1e19/triton-3.8.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1f0497218e26b7d79773ad9c2a3fa3b539ee69f587a13fac2e552b1d322a8015", size = 247975122, upload-time = "2026-08-28T15:56:04.112Z" },+ { url = "https://files.pythonhosted.org/packages/b8/59/bf0e9493118bb353ab59a5d6a65db3618d9b314417cc1459f0121e0ec5c9/triton-3.8.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1f6b48d0591929a3867973acac3dccd4e058585f91bfb41022de496c9ffab304", size = 226488654, upload-time = "2026-08-28T16:08:47.141Z" },+ { url = "https://files.pythonhosted.org/packages/93/d9/08c75f3459f19ad00425b564058e40efa4bcd79b816064cf27499303ea42/triton-3.8.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:387dae4cb0089a7b6ba1a428ae0782b65c4c58f57d94617cb22ca8593d8ccbca", size = 247972313, upload-time = "2026-08-28T15:56:14.007Z" },+ { url = "https://files.pythonhosted.org/packages/7c/34/429c5592181cfb7361a0a8e0bff218e7224b726709d75da2472b3e819f70/triton-3.8.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b84e7d512490ba529111260fa6f7cad8b254a6bb5fbdf41d5ef9a5e57f52d0a", size = 226591133, upload-time = "2026-08-28T16:09:02.271Z" },+ { url = "https://files.pythonhosted.org/packages/fe/d1/aa8a3e935c37efee7945984fdb64d7e0851bf6d920afd97b2d21f9d23360/triton-3.8.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:74217bb56ed8692759227758e4c4b3bd2d608a209c1a7a081bf361fb4c2c1bf9", size = 248077577, upload-time = "2026-08-28T15:56:24.94Z" },+]++[[package]]+name = "truststore"+version = "0.10.5"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/ee/9f/c5201d42a484c061e528825fc8e2d565f5abd50a4ced6fb7d29c4ec99b2b/truststore-0.10.5.tar.gz", hash = "sha256:30d36967ccaded5cbb38d602c433f53600036c79d502f4533a49b60a03bbefcd", size = 28091, upload-time = "2026-10-12T22:27:31.808Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/51/e9/3a7820be2bb0fe53b6bc9c3be26d3d1158004e4c3ab953aa6840b955b1e9/truststore-0.10.5-py3-none-any.whl", hash = "sha256:9aaaedaefaf06d8b206278cf8b5012bc897f485a874503501e12d776df78951c", size = 19017, upload-time = "2026-10-12T22:27:30.377Z" },+]++[[package]]+name = "typer"+version = "0.27.3"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "annotated-doc" },+ { name = "colorama", marker = "sys_platform == 'win32'" },+ { name = "rich" },+ { name = "shellingham" },+]+sdist = { url = "https://files.pythonhosted.org/packages/03/51/d33db42cc72ffd8c30777547b42d01f0cbf9d95a770457698d0174b3ed71/typer-0.27.3.tar.gz", hash = "sha256:d0396f770a560ab1b0a8504e13b5f254b728cedb05c61cf0359e944e50ce8901", size = 205303, upload-time = "2026-10-06T17:24:16.61Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/07/ea/2e31b67051e91a133189e9c000c222502ddc6969856416de0d095de4c0b0/typer-0.27.3-py3-none-any.whl", hash = "sha256:e50022f28b82a86313e54501317a1db64bf8f8d036ff8cfe5ca7e47675454aff", size = 123312, upload-time = "2026-10-06T17:24:15.054Z" },+]++[[package]]+name = "typing-extensions"+version = "4.16.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz", hash = "sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5", size = 113555, upload-time = "2026-07-02T08:40:05.92Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", hash = "sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", size = 45571, upload-time = "2026-07-02T08:40:04.659Z" },+]diff --git a/games/ATTRIBUTION.md b/games/ATTRIBUTION.mdnew file mode 100644index 0000000..5fb34a2--- /dev/null+++ b/games/ATTRIBUTION.md@@ -0,0 +1,127 @@+# Game credits++ROMs in `roms/`. CHIP-8 Archive games are CC0. The classics come from the Revival+Studios CHIP-8 pack, which "can be freely distributed in its original form"; each+game belongs to its author.++| Game | Authors | Year | License | Source |+|---|---|---|---|---|+| 15 Puzzle | Roger Ivie | 19xx | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/15%20Puzzle%20%5BRoger%20Ivie%5D.ch8) |+| 1D Cellular Automata | SharpenedSpoon | 2014 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/1dcell.ch8) |+| 8CE Attourny - Disc 1 | SystemLogoff | 2016 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/8ceattourny_d1.ch8) |+| 8CE Attourny - Disc 2 | SystemLogoff | 2016 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/8ceattourny_d2.ch8) |+| 8CE Attourny - Disc 3 | SystemLogoff | 2016 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/8ceattourny_d3.ch8) |+| Addition Problems | Paul C. Moews | 1979 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Addition%20Problems%20%5BPaul%20C.%20Moews%5D.ch8) |+| Airplane | unknown | 19xx | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Airplane.ch8) |+| Animal Race | Brian Astle | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Animal%20Race%20%5BBrian%20Astle%5D.ch8) |+| Astro Dodge | Revival Studios, Martijn Wenting | 2008 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Astro%20Dodge%20%5BRevival%20Studios%2C%202008%5D.ch8) |+| Bad Kaiju Ju | MattBooth | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/BadKaiJuJu.ch8) |+| Biorhythm | Jef Winsor | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Biorhythm%20%5BJef%20Winsor%5D.ch8) |+| Blinky | Hans Christian Egeberg | 1991 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Blinky%20%5BHans%20Christian%20Egeberg%2C%201991%5D.ch8) |+| Blitz | David Winter | 1996 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Blitz%20%5BDavid%20Winter%5D.ch8) |+| Bowling | Gooitzen van der Wal | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Bowling%20%5BGooitzen%20van%20der%20Wal%5D.ch8) |+| Br8kout | SharpenedSpoon | 2014 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/br8kout.ch8) |+| Breakout | David Winter | 1997 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Breakout%20%28Brix%20hack%29%20%5BDavid%20Winter%2C%201997%5D.ch8) |+| Breakout | Carmelo Cortez | 1979 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Breakout%20%5BCarmelo%20Cortez%2C%201979%5D.ch8) |+| Brick | unknown | 1990 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Brick%20%28Brix%20hack%2C%201990%29.ch8) |+| Brix | Andreas Gustafsson | 1990 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Brix%20%5BAndreas%20Gustafsson%2C%201990%5D.ch8) |+| Carbon8 | Chromatophore | 2018 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/carbon8.ch8) |+| Cave | unknown | 19xx | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Cave.ch8) |+| Cave Explorer | JohnEarnest | 2014 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/caveexplorer.ch8) |+| Chipquarium | mattmik | 2016 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/chipquarium.ch8) |+| ChipWar | JohnEarnest | 2014 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/chipwar.ch8) |+| Coin Flipping | Carmelo Cortez | 1978 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Coin%20Flipping%20%5BCarmelo%20Cortez%2C%201978%5D.ch8) |+| Connect 4 | David Winter | 1996 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Connect%204%20%5BDavid%20Winter%5D.ch8) |+| Craps | Camerlo Cortez | 1978 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Craps%20%5BCamerlo%20Cortez%2C%201978%5D.ch8) |+| Danm8ku | buffi | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/danm8ku.ch8) |+| Deflection | John Fort | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Deflection%20%5BJohn%20Fort%5D.ch8) |+| Dino Run | Andrezinrc | 2026 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/dinorun.ch8) |+| down8 | tinaun | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/down8.ch8) |+| Figures | unknown | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Figures.ch8) |+| Filter | unknown | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Filter.ch8) |+| Flight Runner | TodPunk | 2014 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/flightrunner.ch8) |+| Fuse | JohnEarnest | 2016 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/fuse.ch8) |+| Ghost Escape | TomRintjema | 2016 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/ghostEscape.ch8) |+| Glitch Ghost | JackieKircher | 2014 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/glitchGhost.ch8) |+| Guess the Number | David Winter | 1996 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Guess%20%5BDavid%20Winter%5D.ch8) |+| Hi-Lo | Jef Winsor | 1978 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Hi-Lo%20%5BJef%20Winsor%2C%201978%5D.ch8) |+| Hidden | David Winter | 1996 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Hidden%20%5BDavid%20Winter%2C%201996%5D.ch8) |+| Horse World Online | TomRintjema | 2014 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/horseWorldOnline.ch8) |+| Kaleidoscope | Joseph Weisbecker | 1978 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Kaleidoscope%20%5BJoseph%20Weisbecker%2C%201978%5D.ch8) |+| Knumber Knower | JohnEarnest | 2023 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/knumberknower.ch8) |+| Landing | unknown | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Landing.ch8) |+| Lunar Lander | unknown | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Lunar%20Lander%20%28Udo%20Pernisz%2C%201979%29.ch8) |+| Masquer8 | Chromatophore | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/masquer8.ch8) |+| Mastermind | WilliamDonnelly | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/mastermind.ch8) |+| Mastermind FourRow | Robert Lindley | 1978 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Mastermind%20FourRow%20%28Robert%20Lindley%2C%201978%29.ch8) |+| Mato8 | nortti | 2023 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/mato8.ch8) |+| Merlin | David Winter | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Merlin%20%5BDavid%20Winter%5D.ch8) |+| Mini Lights Out | tobiasvl | 2019 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/mini-lights-out.ch8) |+| Missile Command | David Winter | 1996 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Missile%20%5BDavid%20Winter%5D.ch8) |+| Most Dangerous Game | Peter Maruhnic | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Most%20Dangerous%20Game%20%5BPeter%20Maruhnic%5D.ch8) |+| Nim | Carmelo Cortez | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Nim%20%5BCarmelo%20Cortez%2C%201978%5D.ch8) |+| Octo Rancher | SystemLogoff | 2018 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octorancher.ch8) |+| Octo: a Chip 8 Story | SystemLogoff | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octoachip8story.ch8) |+| Octojam 1 Title | JohnEarnest | 2014 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam1title.ch8) |+| Octojam 10 Title | JohnEarnest | 2023 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam10title.ch8) |+| Octojam 2 Title | JohnEarnest | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam2title.ch8) |+| Octojam 3 Title | JohnEarnest | 2016 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam3title.ch8) |+| Octojam 4 Title | JohnEarnest | 2017 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam4title.ch8) |+| Octojam 5 Title | JohnEarnest | 2018 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam5title.ch8) |+| Octojam 6 Title | JohnEarnest | 2019 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam6title.ch8) |+| Octojam 7 Title | JohnEarnest | 2020 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam7title.ch8) |+| Octojam 8 Title | JohnEarnest | 2021 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam8title.ch8) |+| Octojam 9 Title | JohnEarnest | 2022 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/octojam9title.ch8) |+| Outlaw | JohnEarnest | 2014 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/outlaw.ch8) |+| Paddles | unknown | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Paddles.ch8) |+| Pet Dog | SystemLogoff | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/petdog.ch8) |+| Piper | Aeris, JordanMecom, LillianWang | 2017 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/piper.ch8) |+| Pong | 1 player | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Pong%20%281%20player%29.ch8) |+| Pong | Paul Vervalin | 1990 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Pong%20%5BPaul%20Vervalin%2C%201990%5D.ch8) |+| Pong 2 | David Winter | 1997 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Pong%202%20%28Pong%20hack%29%20%5BDavid%20Winter%2C%201997%5D.ch8) |+| Programmable Spacefighters | Jef Winsor | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Programmable%20Spacefighters%20%5BJef%20Winsor%5D.ch8) |+| Pumpkin "Dreess" Up | SystemLogoff | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/pumpkindressup.ch8) |+| Puzzle | unknown | 199x | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Puzzle.ch8) |+| Reversi | unknown | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Reversi%20%5BPhilip%20Baltzer%5D.ch8) |+| Rocket | Joseph Weisbecker | 1978-12 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Rocket%20%5BJoseph%20Weisbecker%2C%201978%5D.ch8) |+| Rocket Launch | Jonas Lindstedt | 19xx | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Rocket%20Launch%20%5BJonas%20Lindstedt%5D.ch8) |+| Rocket Launcher | unknown | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Rocket%20Launcher.ch8) |+| RPS | SystemLogoff | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/RPS.ch8) |+| Rush Hour | Hap | 2006-12-17 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Rush%20Hour%20%5BHap%2C%202006%5D.ch8) |+| Russian Roulette | Carmelo Cortez | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Russian%20Roulette%20%5BCarmelo%20Cortez%2C%201978%5D.ch8) |+| Sequence Shoot | Joyce Weisbecker | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Sequence%20Shoot%20%5BJoyce%20Weisbecker%5D.ch8) |+| Shooting Stars | Philip Baltzer | 1978 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Shooting%20Stars%20%5BPhilip%20Baltzer%2C%201978%5D.ch8) |+| Slide | Joyce Weisbecker | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Slide%20%5BJoyce%20Weisbecker%5D.ch8) |+| Slippery Slope | JohnEarnest | 2018 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/slipperyslope.ch8) |+| Snek | JohnEarnest | 2021 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/snek.ch8) |+| Soccer | unknown | 19xx | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Soccer.ch8) |+| Space Flight | unknown | 19xx | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Space%20Flight.ch8) |+| Space Intercept | unknown | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Space%20Intercept%20%5BJoseph%20Weisbecker%2C%201978%5D.ch8) |+| Space Invaders | David Winter | 1996 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Space%20Invaders%20%5BDavid%20Winter%5D.ch8) |+| Space Racer | WilliamDonnelly | 2017 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/spaceracer.ch8) |+| Spacejam! | WilliamDonnelly | 2015 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/spacejam.ch8) |+| Spock Paper Scissors | fontz | 2020 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/spockpaperscissors.ch8) |+| Spooky Spot | Joseph Weisbecker | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Spooky%20Spot%20%5BJoseph%20Weisbecker%2C%201978%5D.ch8) |+| Squash | David Winter | 1997 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Squash%20%5BDavid%20Winter%5D.ch8) |+| Submarine | Carmelo Cortez | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Submarine%20%5BCarmelo%20Cortez%2C%201978%5D.ch8) |+| Sum Fun | Joyce Weisbecker | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Sum%20Fun%20%5BJoyce%20Weisbecker%5D.ch8) |+| Super Pong | offstatic | 2021 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/superpong.ch8) |+| SuperWorm V4 | RB-Revival Studios, Martijn Wenting | 2007 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Worm%20V4%20%5BRB-Revival%20Studios%2C%202007%5D.ch8) |+| Syzygy | Roy Trevino | 1990 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Syzygy%20%5BRoy%20Trevino%2C%201990%5D.ch8) |+| Tank Battle | unknown | 197x | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Tank.ch8) |+| Tank! | Rectus | 2018 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/tank.ch8) |+| Tapeworm | JDR | 1999 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Tapeworm%20%5BJDR%2C%201999%5D.ch8) |+| Tetris | Fran Dachille | 1991 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Tetris%20%5BFran%20Dachille%2C%201991%5D.ch8) |+| TIC-TAC-TOE | unknown | 1996 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Tic-Tac-Toe%20%5BDavid%20Winter%5D.ch8) |+| Tick | StanislavDidus | 2026 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/tick.ch8) |+| Timebomb | unknown | 19xx | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Timebomb.ch8) |+| TOMB STON TIPP | TomRintjema | 2018 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/tombstontipp.ch8) |+| Tron | unknown | 19xx | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Tron.ch8) |+| UFO | Lutz V | 1992 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/UFO%20%5BLutz%20V%2C%201992%5D.ch8) |+| Vers | JMN | 1991 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Vers%20%5BJMN%2C%201991%5D.ch8) |+| Vertical Brix | Paul Robson | 1996 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Vertical%20Brix%20%5BPaul%20Robson%2C%201996%5D.ch8) |+| Wall | David Winter | 1997 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Wall%20%5BDavid%20Winter%5D.ch8) |+| WDL | JohnEarnest | 2022 | CC0 | [link](https://github.com/JohnEarnest/chip8Archive/master/roms/wdl.ch8) |+| Wipe Off | Joseph Weisbecker | 19xx | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/Wipe%20Off%20%5BJoseph%20Weisbecker%5D.ch8) |+| X-Mirror | unknown | | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/X-Mirror.ch8) |+| ZeroPong | zeroZshadow | 2007 | freely distributable (Revival Studios CHIP-8 pack) | [link](https://github.com/kripod/chip8-roms/blob/master/games/ZeroPong%20%5BzeroZshadow%2C%202007%5D.ch8) |diff --git a/games/manifest.json b/games/manifest.jsonnew file mode 100644index 0000000..9067176--- /dev/null+++ b/games/manifest.json@@ -0,0 +1,3951 @@+{+ "games": [+ {+ "id": "octojam-1-title",+ "title": "Octojam 1 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2014",+ "description": "Greeting program for the Octo-ber 2014 Chip8 Game Jam.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam1title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-1-title.ch8",+ "sha1": "f26993a4afd5cda2fea19935773fd3db54866623",+ "status": "unsupported",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {+ "DXY0 (SCHIP 16x16 sprite)": 4+ },+ "tokens_per_instruction": 5.56,+ "realtime_tok_per_s": 2335+ },+ "tap": []+ },+ {+ "id": "octojam-2-title",+ "title": "Octojam 2 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2015",+ "description": "This title card draws a nearly full screen 64x30 animation with very low flicker.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam2title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-2-title.ch8",+ "sha1": "11c68038d64a09be549a6c1e50724808914d8991",+ "status": "ok",+ "result": {+ "verified_records": 23303,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 21.82,+ "realtime_tok_per_s": 9165+ },+ "tap": []+ },+ {+ "id": "octojam-3-title",+ "title": "Octojam 3 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2016",+ "description": "This title card disambiguates at runtime between a COSMAC VIP, SCHIP, and Octo.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam3title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-3-title.ch8",+ "sha1": "493c76b9d9252e1d349d408d341daff5267f35fa",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.83,+ "realtime_tok_per_s": 2870+ },+ "tap": []+ },+ {+ "id": "octojam-4-title",+ "title": "Octojam 4 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2017",+ "description": "This title card demonstrates the new :macro and :calc directives.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam4title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 30,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-4-title.ch8",+ "sha1": "9f55f7abc8f2bc4b59a01515f1d887a6568a8ab4",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 8.04,+ "realtime_tok_per_s": 14475+ },+ "tap": []+ },+ {+ "id": "octojam-5-title",+ "title": "Octojam 5 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2018",+ "description": "Greeting program for the fifth annual 2018 Octojam.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam5title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-5-title.ch8",+ "sha1": "d6d8efef811350e7fba6197024c4973b360749b8",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 8.2,+ "realtime_tok_per_s": 3443+ },+ "tap": []+ },+ {+ "id": "octojam-6-title",+ "title": "Octojam 6 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2019",+ "description": "Greeting program for the sixth annual 2019 Octojam.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam6title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-6-title.ch8",+ "sha1": "18aef6d2d3b560681038d0dda2273d780dc1daa5",+ "status": "ok",+ "result": {+ "verified_records": 23304,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 25.22,+ "realtime_tok_per_s": 10592+ },+ "tap": []+ },+ {+ "id": "octojam-7-title",+ "title": "Octojam 7 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2020",+ "description": "Greeting program for the seventh annual 2020 Octojam.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam7title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 30,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-7-title.ch8",+ "sha1": "9a9c341571ace516c9789b1eb92590833af13239",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 9.83,+ "realtime_tok_per_s": 17694+ },+ "tap": []+ },+ {+ "id": "octojam-8-title",+ "title": "Octojam 8 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2021",+ "description": "Greeting program for the eighth annual 2021 Octojam.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam8title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-8-title.ch8",+ "sha1": "959ed7d6b61e667bb59d1b497401258463f88454",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 9.49,+ "realtime_tok_per_s": 3986+ },+ "tap": []+ },+ {+ "id": "octojam-9-title",+ "title": "Octojam 9 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2022",+ "description": "Greetings program for the ninth annual 2022 Octojam.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam9title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-9-title.ch8",+ "sha1": "5a6366decb08df66da8bd685b497ccce0884c307",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.47,+ "realtime_tok_per_s": 4925+ },+ "tap": []+ },+ {+ "id": "octojam-10-title",+ "title": "Octojam 10 Title",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2023",+ "description": "Greetings program for the tenth annual 2023 Octojam.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octojam10title.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octojam-10-title.ch8",+ "sha1": "1dda7d92450c8eee2753a9350a1a0d524b1019b4",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 10.96,+ "realtime_tok_per_s": 4602+ },+ "tap": []+ },+ {+ "id": "fuse",+ "title": "Fuse",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2016",+ "description": "A minimalist reinterpretation of the classic Pipe Dream.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/fuse.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/fuse.ch8",+ "sha1": "0cd895dc3d489d0e40656218900a04310e95f560",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.69,+ "realtime_tok_per_s": 4225+ },+ "tap": []+ },+ {+ "id": "slippery-slope",+ "title": "Slippery Slope",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2018",+ "description": "The ice we skate is getting pretty thin. The water's getting warm so you might as well swim.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/slipperyslope.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 30,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/slippery-slope.ch8",+ "sha1": "9d834860f455aec7e95fb886984497e5be501610",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.25,+ "realtime_tok_per_s": 11256+ },+ "tap": []+ },+ {+ "id": "chipwar",+ "title": "ChipWar",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2014",+ "description": "A (loose) adaptation of the classic DiceWars.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/chipwar.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/chipwar.ch8",+ "sha1": "ab5cbf267d74c168e174041b9594ae856cbd671d",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.24,+ "realtime_tok_per_s": 6514+ },+ "tap": []+ },+ {+ "id": "cave-explorer",+ "title": "Cave Explorer",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2014",+ "description": "Do you have what it takes to be a Cave Explorer?",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/caveexplorer.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 20,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/cave-explorer.ch8",+ "sha1": "400dbd1aa2b79b9b8546bc615bfb735c1bd1d268",+ "status": "ok",+ "result": {+ "verified_records": 21155,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.19,+ "realtime_tok_per_s": 6226+ },+ "controls": "Arrow keys move",+ "featured": 9,+ "tap": []+ },+ {+ "id": "outlaw",+ "title": "Outlaw",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2014",+ "description": "An adaptation of the Atari 2600 game \"Outlaw\" for the Chip8.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/outlaw.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/outlaw.ch8",+ "sha1": "a9d3c975a5e733646a04f6e61deebcd0ad50f700",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.06,+ "realtime_tok_per_s": 6352+ },+ "controls": "Arrow keys move, Space shoots",+ "featured": 10,+ "tap": []+ },+ {+ "id": "rps",+ "title": "RPS",+ "authors": [+ "SystemLogoff"+ ],+ "year": "2015",+ "description": "Classic \"Rock Paper Scissors\" game for the Chip8.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/RPS.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/rps.ch8",+ "sha1": "a6f3ac2d89cdc1d7b22013301863bad6a4fb7318",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.13,+ "realtime_tok_per_s": 2156+ },+ "tap": []+ },+ {+ "id": "octo-a-chip-8-story",+ "title": "Octo: a Chip 8 Story",+ "authors": [+ "SystemLogoff"+ ],+ "year": "2015",+ "description": "A visual novel game for the Chip8.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octoachip8story.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octo-a-chip-8-story.ch8",+ "sha1": "39970ccfd3a3f00180d53464d4fd7862193eaf0f",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.59,+ "realtime_tok_per_s": 3187+ },+ "tap": []+ },+ {+ "id": "pumpkin-dreess-up",+ "title": "Pumpkin \"Dreess\" Up",+ "authors": [+ "SystemLogoff"+ ],+ "year": "2015",+ "description": "Classic Adobe Flash games, now in Chip8.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/pumpkindressup.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/pumpkin-dreess-up.ch8",+ "sha1": "b2abb5312f0ad28421c1190a65a73d98d4ebf401",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.19,+ "realtime_tok_per_s": 3018+ },+ "tap": []+ },+ {+ "id": "pet-dog",+ "title": "Pet Dog",+ "authors": [+ "SystemLogoff"+ ],+ "year": "2015",+ "description": "Undertale fangame made in Chip8.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/petdog.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/pet-dog.ch8",+ "sha1": "3be683d1ac0b27ae47a09984e420853fff0b7e0d",+ "status": "ok",+ "result": {+ "verified_records": 23303,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 12.86,+ "realtime_tok_per_s": 5403+ },+ "tap": []+ },+ {+ "id": "8ce-attourny--disc-1",+ "title": "8CE Attourny - Disc 1",+ "authors": [+ "SystemLogoff"+ ],+ "year": "2016",+ "description": "Exciting court battles in Chip8! Disc 1/3 of an exciting multi-tape game.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/8ceattourny_d1.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/8ce-attourny--disc-1.ch8",+ "sha1": "3ddf7b76b8f63d0089e00e3b518f78c213b74b1e",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.05,+ "realtime_tok_per_s": 2961+ },+ "tap": []+ },+ {+ "id": "8ce-attourny--disc-2",+ "title": "8CE Attourny - Disc 2",+ "authors": [+ "SystemLogoff"+ ],+ "year": "2016",+ "description": "Exciting court battles in Chip8! Disc 2/3 of an exciting multi-tape game.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/8ceattourny_d2.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/8ce-attourny--disc-2.ch8",+ "sha1": "82fd0d202a068bedfb869fc303fdeae0c814024f",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.15,+ "realtime_tok_per_s": 2582+ },+ "tap": []+ },+ {+ "id": "8ce-attourny--disc-3",+ "title": "8CE Attourny - Disc 3",+ "authors": [+ "SystemLogoff"+ ],+ "year": "2016",+ "description": "Exciting court battles in Chip8! Disc 3/3 of an exciting multi-tape game.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/8ceattourny_d3.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/8ce-attourny--disc-3.ch8",+ "sha1": "419a0110d41332457c15ae09fff62cbd7ad197fc",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.04,+ "realtime_tok_per_s": 2956+ },+ "tap": []+ },+ {+ "id": "octo-rancher",+ "title": "Octo Rancher",+ "authors": [+ "SystemLogoff"+ ],+ "year": "2018",+ "description": "Train monsters and become a real ranching champion. Read the manual at https://systemlogoff.com/share/Octo-Rancher-Manual.pdf",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/octorancher.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/octo-rancher.ch8",+ "sha1": "8263bac7d98d94097171f0a5dc6f210f77543080",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.75,+ "realtime_tok_per_s": 2413+ },+ "tap": []+ },+ {+ "id": "tank",+ "title": "Tank!",+ "authors": [+ "Rectus"+ ],+ "year": "2018",+ "description": "A two player artillery game.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/tank.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 200,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/tank.ch8",+ "sha1": "3f9ef8dec999574a188ec3b9615cff9888283c85",+ "status": "ok",+ "result": {+ "verified_records": 20116,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.73,+ "realtime_tok_per_s": 44816+ },+ "tap": []+ },+ {+ "id": "flight-runner",+ "title": "Flight Runner",+ "authors": [+ "TodPunk"+ ],+ "year": "2014",+ "description": "A simple flight runner for the Chip8.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/flightrunner.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/flight-runner.ch8",+ "sha1": "821751787374cc362f4c58759961f0aa7a2fd410",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.63,+ "realtime_tok_per_s": 4168+ },+ "controls": "Up/Down steer",+ "featured": 11,+ "tap": []+ },+ {+ "id": "masquer8",+ "title": "Masquer8",+ "authors": [+ "Chromatophore"+ ],+ "year": "2015",+ "description": "Cuttlefish pattern matching game for chip8.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/masquer8.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/masquer8.ch8",+ "sha1": "fb48e162c7f2e8853909acc5534b55fb55030f9f",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.53,+ "realtime_tok_per_s": 5879+ },+ "tap": []+ },+ {+ "id": "carbon8",+ "title": "Carbon8",+ "authors": [+ "Chromatophore"+ ],+ "year": "2018",+ "description": "A period correct movie tie in game for a popular 1980s franchise.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/carbon8.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/carbon8.ch8",+ "sha1": "2e0f2268c9a3be0fbb839f918336f161815bb80a",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.32,+ "realtime_tok_per_s": 4791+ },+ "tap": []+ },+ {+ "id": "bad-kaiju-ju",+ "title": "Bad Kaiju Ju",+ "authors": [+ "MattBooth"+ ],+ "year": "2015",+ "description": "The year is 1977. 90% of the world's landmass is now underwater and the world is at war...",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/BadKaiJuJu.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/bad-kaiju-ju.ch8",+ "sha1": "0f479a10fec51d159866e5760069cd18bdfd293f",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.35,+ "realtime_tok_per_s": 2667+ },+ "tap": []+ },+ {+ "id": "piper",+ "title": "Piper",+ "authors": [+ "Aeris",+ "JordanMecom",+ "LillianWang"+ ],+ "year": "2017",+ "description": "Play as a small bird to collect goodies and avoid waves.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/piper.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 200,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/piper.ch8",+ "sha1": "7da3eba52a8d8025ddf14ee40d28f151585529a0",+ "status": "ok",+ "result": {+ "verified_records": 20116,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.54,+ "realtime_tok_per_s": 54516+ },+ "tap": []+ },+ {+ "id": "mini-lights-out",+ "title": "Mini Lights Out",+ "authors": [+ "tobiasvl"+ ],+ "year": "2019",+ "description": "Turn the lights out",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/mini-lights-out.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/mini-lights-out.ch8",+ "sha1": "a3f0eae99964b873eb1adbd3e8bcb90d15f762c3",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.5,+ "realtime_tok_per_s": 2729+ },+ "tap": []+ },+ {+ "id": "horse-world-online",+ "title": "Horse World Online",+ "authors": [+ "TomRintjema"+ ],+ "year": "2014",+ "description": "An online horse breeding game.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/horseWorldOnline.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 30,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/horse-world-online.ch8",+ "sha1": "28ac3467fbb4544a3e3a1ec3cd27d9e819ac7323",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.18,+ "realtime_tok_per_s": 7515+ },+ "tap": []+ },+ {+ "id": "ghost-escape",+ "title": "Ghost Escape",+ "authors": [+ "TomRintjema"+ ],+ "year": "2016",+ "description": "Seek revenge on mortal men in this exciting adventure!",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/ghostEscape.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/ghost-escape.ch8",+ "sha1": "1e3be162480380b6276d0848e1c71576b4c041f2",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.79,+ "realtime_tok_per_s": 2432+ },+ "tap": []+ },+ {+ "id": "tomb-ston-tipp",+ "title": "TOMB STON TIPP",+ "authors": [+ "TomRintjema"+ ],+ "year": "2018",+ "description": "Take turns pressing buttons to see who wins.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/tombstontipp.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/tomb-ston-tipp.ch8",+ "sha1": "6e7cb52ec99e10f934b76eaf3fddeb8f2e2e14e1",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.75,+ "realtime_tok_per_s": 2415+ },+ "tap": []+ },+ {+ "id": "br8kout",+ "title": "Br8kout",+ "authors": [+ "SharpenedSpoon"+ ],+ "year": "2014",+ "description": "You can't even imagine how amazing this breakout clone is!",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/br8kout.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 7,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/br8kout.ch8",+ "sha1": "31fc1c53cc610a9f4b9c5705c5a0f33fc028d123",+ "status": "ok",+ "result": {+ "verified_records": 23302,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.09,+ "realtime_tok_per_s": 2560+ },+ "controls": "Left/Right move the paddle",+ "featured": 8,+ "tap": []+ },+ {+ "id": "1d-cellular-automata",+ "title": "1D Cellular Automata",+ "authors": [+ "SharpenedSpoon"+ ],+ "year": "2014",+ "description": "An implementation of 1D Cellular Automata.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/1dcell.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 1000,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/1d-cellular-automata.ch8",+ "sha1": "018442698067c95d67e27a94e6642c11f049f108",+ "status": "ok",+ "result": {+ "verified_records": 20024,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.38,+ "realtime_tok_per_s": 322707+ },+ "tap": []+ },+ {+ "id": "spacejam",+ "title": "Spacejam!",+ "authors": [+ "WilliamDonnelly"+ ],+ "year": "2015",+ "description": "An extended and enhanced version loosely based on ShipTunnel from the 2014 OctoJam.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/spacejam.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 100,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/spacejam.ch8",+ "sha1": "9f9a4affbf7afd70bb594fb321e16579318c0164",+ "status": "ok",+ "result": {+ "verified_records": 20235,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.86,+ "realtime_tok_per_s": 23140+ },+ "tap": []+ },+ {+ "id": "mastermind",+ "title": "Mastermind",+ "authors": [+ "WilliamDonnelly"+ ],+ "year": "2015",+ "description": "The classic puzzle game of deduction.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/mastermind.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 20,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/mastermind.ch8",+ "sha1": "898ef1505c874065697ffc6cba688367e143d82e",+ "status": "ok",+ "result": {+ "verified_records": 21155,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.83,+ "realtime_tok_per_s": 6997+ },+ "tap": []+ },+ {+ "id": "space-racer",+ "title": "Space Racer",+ "authors": [+ "WilliamDonnelly"+ ],+ "year": "2017",+ "description": "Based on the classic 2-player 'Space Race' video game.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/spaceracer.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 20,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9+ },+ "notes": [],+ "file": "roms/space-racer.ch8",+ "sha1": "659cb966e976fcbcae76f6a8a07c65be4d18aae8",+ "status": "ok",+ "result": {+ "verified_records": 21155,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 8.99,+ "realtime_tok_per_s": 10782+ },+ "tap": []+ },+ {+ "id": "down8",+ "title": "down8",+ "authors": [+ "tinaun"+ ],+ "year": "2015",+ "description": "you fall down an endless hole and then you die!",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/down8.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/down8.ch8",+ "sha1": "1368d7eae124661aacaf3411819ca9c113c0c10c",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.88,+ "realtime_tok_per_s": 4395+ },+ "controls": "Left/Right move",+ "featured": 12,+ "tap": []+ },+ {+ "id": "danm8ku",+ "title": "Danm8ku",+ "authors": [+ "buffi"+ ],+ "year": "2015",+ "description": "Bullet hell for the Chip-8! Survive 8 waves of exciting bullet patterns!",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/danm8ku.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 1000,+ "keys": {+ "up": 5,+ "down": 8,+ "left": 7,+ "right": 9+ },+ "notes": [],+ "file": "roms/danm8ku.ch8",+ "sha1": "ff6b8ac59bf281cd4b5ab6e161600b00f85a0265",+ "status": "ok",+ "result": {+ "verified_records": 20024,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.23,+ "realtime_tok_per_s": 313647+ },+ "tap": []+ },+ {+ "id": "chipquarium",+ "title": "Chipquarium",+ "authors": [+ "mattmik"+ ],+ "year": "2016",+ "description": "A CHIP-8 fish tank simulator.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/chipquarium.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/chipquarium.ch8",+ "sha1": "f4392681b1fa38d7ad0a7d7a59cecf247ac1457a",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.65,+ "realtime_tok_per_s": 5089+ },+ "tap": []+ },+ {+ "id": "glitch-ghost",+ "title": "Glitch Ghost",+ "authors": [+ "JackieKircher"+ ],+ "year": "2014",+ "description": "glitch ghost is all alone in the cemetery, please help them make some friends by turning the trespassing humans into new ghosts!",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/glitchGhost.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 200,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/glitch-ghost.ch8",+ "sha1": "2cdcb3c29a5f013a991db5909ca8e18e27b3c42b",+ "status": "ok",+ "result": {+ "verified_records": 20116,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.95,+ "realtime_tok_per_s": 47360+ },+ "tap": []+ },+ {+ "id": "spock-paper-scissors",+ "title": "Spock Paper Scissors",+ "authors": [+ "fontz"+ ],+ "year": "2020",+ "description": "Intergalactic Rock Paper Scissors.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/spockpaperscissors.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/spock-paper-scissors.ch8",+ "sha1": "b66f55f83eb264d2b73c0b4ac81ea5044bf73138",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.49,+ "realtime_tok_per_s": 4042+ },+ "tap": []+ },+ {+ "id": "snek",+ "title": "Snek",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2021",+ "description": "Minimalist Snake in 65 bytes.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/snek.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 1000,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/snek.ch8",+ "sha1": "bb5740042385cae10724b051208bb95e5341f56d",+ "status": "ok",+ "result": {+ "verified_records": 20024,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.68,+ "realtime_tok_per_s": 220539+ },+ "tap": []+ },+ {+ "id": "super-pong",+ "title": "Super Pong",+ "authors": [+ "offstatic"+ ],+ "year": "2021",+ "description": "A fast pong-like.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/superpong.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 30,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/super-pong.ch8",+ "sha1": "2c761f70a44e521ee848834cfdd2bd1646157d29",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.95,+ "realtime_tok_per_s": 7110+ },+ "controls": "WASD moves the paddles, Space (E) resets",+ "featured": 5,+ "tap": []+ },+ {+ "id": "wdl",+ "title": "WDL",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2022",+ "description": "That word game, but shorter.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/wdl.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/wdl.ch8",+ "sha1": "a8d6e9b1976c99ddc0c4818828a6d3cb3ae6f348",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.95,+ "realtime_tok_per_s": 3555+ },+ "tap": []+ },+ {+ "id": "knumber-knower",+ "title": "Knumber Knower",+ "authors": [+ "JohnEarnest"+ ],+ "year": "2023",+ "description": "A game of bad typography.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/knumberknower.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "right": 9+ },+ "notes": [],+ "file": "roms/knumber-knower.ch8",+ "sha1": "95f4af2d410c1acfd1b0657721a870a57c55b49b",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.05,+ "realtime_tok_per_s": 5445+ },+ "tap": []+ },+ {+ "id": "dino-run",+ "title": "Dino Run",+ "authors": [+ "Andrezinrc"+ ],+ "year": "2026",+ "description": "An endless runner game for Chip-8.",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/dinorun.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/dino-run.ch8",+ "sha1": "1a7a2138161b06e083722f978f8cbaaa1ad5344d",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.65,+ "realtime_tok_per_s": 4189+ },+ "controls": "Up or Space jumps",+ "featured": 13,+ "tap": []+ },+ {+ "id": "tick",+ "title": "Tick",+ "authors": [+ "StanislavDidus"+ ],+ "year": "2026",+ "description": "Rhythm one button game for Chip-8",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/tick.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [+ "logic"+ ],+ "ipf": 12,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/tick.ch8",+ "sha1": "3d81c0462c722c2d8a79f6e1b571e9aa21d36b1f",+ "status": "ok",+ "result": {+ "verified_records": 21916,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.26,+ "realtime_tok_per_s": 5227+ },+ "tap": []+ },+ {+ "id": "mato8",+ "title": "Mato8",+ "authors": [+ "nortti"+ ],+ "year": "2023",+ "description": "Snake game (matopeli in Finnish)",+ "source": "https://github.com/JohnEarnest/chip8Archive/master/roms/mato8.ch8",+ "license": "CC0",+ "platform": "octo",+ "quirks": [],+ "ipf": 20,+ "keys": {+ "up": 5,+ "left": 7,+ "down": 8,+ "right": 9,+ "a": 6+ },+ "notes": [],+ "file": "roms/mato8.ch8",+ "sha1": "673f98c66168190aa5b1e5dda5d0d80da7826dd1",+ "status": "ok",+ "result": {+ "verified_records": 21155,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.18,+ "realtime_tok_per_s": 6215+ },+ "tap": []+ },+ {+ "id": "15-puzzle",+ "title": "15 Puzzle",+ "authors": [+ "Roger Ivie"+ ],+ "year": "19xx",+ "description": "15 Puzzle, by Roger Ivie Rearrange the tiles into the sequence 0-F. Moves tile by it associated key.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/15%20Puzzle%20%5BRoger%20Ivie%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/15-puzzle.ch8",+ "sha1": "ea9af3c09b0d9e265fcd92bcc5d51a2939fdf27a",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.2,+ "realtime_tok_per_s": 3777+ },+ "tap": []+ },+ {+ "id": "addition-problems",+ "title": "Addition Problems",+ "authors": [+ "Paul C. Moews"+ ],+ "year": "1979",+ "description": "Addition Problems, by Paul C. Moews Do simple math.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Addition%20Problems%20%5BPaul%20C.%20Moews%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/addition-problems.ch8",+ "sha1": "feaa2b999737630a6402e990df4d0558f79ba43e",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 10.73,+ "realtime_tok_per_s": 9657+ },+ "tap": []+ },+ {+ "id": "airplane",+ "title": "Airplane",+ "authors": [],+ "year": "19xx",+ "description": "Airplane - Blitz type of bombing game. Hit \"8\" to drop a bomb.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Airplane.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/airplane.ch8",+ "sha1": "fca71182a8838b686573e69b22aff945d79fe1d0",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.78,+ "realtime_tok_per_s": 5204+ },+ "tap": []+ },+ {+ "id": "animal-race",+ "title": "Animal Race",+ "authors": [+ "Brian Astle"+ ],+ "year": null,+ "description": "This version does not have the Fx55/Fx65 (load_store_quirk) issue (where score won't increment and rendered garbage). I believe this is the original version.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Animal%20Race%20%5BBrian%20Astle%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/animal-race.ch8",+ "sha1": "a27dcf88a931f70c3ccf3c01a5410b263bac48bc",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.67,+ "realtime_tok_per_s": 6005+ },+ "tap": []+ },+ {+ "id": "astro-dodge",+ "title": "Astro Dodge",+ "authors": [+ "Revival Studios",+ "Martijn Wenting"+ ],+ "year": "2008",+ "description": "----------------------------------------------------------------------------- ///////////////// ////////////////// //// //// //// /////////// //// /////////// //// //// //// /////////// //// ////////// www.revival-studios.com\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Astro%20Dodge%20%5BRevival%20Studios%2C%202008%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled",+ "not in the CHIP-8 database as a lores CHIP-8 game; COSMAC quirks assumed"+ ],+ "file": "roms/astro-dodge.ch8",+ "sha1": "ac621d9fcada302ba6965768229ef130630bc525",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.24,+ "realtime_tok_per_s": 6514+ },+ "tap": []+ },+ {+ "id": "biorhythm",+ "title": "Biorhythm",+ "authors": [+ "Jef Winsor"+ ],+ "year": null,+ "description": "13. VIP Biorhythm The theory of Biorhythm states that there are thre predominant cycles that can influence human behavior. These include a 23-day physical cycle, a 28-day emotional cycle and a 33-day intellectual cycle. All three cycles start at birth and continue throughout\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Biorhythm%20%5BJef%20Winsor%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/biorhythm.ch8",+ "sha1": "3368d56efeb584c509bafb548f1ee5e71ac1bc70",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.66,+ "realtime_tok_per_s": 5995+ },+ "tap": []+ },+ {+ "id": "blinky",+ "title": "Blinky",+ "authors": [+ "Hans Christian Egeberg"+ ],+ "year": "1991",+ "description": "Blinky (1991), by Hans Christian Egeberg Pacman clone. 3, 6 - down/up. 7, 8 - left/right",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Blinky%20%5BHans%20Christian%20Egeberg%2C%201991%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "superchip",+ "quirks": [+ "shift",+ "loadstore",+ "clip",+ "jump"+ ],+ "ipf": 30,+ "keys": {+ "up": 6,+ "down": 3,+ "left": 7,+ "right": 8+ },+ "notes": [],+ "file": "roms/blinky.ch8",+ "sha1": "d40abc54374e4343639f993e897e00904ddf85d9",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.41,+ "realtime_tok_per_s": 9732+ },+ "controls": "Arrow keys steer through the maze",+ "featured": 3,+ "tap": []+ },+ {+ "id": "blitz",+ "title": "Blitz",+ "authors": [+ "David Winter"+ ],+ "year": "1996",+ "description": "Blitz, by David Winter This game is a BOMBER clone. You are in a plane, and you must destroy the towers of a town. Your plane is flying left to right, and goes down. Use 5 to drop a bomb. The game ends when you crash yourself on a tower...",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Blitz%20%5BDavid%20Winter%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "a": 5+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/blitz.ch8",+ "sha1": "6f6509f38220e057a7e32ebb22dd353c1078e3e7",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.52,+ "realtime_tok_per_s": 4071+ },+ "tap": []+ },+ {+ "id": "bowling",+ "title": "Bowling",+ "authors": [+ "Gooitzen van der Wal"+ ],+ "year": null,+ "description": "7. VIP Bowling Bowling is a great game for recreation and competion requiring skill and a little bit of luck. This program simulates bowling closely with regular scoring and the option of using three different spins on the ball. How to play VIP Bowling 1. Load the CHIP-8\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Bowling%20%5BGooitzen%20van%20der%20Wal%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/bowling.ch8",+ "sha1": "b3fed4ed1eb0ed693c9731dbe53b29a76236c781",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.43,+ "realtime_tok_per_s": 4884+ },+ "tap": []+ },+ {+ "id": "breakout",+ "title": "Breakout",+ "authors": [+ "David Winter"+ ],+ "year": "1997",+ "description": "Breakout (Brix hack) 1997, by David Winter This game is an \"arkanoid\" precursor. You have 5 lives, and your goal is the destruction of all the brixs. Use 4 and 6 to move your paddle. The game ends when all the brixs are destroyed. This game is the same than BRIX, but has\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Breakout%20%28Brix%20hack%29%20%5BDavid%20Winter%2C%201997%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "left": 4,+ "right": 6+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/breakout.ch8",+ "sha1": "237756a4014fb3aa82a29246a7cdd534f8dc2dbb",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.71,+ "realtime_tok_per_s": 4237+ },+ "tap": []+ },+ {+ "id": "breakout-2",+ "title": "Breakout",+ "authors": [+ "Carmelo Cortez"+ ],+ "year": "1979",+ "description": "Breakout, by: Carmelo Cortez The game, Breakout, is a variation of the Wipe-Off game. You have six walls and 20 balls to start. To win you must get through all walls to the top of the screen. At the end of the game the program will show the number of times you hit the walls and\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Breakout%20%5BCarmelo%20Cortez%2C%201979%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/breakout-2.ch8",+ "sha1": "193915dcde1365ae054c4eaa21a35baa27cd3356",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.55,+ "realtime_tok_per_s": 4993+ },+ "tap": []+ },+ {+ "id": "brick",+ "title": "Brick",+ "authors": [],+ "year": "1990",+ "description": "BRICK (Brix hack), 1990 A modified version of BRIX, a CHIP-8 game. Original BRIX by Andreas Gustafsson. This one is a solid wall; no air between bricks! 4, 6 - left/right",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Brick%20%28Brix%20hack%2C%201990%29.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/brick.ch8",+ "sha1": "91442577a6bbf8c3267f2df95fdfc50baebe176d",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.12,+ "realtime_tok_per_s": 4605+ },+ "tap": []+ },+ {+ "id": "brix",+ "title": "Brix",+ "authors": [+ "Andreas Gustafsson"+ ],+ "year": "1990",+ "description": "Brix - by Andreas Gustafsson (1990) ----------------------------------- This game is an \"arkanoid\" precursor. You have 5 lives, and your goal is the destruction of all the brixs. Use 4 and 6 to move your paddle. The game ends when all the brixs are destroyed.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Brix%20%5BAndreas%20Gustafsson%2C%201990%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "left": 4,+ "right": 6+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/brix.ch8",+ "sha1": "f13766c14aeb02ad8d4d103cb5eadd282d20cddc",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.48,+ "realtime_tok_per_s": 4033+ },+ "controls": "Left/Right move the paddle",+ "featured": 2,+ "tap": []+ },+ {+ "id": "cave",+ "title": "Cave",+ "authors": [],+ "year": "19xx",+ "description": "Cave (199x) Guide the dot to end of the cave maze without bumping into a wall. Navigation: 4, 6, 2, 8.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Cave.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/cave.ch8",+ "sha1": "5c82520906073287a3ef781746c67207ca084d93",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.7,+ "realtime_tok_per_s": 4226+ },+ "tap": []+ },+ {+ "id": "coin-flipping",+ "title": "Coin Flipping",+ "authors": [+ "Carmelo Cortez"+ ],+ "year": "1978",+ "description": "Coin Flipping (1978), by Carmelo Cortez The game is a Coin FlLpping program. Flip run up and the computer starts to flip a coin, and at the same tine shosing heads and tails on the screen, stopping at the value set in VC.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Coin%20Flipping%20%5BCarmelo%20Cortez%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/coin-flipping.ch8",+ "sha1": "614a2b3d0bb5d62a16d963ac2d3a79eb3dd22742",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.52,+ "realtime_tok_per_s": 4068+ },+ "tap": []+ },+ {+ "id": "connect-4",+ "title": "Connect 4",+ "authors": [+ "David Winter"+ ],+ "year": "1996",+ "description": "Connect 4, by David Winter This game is for two players. The goal is to align 4 coins in the game area. Each player's coins are colored. When you drop a coin, it is paced on the latest dropped coin in the same column, or at the bottom if the column is empty. Once the column is\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Connect%204%20%5BDavid%20Winter%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "superchip",+ "quirks": [+ "shift",+ "loadstore",+ "clip",+ "jump"+ ],+ "ipf": 30,+ "keys": {+ "left": 4,+ "right": 6,+ "a": 5+ },+ "notes": [],+ "file": "roms/connect-4.ch8",+ "sha1": "2d10c07b532f4fa7c07a07324ba26ca39fe484fd",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.85,+ "realtime_tok_per_s": 8727+ },+ "tap": []+ },+ {+ "id": "craps",+ "title": "Craps",+ "authors": [+ "Camerlo Cortez"+ ],+ "year": "1978",+ "description": "Craps (1978) by: Camerlo Cortez To use the Craps program, press any key to roll dice. 7 or 11 wins, 12, 2 or 3 loses on first roll. The second roll must match the first to win, but if you roll a seven you lose. This program could be expanded to include on-the-screen scoring of\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Craps%20%5BCamerlo%20Cortez%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/craps.ch8",+ "sha1": "35158696bd94ea22ef34e899fff1f15f7154d4fd",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.62,+ "realtime_tok_per_s": 3258+ },+ "tap": []+ },+ {+ "id": "deflection",+ "title": "Deflection",+ "authors": [+ "John Fort"+ ],+ "year": null,+ "description": "8. VIP DEFLECTION In the VIP Deflection game you position mirrors anywhere on the display screen. The object of the game is to deflect a ball of the mirrors a maximum number of times before hitting the target. The number of deflections times the target number gives you\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Deflection%20%5BJohn%20Fort%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/deflection.ch8",+ "sha1": "8e5f19d8ae9f3346779613359610967a5ed95fa8",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.2,+ "realtime_tok_per_s": 3780+ },+ "tap": []+ },+ {+ "id": "figures",+ "title": "Figures",+ "authors": [],+ "year": null,+ "description": "Figures - by ??? (unknown author)(xxxx) --------------------------------------- Kind of Tetris with numbers. Controls: --------- 4 and 6 to move 2 to do something (?)",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Figures.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/figures.ch8",+ "sha1": "3b2bf5dc7ffb5f3fbe168e802079f79730535ca8",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.19,+ "realtime_tok_per_s": 4669+ },+ "tap": []+ },+ {+ "id": "filter",+ "title": "Filter",+ "authors": [],+ "year": null,+ "description": "Filter Catch the drop coming from the pipe at the top of the screen with your paddle.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Filter.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/filter.ch8",+ "sha1": "ae71a7b081a947f1760cdc147759803aea45e751",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.02,+ "realtime_tok_per_s": 3619+ },+ "tap": []+ },+ {+ "id": "guess-the-number",+ "title": "Guess the Number",+ "authors": [+ "David Winter"+ ],+ "year": "1996",+ "description": "Guess, by David Winter Think to a number between 1 and 63. CHIP8 shows you several boards and you have to tell if you see your number in them. Press 5 if so, or another key if not. CHIP8 gives you the number...",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Guess%20%5BDavid%20Winter%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "a": 5,+ "b": 6+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/guess-the-number.ch8",+ "sha1": "137cb8397456f53fcab216124458238bc18c0965",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.72,+ "realtime_tok_per_s": 4248+ },+ "tap": []+ },+ {+ "id": "hi-lo",+ "title": "Hi-Lo",+ "authors": [+ "Jef Winsor"+ ],+ "year": "1978",+ "description": "Hi-Lo, by Jef Winsor You have 10 chances to guess the value of a random number between 00 and 99 selected by the program. The number at the right of the screen shows the number of the guess you are using. Enter a two digit number and the computer tells you if you are high or\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Hi-Lo%20%5BJef%20Winsor%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/hi-lo.ch8",+ "sha1": "dbb52193db4063149c3d8768ab47dd740d90955c",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.56,+ "realtime_tok_per_s": 3207+ },+ "tap": []+ },+ {+ "id": "hidden",+ "title": "Hidden",+ "authors": [+ "David Winter"+ ],+ "year": "1996",+ "description": "HIDDEN is a \"memory\" game. It is very simple to play. The rules are as follow: your goal is to find all the identical cards in a minimum time. You are playing in a 4*4 card grid. You can see only two cards at the same time. Once this time passed, these two cards will remain\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Hidden%20%5BDavid%20Winter%2C%201996%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 2,+ "left": 4,+ "right": 6,+ "down": 8,+ "a": 5+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/hidden.ch8",+ "sha1": "050f07a54371da79f924dd0227b89d07b4f2aed0",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.94,+ "realtime_tok_per_s": 7143+ },+ "tap": []+ },+ {+ "id": "kaleidoscope",+ "title": "Kaleidoscope",+ "authors": [+ "Joseph Weisbecker"+ ],+ "year": "1978",+ "description": "Kaleidoscope (1978), by Joseph Weisbecker Four spots appear in a group at the center of the screen. Press keys 2, 4, 6, or 8 to create a pattern. Keep your pattern smaller than 138 key depressions. Push key 0 to terminate pattern entry. Pushing key 0 causes your pattern to be\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Kaleidoscope%20%5BJoseph%20Weisbecker%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 2,+ "left": 4,+ "right": 6,+ "down": 8,+ "a": 5,+ "b": 0+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/kaleidoscope.ch8",+ "sha1": "fc724ae0125f5f1ac94a79fe3afc6318b1f57556",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.82,+ "realtime_tok_per_s": 5236+ },+ "tap": []+ },+ {+ "id": "landing",+ "title": "Landing",+ "authors": [],+ "year": null,+ "description": "Landing - by ??? (unknown author)(xxxx) --------------------------------------- Try to flatten the field for landing (?). Controls: --------- Key 9 drops a bomb.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Landing.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/landing.ch8",+ "sha1": "72fb3e0a4572bdb81f484df7948a8bc736fe78d0",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.92,+ "realtime_tok_per_s": 4431+ },+ "tap": []+ },+ {+ "id": "lunar-lander",+ "title": "Lunar Lander",+ "authors": [],+ "year": null,+ "description": "15. Lunar Lander You are descending to the floor of a 250-foot crater. The right side of the TV display shows amount of fuel, feet from bottom and fall rate per cycle. Lenght of fall rate cycle changes from about 1/4 second to one second. The fall rate increases by one foot per\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Lunar%20Lander%20%28Udo%20Pernisz%2C%201979%29.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/lunar-lander.ch8",+ "sha1": "72e8f3a10a32bd7fb91322ecab87249f95e81e57",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.77,+ "realtime_tok_per_s": 6091+ },+ "tap": []+ },+ {+ "id": "mastermind-fourrow",+ "title": "Mastermind FourRow",+ "authors": [+ "Robert Lindley"+ ],+ "year": "1978",+ "description": "MASTERMIND by Robert Lindley I have progranmed two versLons of the game Mastermind. This game is distributed by Invicta Plastics, Suite 940, 200 - 5th Ave., New York, NY 10010, and is available most pLaces where toys and games are sold. For complete details of the game, please\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Mastermind%20FourRow%20%28Robert%20Lindley%2C%201978%29.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/mastermind-fourrow.ch8",+ "sha1": "669e32b6f42f52da658e428f501aabcdfa37fb2e",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.69,+ "realtime_tok_per_s": 4223+ },+ "tap": []+ },+ {+ "id": "merlin",+ "title": "Merlin",+ "authors": [+ "David Winter"+ ],+ "year": null,+ "description": "Merlin, by David Winter This is the SIMON game. The goal is to remember in which order the squares are lighted. The game begins by lighting 4 random squares, and then asks you to light the squares in the correct order. You win a level when you give the exact order, and each\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Merlin%20%5BDavid%20Winter%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/merlin.ch8",+ "sha1": "d979858bb9ffd07b48f52f92a8bcac0199f3623e",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.97,+ "realtime_tok_per_s": 3577+ },+ "tap": []+ },+ {+ "id": "missile-command",+ "title": "Missile Command",+ "authors": [+ "David Winter"+ ],+ "year": "1996",+ "description": "Missile Command - by David Winter (19xx) ---------------------------------------- You must shoot the 8 targets on the screen using key 8. Your shooter moves a little bit faster each time you shoot. You have 12 missiles to shoot all the targets, and you win 5 points per target\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Missile%20%5BDavid%20Winter%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "a": 8+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/missile-command.ch8",+ "sha1": "0d0cc129dad3c45ba672f85fec71a668232212cc",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.78,+ "realtime_tok_per_s": 3399+ },+ "controls": "Space fires",+ "featured": 7,+ "tap": []+ },+ {+ "id": "most-dangerous-game",+ "title": "Most Dangerous Game",+ "authors": [+ "Peter Maruhnic"+ ],+ "year": null,+ "description": "10. VIP Most Dangerous Game VIP Most Dangerous Game pits a hunter against a hunted in a maze. The hunter must shoot the hunted before either time runs out or the hunted escapes the maze. However, neither the hunted nor the hunter can see a wall in the maze until he runs into\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Most%20Dangerous%20Game%20%5BPeter%20Maruhnic%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/most-dangerous-game.ch8",+ "sha1": "fa7c04f68d78e0faf6d136a3babe3943fc2e02f1",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.79,+ "realtime_tok_per_s": 3409+ },+ "tap": []+ },+ {+ "id": "nim",+ "title": "Nim",+ "authors": [+ "Carmelo Cortez"+ ],+ "year": null,+ "description": "Nim (1978), by Carmelo Cortez The Nim Game is a little less graphic than most games. The player may go first by pressing. \"F\" key, any other let the computer go first. You subtract 1, 2 or 3 fron the score. The one who ends up with the last number loses!",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Nim%20%5BCarmelo%20Cortez%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/nim.ch8",+ "sha1": "4031dae5c7545a1adc160a661be36f19fc1d47b2",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.13,+ "realtime_tok_per_s": 3717+ },+ "tap": []+ },+ {+ "id": "paddles",+ "title": "Paddles",+ "authors": [],+ "year": null,+ "description": "Paddles - Simple Pong game --------------------------------------- Controls: --------- F = 1P game E = 2P game 7/9 = P1 left/right 4/6 = P2 left/right",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Paddles.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/paddles.ch8",+ "sha1": "a18f1e3897416180b32e47ddc82cba9aca2c8d52",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.51,+ "realtime_tok_per_s": 4962+ },+ "tap": []+ },+ {+ "id": "pong-1-player",+ "title": "Pong",+ "authors": [+ "1 player"+ ],+ "year": null,+ "description": "Single player pong game",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Pong%20%281%20player%29.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/pong-1-player.ch8",+ "sha1": "607c4f7f4e4dce9f99d96b3182bfe7e88bb090ee",+ "tap": [],+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.96,+ "realtime_tok_per_s": 5366+ }+ },+ {+ "id": "pong-2",+ "title": "Pong 2",+ "authors": [+ "David Winter"+ ],+ "year": "1997",+ "description": "Pong 2 (Pong hack) - by David Winter (1997) ------------------------------------------- Here is the well known pong game. Two versions are available. The only difference is that PONG2 is mostly like the original game. Player 1 uses 1 and 4, player 2 uses C and D.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Pong%202%20%28Pong%20hack%29%20%5BDavid%20Winter%2C%201997%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 1,+ "down": 4,+ "player2Up": 12,+ "player2Down": 13+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/pong-2.ch8",+ "sha1": "1830eb401ba8789a477dfcf294873a5479ebcfe8",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.1,+ "realtime_tok_per_s": 5488+ },+ "tap": []+ },+ {+ "id": "pong",+ "title": "Pong",+ "authors": [+ "Paul Vervalin"+ ],+ "year": "1990",+ "description": "OK. here is PONG version 1.1. The ball is a little faster in this version making play a little more realistic. I know PONG 1.0 was just posted yesterday, but I think this version is significantly better, so here it is. Use keys 7 and 4 move left player and / and * move right\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Pong%20%5BPaul%20Vervalin%2C%201990%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 1,+ "down": 4,+ "player2Up": 12,+ "player2Down": 13+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/pong.ch8",+ "sha1": "b232ef880bd6060fb45fa6effed7edf0ae95670e",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.93,+ "realtime_tok_per_s": 5335+ },+ "controls": "Left paddle A/D (or W/S), right paddle Left/Right (or Up/Down)",+ "featured": 4,+ "tap": []+ },+ {+ "id": "programmable-spacefighters",+ "title": "Programmable Spacefighters",+ "authors": [+ "Jef Winsor"+ ],+ "year": null,+ "description": "14. VIP Programmable Spacefighters Programmable Spacefighters is a combat game involving 2 to 8 spaceships competing for the domination of a contained field in space. The field of play is a two-dimensional representation of an oblong spheroid. The movement and fire of each\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Programmable%20Spacefighters%20%5BJef%20Winsor%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/programmable-spacefighters.ch8",+ "sha1": "726cb39afa7e17725af7fab37d153277d86bff77",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.69,+ "realtime_tok_per_s": 4219+ },+ "tap": []+ },+ {+ "id": "puzzle",+ "title": "Puzzle",+ "authors": [],+ "year": "199x",+ "description": "Puzzle - 4x4 tile puzzle - rearrange them into the order 0-F -------------------------------------- Wait for randomization... Instead of moving the item by pressing his associated key, move it UP DOWN LEFT RIGHT with respectively 2 8 4 6. Up and Down are inverted as the game\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Puzzle.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/puzzle.ch8",+ "sha1": "1293db0ccccbe7dd3fc5a09a2abc5d7b175e18e0",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.26,+ "realtime_tok_per_s": 5634+ },+ "tap": []+ },+ {+ "id": "reversi",+ "title": "Reversi",+ "authors": [],+ "year": null,+ "description": "6. VIP REVERSI Reversi is a game over 100 years old, which has become popular recently under the name Othello. The game is played on a 8x8 square, using two kinds of markers. In VIP Reversi one player has the open markers and the other player the solid markers. The score for\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Reversi%20%5BPhilip%20Baltzer%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/reversi.ch8",+ "sha1": "ff639eceaf221ae66151a03779b41fae7118d2d8",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.6,+ "realtime_tok_per_s": 4138+ },+ "tap": []+ },+ {+ "id": "rocket",+ "title": "Rocket",+ "authors": [+ "Joseph Weisbecker"+ ],+ "year": "1978-12",+ "description": "Rocket - by Joseph Weisbecker (1978) ------------------------------------ from Byte Magazine Dec.1978 An enemy UFO will be constantly moving from left to right across the top of the screen. A single digit score will be displayed at the lower right. A rocket ship will appear at\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Rocket%20%5BJoseph%20Weisbecker%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/rocket.ch8",+ "sha1": "3d1d029d6e31206d245c0ba881c0d1f003953bad",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.06,+ "realtime_tok_per_s": 6356+ },+ "tap": []+ },+ {+ "id": "rocket-launch",+ "title": "Rocket Launch",+ "authors": [+ "Jonas Lindstedt"+ ],+ "year": "19xx",+ "description": "Another Rocket launcher game by Jonas Lindstedt. Press \"B\" to start. Press \"4\" to go LEFT and \"6\" to go RIGHT.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Rocket%20Launch%20%5BJonas%20Lindstedt%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "superchip",+ "quirks": [+ "shift",+ "loadstore",+ "clip",+ "jump"+ ],+ "ipf": 30,+ "keys": {},+ "notes": [],+ "file": "roms/rocket-launch.ch8",+ "sha1": "5e70f91ca08e9b9e9de61670492e3db2d7f7d57a",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.54,+ "realtime_tok_per_s": 9979+ },+ "tap": []+ },+ {+ "id": "rocket-launcher",+ "title": "Rocket Launcher",+ "authors": [],+ "year": null,+ "description": "",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Rocket%20Launcher.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/rocket-launcher.ch8",+ "sha1": "e2005db6391f589534dd2d63a95b429338bd667c",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.57,+ "realtime_tok_per_s": 5016+ },+ "tap": []+ },+ {+ "id": "rush-hour",+ "title": "Rush Hour",+ "authors": [+ "Hap"+ ],+ "year": "2006-12-17",+ "description": "Rush Hour (v1.1) for CHIP-8 by hap 08-02-08, http://hap.samor.nl/ Originally released on 17-12-06. Version 1.1 improves a few things. Based on a boardgame by Nobuyuki Yoshigahara \"Nob\" and ThinkFun, http://www.thinkfun.com/ This game contains 170 puzzles. Most were taken from\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Rush%20Hour%20%5BHap%2C%202006%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/rush-hour.ch8",+ "sha1": "4639f86beb0a203ae512b85d3b56d813b2dea7b4",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.2,+ "realtime_tok_per_s": 3782+ },+ "tap": []+ },+ {+ "id": "russian-roulette",+ "title": "Russian Roulette",+ "authors": [+ "Carmelo Cortez"+ ],+ "year": null,+ "description": "Russian Roulette (1978), by Carmelo Cortez This game is called Russian RouLette. Press any key to Spin and pull the Trigger. A \"Click\" or \"Bang\" will show, get ten \"clicks\" in a row and you win.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Russian%20Roulette%20%5BCarmelo%20Cortez%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/russian-roulette.ch8",+ "sha1": "24960090b2afc9de2a4cb3ee7daf6a21456bb49b",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.59,+ "realtime_tok_per_s": 3227+ },+ "tap": []+ },+ {+ "id": "sequence-shoot",+ "title": "Sequence Shoot",+ "authors": [+ "Joyce Weisbecker"+ ],+ "year": null,+ "description": "12. VIP Sequence Shoot You score points by having the sharp-shooter hit the targets in the proper sequence. How to Play VIP Sequence Shoot 1. Load the CHIP-8 interpreter at 0000-01FF and the game starting at 0200. 2. Turn RUN switch on. 3. Pressing the Key C causes the little\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Sequence%20Shoot%20%5BJoyce%20Weisbecker%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/sequence-shoot.ch8",+ "sha1": "448f9d30d2157ab42679b809d4fb0b43d145f74f",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.66,+ "realtime_tok_per_s": 3294+ },+ "tap": []+ },+ {+ "id": "shooting-stars",+ "title": "Shooting Stars",+ "authors": [+ "Philip Baltzer"+ ],+ "year": "1978",+ "description": "Shooting Stars (1978), by Philip Baltzer",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Shooting%20Stars%20%5BPhilip%20Baltzer%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/shooting-stars.ch8",+ "sha1": "443550abf646bc7f475ef0466f8e1232ec7474f3",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.06,+ "realtime_tok_per_s": 4552+ },+ "tap": []+ },+ {+ "id": "slide",+ "title": "Slide",+ "authors": [+ "Joyce Weisbecker"+ ],+ "year": null,+ "description": "5. VIP SLIDE Slide is a two-person game. Each player tries to slide a \"puck\" over the high-scoring \"spots\" without hitting the back wall. How To Play VIP Slide 1. Load the CHIP-8 interpretor at 0000-01FF and the game starting at 0200. 2. Turn RUN switch on. 3. The puck moves up\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Slide%20%5BJoyce%20Weisbecker%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/slide.ch8",+ "sha1": "7623fa0fa915979226566b24107360e7537735f4",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.33,+ "realtime_tok_per_s": 3896+ },+ "tap": []+ },+ {+ "id": "soccer",+ "title": "Soccer",+ "authors": [],+ "year": "19xx",+ "description": "Soccer - by ??? (unknown author)(19xx) -------------------------------------- Pong-like. 1/4 = P1 up/down C/D = P2 up/down",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Soccer.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/soccer.ch8",+ "sha1": "6df358d77961a0bf21e98876f9f616791cba31e3",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.41,+ "realtime_tok_per_s": 4872+ },+ "tap": []+ },+ {+ "id": "space-flight",+ "title": "Space Flight",+ "authors": [],+ "year": "19xx",+ "description": "Space flight game </br>Fly through the asteroid field. Use 1 and 4 key to navigate space ship and E/F to start the game.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Space%20Flight.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/space-flight.ch8",+ "sha1": "aa4f1a282bd64a2364102abf5737a4205365a2b4",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.96,+ "realtime_tok_per_s": 3563+ },+ "tap": []+ },+ {+ "id": "space-intercept",+ "title": "Space Intercept",+ "authors": [],+ "year": null,+ "description": "Space Intercept (1978), by Joseph Weisbecker At startup, Press 1 to select the large UFO which counts 5 points when hit or 2 to select the small UFO which counts 15 points when hit. Launch your rocket by pressing key 4,5 or 6. You get 15 rockets as shown in the lower right\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Space%20Intercept%20%5BJoseph%20Weisbecker%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/space-intercept.ch8",+ "sha1": "ed829190e37815771e7a8c675ba0074996a2ddb0",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.5,+ "realtime_tok_per_s": 4952+ },+ "tap": []+ },+ {+ "id": "space-invaders",+ "title": "Space Invaders",+ "authors": [+ "David Winter"+ ],+ "year": "1996",+ "description": "Space Invaders (1978), by David Winter The well known game. Destroy the invaders with your ship. Shoot with 5, move with 4 and 6. Press 5 to begin a game.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Space%20Invaders%20%5BDavid%20Winter%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "superchip",+ "quirks": [+ "shift",+ "loadstore",+ "clip",+ "jump"+ ],+ "ipf": 30,+ "keys": {+ "left": 4,+ "right": 6,+ "a": 5+ },+ "notes": [],+ "file": "roms/space-invaders.ch8",+ "sha1": "5c28a5f85289c9d859f95fd5eadbdcb1c30bb08b",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.81,+ "realtime_tok_per_s": 8659+ },+ "controls": "Left/Right move, Space fires (press Space to start)",+ "featured": 1,+ "tap": []+ },+ {+ "id": "spooky-spot",+ "title": "Spooky Spot",+ "authors": [+ "Joseph Weisbecker"+ ],+ "year": null,+ "description": "Spooky Spot (1978), by Joseph Weisbecker Now you can let the computer make your big decisions or predict the future just like governmentt or industry leaders do. You will see the words YES and NO at the right of the screen. Ask the computer any question that can be answered\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Spooky%20Spot%20%5BJoseph%20Weisbecker%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/spooky-spot.ch8",+ "sha1": "1bd92042717c3bc4f7f34cab34be2887145a6704",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.8,+ "realtime_tok_per_s": 3421+ },+ "tap": []+ },+ {+ "id": "squash",+ "title": "Squash",+ "authors": [+ "David Winter"+ ],+ "year": "1997",+ "description": "Squash, by David Winter Bounce a ball around a squash court with your paddle",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Squash%20%5BDavid%20Winter%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 1,+ "down": 4+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/squash.ch8",+ "sha1": "a58ec7cc63707f9e7274026de27c15ec1d9945bd",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.77,+ "realtime_tok_per_s": 4292+ },+ "tap": []+ },+ {+ "id": "submarine",+ "title": "Submarine",+ "authors": [+ "Carmelo Cortez"+ ],+ "year": null,+ "description": "Submarine (1978), by Carmelo Cortez The Sub Game is my favorite. Press \"5\" key to fire depth charges at the subs below. You score 15 points for a small sub and 5 points for the larger. You get 25 depth charges to start.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Submarine%20%5BCarmelo%20Cortez%2C%201978%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/submarine.ch8",+ "sha1": "89aadf7c28bcd1c11e71ad9bd6eeaf0e7be474f3",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.27,+ "realtime_tok_per_s": 6547+ },+ "tap": []+ },+ {+ "id": "sum-fun",+ "title": "Sum Fun",+ "authors": [+ "Joyce Weisbecker"+ ],+ "year": null,+ "description": "11. VIP Sum Fun The object of this game is to add up the three digits which appear in the middle of the screen and then hit the key representing the total as fast as you can. How to Play VIP Sum FUn 1. Load the CHIP-8 interpreter at 0000-01FF and the game starting at 0200. 2.\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Sum%20Fun%20%5BJoyce%20Weisbecker%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/sum-fun.ch8",+ "sha1": "83a2f9c8153be955c28e788bd803aa1d25131330",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.14,+ "realtime_tok_per_s": 3729+ },+ "tap": []+ },+ {+ "id": "syzygy",+ "title": "Syzygy",+ "authors": [+ "Roy Trevino"+ ],+ "year": "1990",+ "description": "SYZYGY v0.1 One of the first games I remember playing on a computer was called \"syzygy\" on a now ancient TRS-80 Model 1. It has since appeared on other computers under various names. Why it was called syzygy, I have no idea (consult Websters). However, since the HP48SX has\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Syzygy%20%5BRoy%20Trevino%2C%201990%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "chip48",+ "quirks": [+ "shift",+ "loadstore",+ "clip",+ "jump"+ ],+ "ipf": 30,+ "keys": {},+ "notes": [+ "platform increments I by X after FX55/FX65; run with I unchanged"+ ],+ "file": "roms/syzygy.ch8",+ "sha1": "1bdb4ddaa7049266fa3226851f28855a365cfd12",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.42,+ "realtime_tok_per_s": 7954+ },+ "tap": []+ },+ {+ "id": "tank-battle",+ "title": "Tank Battle",+ "authors": [],+ "year": "197x",+ "description": "You are in a tank which has 25 bombs. Your goal is to hit 25 times a mobile target. The game ends when all your bombs are shot. If your tank hits the target, you lose 5 bombs. Use 2 4 6 and 8 to move. This game uses the original CHIP8 keyboard, so directions 2 and 8 are swapped.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Tank.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/tank-battle.ch8",+ "sha1": "18b9d15f4c159e1f0ed58c2d8ec1d89325d3a3b6",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.42,+ "realtime_tok_per_s": 5780+ },+ "tap": []+ },+ {+ "id": "tapeworm",+ "title": "Tapeworm",+ "authors": [+ "JDR"+ ],+ "year": "1999",+ "description": "Tapeworm (1999) Tron clone; guide the snake around and around.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Tapeworm%20%5BJDR%2C%201999%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/tapeworm.ch8",+ "sha1": "775e82a36c93f1b41b42eca94b55acbc4a48cebe",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.9,+ "realtime_tok_per_s": 3513+ },+ "tap": []+ },+ {+ "id": "tetris",+ "title": "Tetris",+ "authors": [+ "Fran Dachille"+ ],+ "year": "1991",+ "description": "TETRIS by Fran Dachille This is my first release of the famous Tetris game on the HP48S. I was inspired by the lack enjoyable games for our favorite handheld. [Not since the Goodies Disks have been available! -jkh-] This game, though it lacks some of the whistles and bangs of\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Tetris%20%5BFran%20Dachille%2C%201991%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "chip48",+ "quirks": [+ "shift",+ "loadstore",+ "clip",+ "jump"+ ],+ "ipf": 15,+ "keys": {+ "left": 5,+ "right": 6,+ "down": 7,+ "a": 4+ },+ "notes": [+ "platform increments I by X after FX55/FX65; run with I unchanged"+ ],+ "file": "roms/tetris.ch8",+ "sha1": "5f518084744bf3cb8733f6e5454dfd1634320563",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.27,+ "realtime_tok_per_s": 7684+ },+ "controls": "Left/Right move, Down drops faster, Space rotates",+ "featured": 0,+ "tap": [+ "a"+ ],+ "game_over": {+ "rows": [+ 6,+ 7+ ],+ "cols": [+ 27,+ 35+ ],+ "seconds": 5,+ "message": "The well is full"+ }+ },+ {+ "id": "tic-tac-toe",+ "title": "TIC-TAC-TOE",+ "authors": [],+ "year": "1996",+ "description": "Tic-Tac-Toe, by David Winter 2-player noughts and crosses board.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Tic-Tac-Toe%20%5BDavid%20Winter%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "superchip",+ "quirks": [+ "shift",+ "loadstore",+ "clip",+ "jump"+ ],+ "ipf": 30,+ "keys": {},+ "notes": [],+ "file": "roms/tic-tac-toe.ch8",+ "sha1": "429d455a4bc53167942bf6fd934d72b0f648dce3",+ "status": "ok",+ "result": {+ "verified_records": 20772,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.07,+ "realtime_tok_per_s": 7326+ },+ "tap": []+ },+ {+ "id": "timebomb",+ "title": "Timebomb",+ "authors": [],+ "year": "19xx",+ "description": "Timebomb - by ??? (unknown author)(19xx) ---------------------------------------- Chronometer. 2/8: Set time. 5: Start.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Timebomb.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/timebomb.ch8",+ "sha1": "67996195539c0ddcd98533a01dffeec6a53a6da1",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 7.47,+ "realtime_tok_per_s": 6724+ },+ "tap": []+ },+ {+ "id": "tron",+ "title": "Tron",+ "authors": [],+ "year": "19xx",+ "description": "Tron - by ??? (unknown)(19xx) ----------------------------- Controls: --------- B=with borders. F=without borders. 0=start. Player 1: 1/4 3/C. Player 2: 7/A 9/E.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Tron.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/tron.ch8",+ "sha1": "a6a6cb2351c20b8f904da07c0ce91bd8161e9317",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.66,+ "realtime_tok_per_s": 4193+ },+ "tap": []+ },+ {+ "id": "ufo",+ "title": "UFO",+ "authors": [+ "Lutz V"+ ],+ "year": "1992",+ "description": "Here's how to play UFO: You have a stationary missle launcher at the bottom of the screen. You can shoot in three directions; left diagonal, straight up, and right diagonal.. using the keys 4, 5, and 6 respectively.. You try to hit one of two objects flying by.. at apparently\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/UFO%20%5BLutz%20V%2C%201992%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "left": 4,+ "a": 5,+ "right": 6+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/ufo.ch8",+ "sha1": "bdb92475acfe11bc7814a2f5eade13fcd09b756a",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.24,+ "realtime_tok_per_s": 5612+ },+ "controls": "Left/Space/Right fire left, up, right",+ "featured": 6,+ "tap": []+ },+ {+ "id": "vers",+ "title": "Vers",+ "authors": [+ "JMN"+ ],+ "year": "1991",+ "description": "Vers - by JMN (1991) -------------------- Another Tron clone. Keys Left Pl. Right Pl. ------------------------------ UP 7 C DOWN A D LEFT 1 B RIGHT 2 F ------------------------------",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Vers%20%5BJMN%2C%201991%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/vers.ch8",+ "sha1": "ade839585ddeb0e3633177df03c1d91589e629eb",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.43,+ "realtime_tok_per_s": 3987+ },+ "tap": []+ },+ {+ "id": "vertical-brix",+ "title": "Vertical Brix",+ "authors": [+ "Paul Robson"+ ],+ "year": "1996",+ "description": "Vertical Brix - by Paul Robson (1996) ------------------------------------- Like BRIX, but the brix are put vertically, and the pad also moves vertically. Start by pressing 7, and move using 4 and 1. *NOTE: 7->5 1->2 4->6",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Vertical%20Brix%20%5BPaul%20Robson%2C%201996%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/vertical-brix.ch8",+ "sha1": "da710f631f8e35534d0b9170bcf892a60f49c43d",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.15,+ "realtime_tok_per_s": 4633+ },+ "tap": []+ },+ {+ "id": "wall",+ "title": "Wall",+ "authors": [+ "David Winter"+ ],+ "year": "1997",+ "description": "Wall - by David Winter (19xx) ----------------------------- One of these PONG variations. Move using 1 and 4. As said in the early seventies: \"AVOID MISSING BALL FOR HIGH SCORE\" !!! *NOTE: 1->2 4->6",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Wall%20%5BDavid%20Winter%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {+ "up": 1,+ "down": 4+ },+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/wall.ch8",+ "sha1": "09ce01c54ddddda42ca5cd171f1ffcfd47355d12",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.27,+ "realtime_tok_per_s": 4744+ },+ "tap": []+ },+ {+ "id": "wipe-off",+ "title": "Wipe Off",+ "authors": [+ "Joseph Weisbecker"+ ],+ "year": "19xx",+ "description": "Wipe Off - by Joseph Weisbecker (19xx) -------------------------------------- Cosmac VIP CDP18S711 Book - Page 42 (VIP-311) This program uses the CHIP-8 INTERPRETER at 0000-01FF Serve the ball by pressing any key. Move the paddle left or right by pressing key 4 or 6. Try to\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Wipe%20Off%20%5BJoseph%20Weisbecker%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/wipe-off.ch8",+ "sha1": "d666688a8fce468a7d88b536bc1ef5f35ba12031",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 5.6,+ "realtime_tok_per_s": 5036+ },+ "tap": []+ },+ {+ "id": "superworm-v4",+ "title": "SuperWorm V4",+ "authors": [+ "RB-Revival Studios",+ "Martijn Wenting"+ ],+ "year": "2007",+ "description": "----------------------------------------------------------------------------- ///////////////// ////////////////// //// //// //// /////////// //// /////////// //// //// //// /////////// //// ////////// www.revival-studios.com\u2026",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/Worm%20V4%20%5BRB-Revival%20Studios%2C%202007%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/superworm-v4.ch8",+ "sha1": "a1c1e0e7b01004be3ee77c69030e6b536cb316e6",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 3.72,+ "realtime_tok_per_s": 3351+ },+ "tap": []+ },+ {+ "id": "x-mirror",+ "title": "X-Mirror",+ "authors": [],+ "year": null,+ "description": "X-Mirror - by ??? (unknown author)(19xx) ---------------------------------------- A Kaleidoscope variant. Draws symmetric patterns using 2/4/6/8.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/X-Mirror.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/x-mirror.ch8",+ "sha1": "bc158d819890f16f105b8a316eeeefe4a0bad875",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 6.03,+ "realtime_tok_per_s": 5422+ },+ "tap": []+ },+ {+ "id": "zeropong",+ "title": "ZeroPong",+ "authors": [+ "zeroZshadow"+ ],+ "year": "2007",+ "description": "ZeroPong - by zeroZshadow (2007) -------------------------------- Player 1: Press \"B\" to move up and \"A\" to move down. Player 2: Press \"2\" to move up and \"8\" to move down. Press \"F\" to start moving the ball.",+ "source": "https://github.com/kripod/chip8-roms/blob/master/games/ZeroPong%20%5BzeroZshadow%2C%202007%5D.ch8",+ "license": "freely distributable (Revival Studios CHIP-8 pack)",+ "platform": "originalChip8",+ "quirks": [+ "clip",+ "logic"+ ],+ "ipf": 15,+ "keys": {},+ "notes": [+ "platform waits for vblank on draw; not modelled"+ ],+ "file": "roms/zeropong.ch8",+ "sha1": "f2e9c480af31a4039af02dd7a2b8d5d1f859704d",+ "status": "ok",+ "result": {+ "verified_records": 21543,+ "error": null,+ "foreign_opcodes": {},+ "tokens_per_instruction": 4.46,+ "realtime_tok_per_s": 4011+ },+ "tap": []+ }+ ]+}\ No newline at end of filediff --git a/games/roms/15-puzzle.ch8 b/games/roms/15-puzzle.ch8new file mode 100644index 0000000..3ef0bc8Binary files /dev/null and b/games/roms/15-puzzle.ch8 differdiff --git a/games/roms/1d-cellular-automata.ch8 b/games/roms/1d-cellular-automata.ch8new file mode 100644index 0000000..2fd5ba6Binary files /dev/null and b/games/roms/1d-cellular-automata.ch8 differdiff --git a/games/roms/8ce-attourny--disc-1.ch8 b/games/roms/8ce-attourny--disc-1.ch8new file mode 100644index 0000000..bbf9353Binary files /dev/null and b/games/roms/8ce-attourny--disc-1.ch8 differdiff --git a/games/roms/8ce-attourny--disc-2.ch8 b/games/roms/8ce-attourny--disc-2.ch8new file mode 100644index 0000000..5416c0eBinary files /dev/null and b/games/roms/8ce-attourny--disc-2.ch8 differdiff --git a/games/roms/8ce-attourny--disc-3.ch8 b/games/roms/8ce-attourny--disc-3.ch8new file mode 100644index 0000000..af79437Binary files /dev/null and b/games/roms/8ce-attourny--disc-3.ch8 differdiff --git a/games/roms/addition-problems.ch8 b/games/roms/addition-problems.ch8new file mode 100644index 0000000..dedf7cbBinary files /dev/null and b/games/roms/addition-problems.ch8 differdiff --git a/games/roms/airplane.ch8 b/games/roms/airplane.ch8new file mode 100644index 0000000..cac8804Binary files /dev/null and b/games/roms/airplane.ch8 differdiff --git a/games/roms/animal-race.ch8 b/games/roms/animal-race.ch8new file mode 100644index 0000000..56718daBinary files /dev/null and b/games/roms/animal-race.ch8 differdiff --git a/games/roms/astro-dodge.ch8 b/games/roms/astro-dodge.ch8new file mode 100644index 0000000..e41bf2cBinary files /dev/null and b/games/roms/astro-dodge.ch8 differdiff --git a/games/roms/bad-kaiju-ju.ch8 b/games/roms/bad-kaiju-ju.ch8new file mode 100644index 0000000..6a83f93Binary files /dev/null and b/games/roms/bad-kaiju-ju.ch8 differdiff --git a/games/roms/biorhythm.ch8 b/games/roms/biorhythm.ch8new file mode 100644index 0000000..1701460Binary files /dev/null and b/games/roms/biorhythm.ch8 differdiff --git a/games/roms/blinky.ch8 b/games/roms/blinky.ch8new file mode 100644index 0000000..235cf98Binary files /dev/null and b/games/roms/blinky.ch8 differdiff --git a/games/roms/blitz.ch8 b/games/roms/blitz.ch8new file mode 100644index 0000000..0d2effaBinary files /dev/null and b/games/roms/blitz.ch8 differdiff --git a/games/roms/bowling.ch8 b/games/roms/bowling.ch8new file mode 100644index 0000000..609d976Binary files /dev/null and b/games/roms/bowling.ch8 differdiff --git a/games/roms/br8kout.ch8 b/games/roms/br8kout.ch8new file mode 100644index 0000000..7e675f0Binary files /dev/null and b/games/roms/br8kout.ch8 differdiff --git a/games/roms/breakout-2.ch8 b/games/roms/breakout-2.ch8new file mode 100644index 0000000..70b50dbBinary files /dev/null and b/games/roms/breakout-2.ch8 differdiff --git a/games/roms/breakout.ch8 b/games/roms/breakout.ch8new file mode 100644index 0000000..622fc4eBinary files /dev/null and b/games/roms/breakout.ch8 differdiff --git a/games/roms/brick.ch8 b/games/roms/brick.ch8new file mode 100644index 0000000..72a366eBinary files /dev/null and b/games/roms/brick.ch8 differdiff --git a/games/roms/brix.ch8 b/games/roms/brix.ch8new file mode 100644index 0000000..ad639d9Binary files /dev/null and b/games/roms/brix.ch8 differdiff --git a/games/roms/carbon8.ch8 b/games/roms/carbon8.ch8new file mode 100644index 0000000..93c59dbBinary files /dev/null and b/games/roms/carbon8.ch8 differdiff --git a/games/roms/cave-explorer.ch8 b/games/roms/cave-explorer.ch8new file mode 100644index 0000000..7e217a2Binary files /dev/null and b/games/roms/cave-explorer.ch8 differdiff --git a/games/roms/cave.ch8 b/games/roms/cave.ch8new file mode 100644index 0000000..d5a865fBinary files /dev/null and b/games/roms/cave.ch8 differdiff --git a/games/roms/chipquarium.ch8 b/games/roms/chipquarium.ch8new file mode 100644index 0000000..54759b6Binary files /dev/null and b/games/roms/chipquarium.ch8 differdiff --git a/games/roms/chipwar.ch8 b/games/roms/chipwar.ch8new file mode 100644index 0000000..5357fa9Binary files /dev/null and b/games/roms/chipwar.ch8 differdiff --git a/games/roms/coin-flipping.ch8 b/games/roms/coin-flipping.ch8new file mode 100644index 0000000..518e1d5Binary files /dev/null and b/games/roms/coin-flipping.ch8 differdiff --git a/games/roms/connect-4.ch8 b/games/roms/connect-4.ch8new file mode 100644index 0000000..200a67aBinary files /dev/null and b/games/roms/connect-4.ch8 differdiff --git a/games/roms/craps.ch8 b/games/roms/craps.ch8new file mode 100644index 0000000..1717024Binary files /dev/null and b/games/roms/craps.ch8 differdiff --git a/games/roms/danm8ku.ch8 b/games/roms/danm8ku.ch8new file mode 100644index 0000000..0f45e5cBinary files /dev/null and b/games/roms/danm8ku.ch8 differdiff --git a/games/roms/deflection.ch8 b/games/roms/deflection.ch8new file mode 100644index 0000000..f218966Binary files /dev/null and b/games/roms/deflection.ch8 differdiff --git a/games/roms/dino-run.ch8 b/games/roms/dino-run.ch8new file mode 100644index 0000000..de7c9ffBinary files /dev/null and b/games/roms/dino-run.ch8 differdiff --git a/games/roms/down8.ch8 b/games/roms/down8.ch8new file mode 100644index 0000000..47fda38Binary files /dev/null and b/games/roms/down8.ch8 differdiff --git a/games/roms/figures.ch8 b/games/roms/figures.ch8new file mode 100644index 0000000..6563badBinary files /dev/null and b/games/roms/figures.ch8 differdiff --git a/games/roms/filter.ch8 b/games/roms/filter.ch8new file mode 100644index 0000000..15ab24cBinary files /dev/null and b/games/roms/filter.ch8 differdiff --git a/games/roms/flight-runner.ch8 b/games/roms/flight-runner.ch8new file mode 100644index 0000000..b379353Binary files /dev/null and b/games/roms/flight-runner.ch8 differdiff --git a/games/roms/fuse.ch8 b/games/roms/fuse.ch8new file mode 100644index 0000000..5ba104dBinary files /dev/null and b/games/roms/fuse.ch8 differdiff --git a/games/roms/ghost-escape.ch8 b/games/roms/ghost-escape.ch8new file mode 100644index 0000000..fef7835Binary files /dev/null and b/games/roms/ghost-escape.ch8 differdiff --git a/games/roms/glitch-ghost.ch8 b/games/roms/glitch-ghost.ch8new file mode 100644index 0000000..b5e14e1Binary files /dev/null and b/games/roms/glitch-ghost.ch8 differdiff --git a/games/roms/guess-the-number.ch8 b/games/roms/guess-the-number.ch8new file mode 100644index 0000000..8f151f1Binary files /dev/null and b/games/roms/guess-the-number.ch8 differdiff --git a/games/roms/hi-lo.ch8 b/games/roms/hi-lo.ch8new file mode 100644index 0000000..90a872aBinary files /dev/null and b/games/roms/hi-lo.ch8 differdiff --git a/games/roms/hidden.ch8 b/games/roms/hidden.ch8new file mode 100644index 0000000..bd6b18dBinary files /dev/null and b/games/roms/hidden.ch8 differdiff --git a/games/roms/horse-world-online.ch8 b/games/roms/horse-world-online.ch8new file mode 100644index 0000000..74606a3Binary files /dev/null and b/games/roms/horse-world-online.ch8 differdiff --git a/games/roms/kaleidoscope.ch8 b/games/roms/kaleidoscope.ch8new file mode 100644index 0000000..b1c8888Binary files /dev/null and b/games/roms/kaleidoscope.ch8 differdiff --git a/games/roms/knumber-knower.ch8 b/games/roms/knumber-knower.ch8new file mode 100644index 0000000..f80e65eBinary files /dev/null and b/games/roms/knumber-knower.ch8 differdiff --git a/games/roms/landing.ch8 b/games/roms/landing.ch8new file mode 100644index 0000000..622e81fBinary files /dev/null and b/games/roms/landing.ch8 differdiff --git a/games/roms/lunar-lander.ch8 b/games/roms/lunar-lander.ch8new file mode 100644index 0000000..e7972b5Binary files /dev/null and b/games/roms/lunar-lander.ch8 differdiff --git a/games/roms/masquer8.ch8 b/games/roms/masquer8.ch8new file mode 100644index 0000000..6001620Binary files /dev/null and b/games/roms/masquer8.ch8 differdiff --git a/games/roms/mastermind-fourrow.ch8 b/games/roms/mastermind-fourrow.ch8new file mode 100644index 0000000..f403d71Binary files /dev/null and b/games/roms/mastermind-fourrow.ch8 differdiff --git a/games/roms/mastermind.ch8 b/games/roms/mastermind.ch8new file mode 100644index 0000000..36759e6Binary files /dev/null and b/games/roms/mastermind.ch8 differdiff --git a/games/roms/mato8.ch8 b/games/roms/mato8.ch8new file mode 100644index 0000000..b016f9aBinary files /dev/null and b/games/roms/mato8.ch8 differdiff --git a/games/roms/merlin.ch8 b/games/roms/merlin.ch8new file mode 100644index 0000000..747843aBinary files /dev/null and b/games/roms/merlin.ch8 differdiff --git a/games/roms/mini-lights-out.ch8 b/games/roms/mini-lights-out.ch8new file mode 100644index 0000000..12da84dBinary files /dev/null and b/games/roms/mini-lights-out.ch8 differdiff --git a/games/roms/missile-command.ch8 b/games/roms/missile-command.ch8new file mode 100644index 0000000..310e2deBinary files /dev/null and b/games/roms/missile-command.ch8 differdiff --git a/games/roms/most-dangerous-game.ch8 b/games/roms/most-dangerous-game.ch8new file mode 100644index 0000000..8407c4fBinary files /dev/null and b/games/roms/most-dangerous-game.ch8 differdiff --git a/games/roms/nim.ch8 b/games/roms/nim.ch8new file mode 100644index 0000000..680c96aBinary files /dev/null and b/games/roms/nim.ch8 differdiff --git a/games/roms/octo-a-chip-8-story.ch8 b/games/roms/octo-a-chip-8-story.ch8new file mode 100644index 0000000..4be77cbBinary files /dev/null and b/games/roms/octo-a-chip-8-story.ch8 differdiff --git a/games/roms/octo-rancher.ch8 b/games/roms/octo-rancher.ch8new file mode 100644index 0000000..198ad93Binary files /dev/null and b/games/roms/octo-rancher.ch8 differdiff --git a/games/roms/octojam-1-title.ch8 b/games/roms/octojam-1-title.ch8new file mode 100644index 0000000..7e8dd9fBinary files /dev/null and b/games/roms/octojam-1-title.ch8 differdiff --git a/games/roms/octojam-10-title.ch8 b/games/roms/octojam-10-title.ch8new file mode 100644index 0000000..aea7bdaBinary files /dev/null and b/games/roms/octojam-10-title.ch8 differdiff --git a/games/roms/octojam-2-title.ch8 b/games/roms/octojam-2-title.ch8new file mode 100644index 0000000..6e19074Binary files /dev/null and b/games/roms/octojam-2-title.ch8 differdiff --git a/games/roms/octojam-3-title.ch8 b/games/roms/octojam-3-title.ch8new file mode 100644index 0000000..31f9dabBinary files /dev/null and b/games/roms/octojam-3-title.ch8 differdiff --git a/games/roms/octojam-4-title.ch8 b/games/roms/octojam-4-title.ch8new file mode 100644index 0000000..202721dBinary files /dev/null and b/games/roms/octojam-4-title.ch8 differdiff --git a/games/roms/octojam-5-title.ch8 b/games/roms/octojam-5-title.ch8new file mode 100644index 0000000..cb65447Binary files /dev/null and b/games/roms/octojam-5-title.ch8 differdiff --git a/games/roms/octojam-6-title.ch8 b/games/roms/octojam-6-title.ch8new file mode 100644index 0000000..a66c8abBinary files /dev/null and b/games/roms/octojam-6-title.ch8 differdiff --git a/games/roms/octojam-7-title.ch8 b/games/roms/octojam-7-title.ch8new file mode 100644index 0000000..5ef5fc3Binary files /dev/null and b/games/roms/octojam-7-title.ch8 differdiff --git a/games/roms/octojam-8-title.ch8 b/games/roms/octojam-8-title.ch8new file mode 100644index 0000000..5c1c989Binary files /dev/null and b/games/roms/octojam-8-title.ch8 differdiff --git a/games/roms/octojam-9-title.ch8 b/games/roms/octojam-9-title.ch8new file mode 100644index 0000000..8a4350bBinary files /dev/null and b/games/roms/octojam-9-title.ch8 differdiff --git a/games/roms/outlaw.ch8 b/games/roms/outlaw.ch8new file mode 100644index 0000000..71eff28Binary files /dev/null and b/games/roms/outlaw.ch8 differdiff --git a/games/roms/paddles.ch8 b/games/roms/paddles.ch8new file mode 100644index 0000000..26af3d2Binary files /dev/null and b/games/roms/paddles.ch8 differdiff --git a/games/roms/pet-dog.ch8 b/games/roms/pet-dog.ch8new file mode 100644index 0000000..8646ef1Binary files /dev/null and b/games/roms/pet-dog.ch8 differdiff --git a/games/roms/piper.ch8 b/games/roms/piper.ch8new file mode 100644index 0000000..38adaa8Binary files /dev/null and b/games/roms/piper.ch8 differdiff --git a/games/roms/pong-1-player.ch8 b/games/roms/pong-1-player.ch8new file mode 100644index 0000000..65d6310Binary files /dev/null and b/games/roms/pong-1-player.ch8 differdiff --git a/games/roms/pong-2.ch8 b/games/roms/pong-2.ch8new file mode 100644index 0000000..6af8d1eBinary files /dev/null and b/games/roms/pong-2.ch8 differdiff --git a/games/roms/pong.ch8 b/games/roms/pong.ch8new file mode 100644index 0000000..e371e91Binary files /dev/null and b/games/roms/pong.ch8 differdiff --git a/games/roms/programmable-spacefighters.ch8 b/games/roms/programmable-spacefighters.ch8new file mode 100644index 0000000..a605320Binary files /dev/null and b/games/roms/programmable-spacefighters.ch8 differdiff --git a/games/roms/pumpkin-dreess-up.ch8 b/games/roms/pumpkin-dreess-up.ch8new file mode 100644index 0000000..d066992Binary files /dev/null and b/games/roms/pumpkin-dreess-up.ch8 differdiff --git a/games/roms/puzzle.ch8 b/games/roms/puzzle.ch8new file mode 100644index 0000000..bec7af0Binary files /dev/null and b/games/roms/puzzle.ch8 differdiff --git a/games/roms/reversi.ch8 b/games/roms/reversi.ch8new file mode 100644index 0000000..8a600f0Binary files /dev/null and b/games/roms/reversi.ch8 differdiff --git a/games/roms/rocket-launch.ch8 b/games/roms/rocket-launch.ch8new file mode 100644index 0000000..32bc252Binary files /dev/null and b/games/roms/rocket-launch.ch8 differdiff --git a/games/roms/rocket-launcher.ch8 b/games/roms/rocket-launcher.ch8new file mode 100644index 0000000..d0825d7Binary files /dev/null and b/games/roms/rocket-launcher.ch8 differdiff --git a/games/roms/rocket.ch8 b/games/roms/rocket.ch8new file mode 100644index 0000000..c5db30eBinary files /dev/null and b/games/roms/rocket.ch8 differdiff --git a/games/roms/rps.ch8 b/games/roms/rps.ch8new file mode 100644index 0000000..3ea42d4Binary files /dev/null and b/games/roms/rps.ch8 differdiff --git a/games/roms/rush-hour.ch8 b/games/roms/rush-hour.ch8new file mode 100644index 0000000..e4253e3Binary files /dev/null and b/games/roms/rush-hour.ch8 differdiff --git a/games/roms/russian-roulette.ch8 b/games/roms/russian-roulette.ch8new file mode 100644index 0000000..bb90dcaBinary files /dev/null and b/games/roms/russian-roulette.ch8 differdiff --git a/games/roms/sequence-shoot.ch8 b/games/roms/sequence-shoot.ch8new file mode 100644index 0000000..8715b80Binary files /dev/null and b/games/roms/sequence-shoot.ch8 differdiff --git a/games/roms/shooting-stars.ch8 b/games/roms/shooting-stars.ch8new file mode 100644index 0000000..9cc078fBinary files /dev/null and b/games/roms/shooting-stars.ch8 differdiff --git a/games/roms/slide.ch8 b/games/roms/slide.ch8new file mode 100644index 0000000..7a487edBinary files /dev/null and b/games/roms/slide.ch8 differdiff --git a/games/roms/slippery-slope.ch8 b/games/roms/slippery-slope.ch8new file mode 100644index 0000000..d4f20f5Binary files /dev/null and b/games/roms/slippery-slope.ch8 differdiff --git a/games/roms/snek.ch8 b/games/roms/snek.ch8new file mode 100644index 0000000..ad5b7f1Binary files /dev/null and b/games/roms/snek.ch8 differdiff --git a/games/roms/soccer.ch8 b/games/roms/soccer.ch8new file mode 100644index 0000000..603ce9bBinary files /dev/null and b/games/roms/soccer.ch8 differdiff --git a/games/roms/space-flight.ch8 b/games/roms/space-flight.ch8new file mode 100644index 0000000..60bc934Binary files /dev/null and b/games/roms/space-flight.ch8 differdiff --git a/games/roms/space-intercept.ch8 b/games/roms/space-intercept.ch8new file mode 100644index 0000000..014fe6bBinary files /dev/null and b/games/roms/space-intercept.ch8 differdiff --git a/games/roms/space-invaders.ch8 b/games/roms/space-invaders.ch8new file mode 100644index 0000000..3ada8dfBinary files /dev/null and b/games/roms/space-invaders.ch8 differdiff --git a/games/roms/space-racer.ch8 b/games/roms/space-racer.ch8new file mode 100644index 0000000..fafafcaBinary files /dev/null and b/games/roms/space-racer.ch8 differdiff --git a/games/roms/spacejam.ch8 b/games/roms/spacejam.ch8new file mode 100644index 0000000..7b444a9Binary files /dev/null and b/games/roms/spacejam.ch8 differdiff --git a/games/roms/spock-paper-scissors.ch8 b/games/roms/spock-paper-scissors.ch8new file mode 100644index 0000000..bf65718Binary files /dev/null and b/games/roms/spock-paper-scissors.ch8 differdiff --git a/games/roms/spooky-spot.ch8 b/games/roms/spooky-spot.ch8new file mode 100644index 0000000..ab2dae8Binary files /dev/null and b/games/roms/spooky-spot.ch8 differdiff --git a/games/roms/squash.ch8 b/games/roms/squash.ch8new file mode 100644index 0000000..8ab9cfaBinary files /dev/null and b/games/roms/squash.ch8 differdiff --git a/games/roms/submarine.ch8 b/games/roms/submarine.ch8new file mode 100644index 0000000..224b108Binary files /dev/null and b/games/roms/submarine.ch8 differdiff --git a/games/roms/sum-fun.ch8 b/games/roms/sum-fun.ch8new file mode 100644index 0000000..3c88450Binary files /dev/null and b/games/roms/sum-fun.ch8 differdiff --git a/games/roms/super-pong.ch8 b/games/roms/super-pong.ch8new file mode 100644index 0000000..1287180Binary files /dev/null and b/games/roms/super-pong.ch8 differdiff --git a/games/roms/superworm-v4.ch8 b/games/roms/superworm-v4.ch8new file mode 100644index 0000000..03bc7b8Binary files /dev/null and b/games/roms/superworm-v4.ch8 differdiff --git a/games/roms/syzygy.ch8 b/games/roms/syzygy.ch8new file mode 100644index 0000000..b0653efBinary files /dev/null and b/games/roms/syzygy.ch8 differdiff --git a/games/roms/tank-battle.ch8 b/games/roms/tank-battle.ch8new file mode 100644index 0000000..ca4bbabBinary files /dev/null and b/games/roms/tank-battle.ch8 differdiff --git a/games/roms/tank.ch8 b/games/roms/tank.ch8new file mode 100644index 0000000..6525b62Binary files /dev/null and b/games/roms/tank.ch8 differdiff --git a/games/roms/tapeworm.ch8 b/games/roms/tapeworm.ch8new file mode 100644index 0000000..de279d9Binary files /dev/null and b/games/roms/tapeworm.ch8 differdiff --git a/games/roms/tetris.ch8 b/games/roms/tetris.ch8new file mode 100644index 0000000..9f5e087Binary files /dev/null and b/games/roms/tetris.ch8 differdiff --git a/games/roms/tic-tac-toe.ch8 b/games/roms/tic-tac-toe.ch8new file mode 100644index 0000000..4d4bc99Binary files /dev/null and b/games/roms/tic-tac-toe.ch8 differdiff --git a/games/roms/tick.ch8 b/games/roms/tick.ch8new file mode 100644index 0000000..be3e865Binary files /dev/null and b/games/roms/tick.ch8 differdiff --git a/games/roms/timebomb.ch8 b/games/roms/timebomb.ch8new file mode 100644index 0000000..76828c8Binary files /dev/null and b/games/roms/timebomb.ch8 differdiff --git a/games/roms/tomb-ston-tipp.ch8 b/games/roms/tomb-ston-tipp.ch8new file mode 100644index 0000000..6e785c8Binary files /dev/null and b/games/roms/tomb-ston-tipp.ch8 differdiff --git a/games/roms/tron.ch8 b/games/roms/tron.ch8new file mode 100644index 0000000..74f7f51Binary files /dev/null and b/games/roms/tron.ch8 differdiff --git a/games/roms/ufo.ch8 b/games/roms/ufo.ch8new file mode 100644index 0000000..7fa5a15Binary files /dev/null and b/games/roms/ufo.ch8 differdiff --git a/games/roms/vers.ch8 b/games/roms/vers.ch8new file mode 100644index 0000000..b0fe240Binary files /dev/null and b/games/roms/vers.ch8 differdiff --git a/games/roms/vertical-brix.ch8 b/games/roms/vertical-brix.ch8new file mode 100644index 0000000..07f4006Binary files /dev/null and b/games/roms/vertical-brix.ch8 differdiff --git a/games/roms/wall.ch8 b/games/roms/wall.ch8new file mode 100644index 0000000..efd029fBinary files /dev/null and b/games/roms/wall.ch8 differdiff --git a/games/roms/wdl.ch8 b/games/roms/wdl.ch8new file mode 100644index 0000000..c04234eBinary files /dev/null and b/games/roms/wdl.ch8 differdiff --git a/games/roms/wipe-off.ch8 b/games/roms/wipe-off.ch8new file mode 100644index 0000000..2d5e513Binary files /dev/null and b/games/roms/wipe-off.ch8 differdiff --git a/games/roms/x-mirror.ch8 b/games/roms/x-mirror.ch8new file mode 100644index 0000000..3525307Binary files /dev/null and b/games/roms/x-mirror.ch8 differdiff --git a/games/roms/zeropong.ch8 b/games/roms/zeropong.ch8new file mode 100644index 0000000..3715699Binary files /dev/null and b/games/roms/zeropong.ch8 differdiff --git a/pyproject.toml b/pyproject.tomlnew file mode 100644index 0000000..0c6eebd--- /dev/null+++ b/pyproject.toml@@ -0,0 +1,33 @@+[project]+name = "chipformer"+version = "0.1.0"+description = "A hand-constructed transformer that executes CHIP-8 programs"+requires-python = ">=3.11"+dependencies = [+ "numpy>=1.26",+ "safetensors>=0.8.0",+]++[project.optional-dependencies]+play = ["pygame>=2.5"]+hf = [+ "torch>=2.14.1",+ "transformers>=5.19.0",+]++[dependency-groups]+dev = [+ "pillow>=12.3.0",+ "pytest>=8",+ "ruff>=0.6",+]++[build-system]+requires = ["hatchling"]+build-backend = "hatchling.build"++[tool.ruff]+line-length = 100++[tool.pytest.ini_options]+testpaths = ["tests"]diff --git a/tests/test_chip8.py b/tests/test_chip8.pynew file mode 100644index 0000000..0ab9751--- /dev/null+++ b/tests/test_chip8.py@@ -0,0 +1,76 @@+"""The CHIP-8 circuit and its compiled weights vs the reference emulator."""++import random+from functools import cache+from pathlib import Path++import numpy as np+import pytest++from chipformer.check import check, run_machine+from chipformer.chip8 import build+from chipformer.compile import compile_graph+from chipformer.dense import forward+from chipformer.dsl.batch import evaluate_batch+from chipformer.dsl.evaluate import Evaluator+from chipformer.fuzz import random_program, random_quirks+from chipformer.session import Session+from chipformer.trace import Machine++ROMS = Path(__file__).resolve().parent.parent / "roms"+PONG = ROMS / "pong.ch8"+TEST_ROMS = sorted((ROMS / "test").glob("*.ch8")) if (ROMS / "test").exists() else []+++@cache+def circuit():+ return build()+++def assert_exact(m: Machine) -> None:+ n, bad = check(circuit(), m)+ assert n > 0+ assert not bad, "\n".join(map(str, bad))++[email protected]("seed", range(6))+def test_fuzz_programs(seed):+ quirks = random_quirks(seed)+ assert_exact(run_machine(random_program(seed), 1500, seed=seed, key_rate=0.5, quirks=quirks))++[email protected](not PONG.exists(), reason="roms/pong.ch8 not provided")[email protected]("quirks", ["chip48", "octo", "cosmac"])+def test_pong(quirks):+ assert_exact(run_machine(PONG.read_bytes(), 2000, seed=1, key_rate=0.3, quirks=quirks))++[email protected]("rom", TEST_ROMS, ids=lambda p: p.name)+def test_rom(rom):+ assert_exact(run_machine(rom.read_bytes(), 1500, seed=2, key_rate=0.3))+++def test_dense_transformer_is_bit_identical():+ g = circuit()+ w, config = compile_graph(g)+ m = run_machine(random_program(7), 80, seed=7, key_rate=0.5)+ r, logits = forward(w, config, m.tokens)+ assert np.array_equal(r, evaluate_batch(g, m.tokens))+ gen = np.array(m.generated[1:])+ assert np.array_equal(logits.argmax(1)[:-1][gen], np.array(m.tokens[1:])[gen])+++def test_free_running_generation():+ rom, quirks = random_program(11), random_quirks(11)+ e = Evaluator(circuit())+ s = Session(lambda tok: int(e.output("next", e.feed(tok))), rom, quirks)+ m = Machine(rom, quirks)+ rng = random.Random(0)+ for i in range(25):+ if i % 10 == 0:+ s.tick(), m.tick()+ k, down = rng.randrange(16), rng.random() < 0.5+ s.set_key(k, down), m.set_key(k, down)+ s.step(), m.step()+ assert s.tokens == m.tokens+ assert s.cells == m.emu.cellsdiff --git a/tests/test_dsl.py b/tests/test_dsl.pynew file mode 100644index 0000000..0140e63--- /dev/null+++ b/tests/test_dsl.py@@ -0,0 +1,157 @@+import random++import pytest++from chipformer import spec+from chipformer.dsl.batch import evaluate_batch, output_batch+from chipformer.dsl.evaluate import Evaluator, LiChao, TieError+from chipformer.dsl.lib import Circuit+from chipformer.ref.emu import apply_writes+from chipformer.trace import Machine+++def run_unary(build, xs):+ """Evaluate out = build(c, x) with x fed as the token id; returns outputs."""+ c = Circuit()+ x = c.token_feature("x", lambda tok: tok)+ c.output("o", build(c, x))+ return output_batch(c, "o", evaluate_batch(c, list(xs))).tolist()+++def run_binary(build, n=256):+ c = Circuit()+ x = c.token_feature("x", lambda tok: tok // n)+ y = c.token_feature("y", lambda tok: tok % n)+ c.output("o", build(c, x, y))+ out = output_batch(c, "o", evaluate_batch(c, list(range(n * n))))+ return {(a, b): out[a * n + b] for a in range(n) for b in range(n)}++[email protected]("name,build,ref,rng", [+ ("step", lambda c, x: c.step(x - 100), lambda v: int(v >= 100), range(300)),+ ("eq", lambda c, x: c.eq(x, 77), lambda v: int(v == 77), range(300)),+ ("eqglu", lambda c, x: c.eqglu(x * 3 + 1, x, 9), lambda v: 28 * (v == 9), range(300)),+ ("div16", lambda c, x: c.floor_div(x, 16, 15), lambda v: v // 16, range(256)),+ ("mod16", lambda c, x: c.divmod(x, 16, 15)[1], lambda v: v % 16, range(256)),+ ("wrap", lambda c, x: c.wrap(x, 256), lambda v: v % 256, range(512)),+ ("bits", lambda c, x: c.from_bits(c.bits(x, 8)), lambda v: v, range(256)),+ ("bit5", lambda c, x: c.bits(x, 8)[5], lambda v: (v >> 5) & 1, range(256)),+ ("bits12", lambda c, x: c.from_bits(c.bits(x, 12)), lambda v: v, range(4096)),+])+def test_unary(name, build, ref, rng):+ assert run_unary(build, rng) == [ref(v) for v in rng]++[email protected]("name,build,ref", [+ ("and", lambda c, x, y: c.band(x, y), lambda a, b: a & b),+ ("or", lambda c, x, y: c.bor(x, y), lambda a, b: a | b),+ ("xor", lambda c, x, y: c.bxor(x, y), lambda a, b: a ^ b),+ ("add8", lambda c, x, y: c.wrap(x + y, 256), lambda a, b: (a + b) & 0xFF),+ ("sub8", lambda c, x, y: c.wrap(x - y + 256, 256), lambda a, b: (a - b) & 0xFF),+ ("ge", lambda c, x, y: c.ge(x, y), lambda a, b: int(a >= b)),+ ("mux", lambda c, x, y: c.mux(c.step(x - 128), y, 255 - y), lambda a, b: b if a >= 128 else 255 - b),+])+def test_binary_exhaustive(name, build, ref):+ got = run_binary(build)+ assert all(v == ref(a, b) for (a, b), v in got.items())+++def test_layers_are_asap():+ c = Circuit()+ x = c.token_feature("x", lambda t: t)+ s = c.step(x) # FFN of layer 1+ (p,) = c.prev([s]) # attention of layer 2+ c.step(p) # FFN of layer 2+ assert [n.level for n in c.nodes] == [2, 3, 4] and c.n_layers() == 2+++def test_lichao_matches_brute():+ rng = random.Random(0)+ lc = LiChao(-50, 50)+ lines = []+ for i in range(300):+ m, b = rng.randrange(-1000, 1000), rng.randrange(-10**6, 10**6)+ lc.insert(m, b, i)+ lines.append((m, b))+ for x in rng.sample(range(-50, 51), 5):+ best = max(m * x + b for m, b in lines)+ j = lc.query(x)+ assert lines[j][0] * x + lines[j][1] == best+++def memory_circuit():+ """Latest-write memory over a canonical trace, queried at a per-position address."""+ c = Circuit()+ is_val = c.token_feature("is_val", lambda t: int(spec.is_val(t)))+ val = c.token_feature("val", lambda t: t - spec.VAL_BASE if spec.is_val(t) else 0)+ addr = c.token_feature("addr", lambda t: t - spec.ADDR_BASE if spec.is_addr(t) else 0)+ addr2 = c.token_feature("addr^2", lambda t: (t - spec.ADDR_BASE) ** 2 if spec.is_addr(t) else 0)+ query = c.pos_feature("query", lambda t: (t * 2654435761) % 64 + spec.V) # pseudo-random reg+ a, a2 = c.prev([addr, addr2])+ v, found = c.latest_keyed(query, a, a2, is_val, [val], spec.NUM_CELLS)+ c.output("read", c.eqglu(v, found - query))+ c.output("query", query)+ return c+++def random_machine(steps: int, seed: int = 3) -> Machine:+ rng = random.Random(seed)+ m = Machine(bytes(rng.randrange(256) for _ in range(400)))+ for i in range(steps):+ if i % 10 == 0:+ m.tick()+ m.step()+ return m+++def expected_reads(tokens, queries):+ cells = [0] * spec.NUM_CELLS+ pending = None+ out = []+ for tok, q in zip(tokens, queries):+ if spec.is_addr(tok):+ pending = tok - spec.ADDR_BASE+ elif spec.is_val(tok):+ apply_writes(cells, [(pending, tok - spec.VAL_BASE)])+ out.append(cells[int(q)])+ return out++[email protected]("mode", ["lichao", "brute"])+def test_memory_reads_incremental(mode):+ m = random_machine(60)+ c = memory_circuit()+ e = Evaluator(c, mode)+ rows = [e.feed(tok) for tok in m.tokens]+ queries = [e.output("query", r) for r in rows]+ assert [e.output("read", r) for r in rows] == expected_reads(m.tokens, queries)+++def test_memory_reads_batch_long():+ m = random_machine(3000)+ c = memory_circuit()+ res = evaluate_batch(c, m.tokens)+ queries = output_batch(c, "query", res)+ assert output_batch(c, "read", res).tolist() == expected_reads(m.tokens, queries)+++def test_batch_equals_incremental():+ m = random_machine(40, seed=9)+ c = memory_circuit()+ e = Evaluator(c)+ rows = [e.feed(tok) for tok in m.tokens]+ assert evaluate_batch(c, m.tokens).tolist() == [[float(v) for v in r] for r in rows]+++def test_ties_are_detected():+ c = Circuit()+ (v,) = c.latest(c.one * 0 + 1, [c.pos])+ c.output("o", v)+ e = Evaluator(c, "brute")+ e.feed(0)+ c2 = Circuit()+ c2.lookup(0, 0, 0, [c2.pos], (0, 0)) # every key scores 0: a tie+ e2 = Evaluator(c2, "brute")+ e2.feed(0)+ with pytest.raises(TieError):+ e2.feed(0)diff --git a/tests/test_engine.py b/tests/test_engine.pynew file mode 100644index 0000000..bd5b5ab--- /dev/null+++ b/tests/test_engine.py@@ -0,0 +1,50 @@+"""The Rust engine vs the circuit, and its free-running traces vs the reference.++Needs `cargo build --release` in engine/ and a compiled model in model/+(`uv run python -m chipformer.compile`).+"""++import subprocess+from pathlib import Path++import numpy as np+import pytest++from chipformer.check import run_machine, verify_trace+from chipformer.chip8 import build+from chipformer.dsl.batch import evaluate_batch, output_batch+from chipformer.fuzz import random_program, random_quirks++ROOT = Path(__file__).resolve().parent.parent+EXE = ROOT / "engine" / "target" / "release" / ("engine.exe" if (ROOT / "engine" / "target" / "release" / "engine.exe").exists() else "engine")+MODEL = ROOT / "model"++pytestmark = pytest.mark.skipif(not (EXE.exists() and (MODEL / "model.safetensors").exists()),+ reason="engine or model not built")+++def test_predict_matches_circuit(tmp_path):+ m = run_machine(random_program(21), 800, seed=21, key_rate=0.5)+ np.array(m.tokens, dtype="<u4").tofile(tmp_path / "t.u32")+ subprocess.run([str(EXE), "predict", str(MODEL), str(tmp_path / "t.u32"), str(tmp_path / "p.u32")],+ check=True, capture_output=True)+ pred = np.fromfile(tmp_path / "p.u32", dtype="<u4").astype(float)+ g = build()+ assert np.array_equal(pred, output_batch(g, "next", evaluate_batch(g, m.tokens)))++[email protected]("rom", ["fuzz", "pong"])+def test_free_running_with_rollover(tmp_path, rom):+ path, quirks = ROOT / "roms" / "pong.ch8", "chip48"+ if rom == "fuzz":+ path, quirks = tmp_path / "fuzz.ch8", ",".join(random_quirks(5))+ path.write_bytes(random_program(5))+ elif not path.exists():+ pytest.skip("roms/pong.ch8 not provided")+ subprocess.run([str(EXE), "run", str(MODEL), str(path), "--instr", "4000", "--seed", "9",+ "--quirks", quirks,+ "--rollover", "8000", "--dump", str(tmp_path / "run.u32")],+ check=True, capture_output=True)+ n, err = verify_trace(np.fromfile(tmp_path / "run.u32", dtype="<u4").tolist())+ assert err is None, err+ assert n > 4000diff --git a/tests/test_games.py b/tests/test_games.pynew file mode 100644index 0000000..dbe26da--- /dev/null+++ b/tests/test_games.py@@ -0,0 +1,33 @@+"""Featured games: the Rust engine plays each one free-running, verified record by record.++Needs `cargo build --release` in engine/ and `uv run python -m chipformer.compile`.+"""++import json+import subprocess+from pathlib import Path++import numpy as np+import pytest++from chipformer.check import verify_trace++ROOT = Path(__file__).resolve().parent.parent+RELEASE = ROOT / "engine" / "target" / "release"+EXE = RELEASE / "engine.exe" if (RELEASE / "engine.exe").exists() else RELEASE / "engine"+MANIFEST = ROOT / "games" / "manifest.json"+FEATURED = sorted((g for g in json.loads(MANIFEST.read_text(encoding="utf-8"))["games"]+ if "featured" in g), key=lambda g: g["featured"]) if MANIFEST.exists() else []++pytestmark = pytest.mark.skipif(not (EXE.exists() and (ROOT / "model").exists()),+ reason="engine or model not built")++[email protected]("game", FEATURED, ids=lambda g: g["id"])+def test_featured_game(tmp_path, game):+ subprocess.run([str(EXE), "run", str(ROOT / "model"), str(ROOT / "games" / game["file"]),+ "--instr", "5000", "--ipf", str(game["ipf"]), "--quirks", ",".join(game["quirks"]),+ "--seed", "3", "--dump", str(tmp_path / "t.u32")], check=True, capture_output=True)+ n, err = verify_trace(np.fromfile(tmp_path / "t.u32", dtype="<u4").tolist())+ assert err is None, err+ assert n > 5000diff --git a/tests/test_hf.py b/tests/test_hf.pynew file mode 100644index 0000000..6f3766e--- /dev/null+++ b/tests/test_hf.py@@ -0,0 +1,82 @@+"""The Hugging Face export: loads through transformers and runs games token-exact."""++from functools import cache++import numpy as np+import pytest++torch = pytest.importorskip("torch")+transformers = pytest.importorskip("transformers")++from chipformer.chip8 import build+from chipformer.compile import compile_graph+from chipformer.dense import forward+from chipformer.hf_export import GAMES, example_game, export+from chipformer.trace import Machine++[email protected](scope="module")+def model(tmp_path_factory):+ out = tmp_path_factory.mktemp("hf")+ export(out)+ return transformers.AutoModelForCausalLM.from_pretrained(out, trust_remote_code=True,+ dtype=torch.float64)+++@cache+def dense_weights():+ return compile_graph(build())+++def run_both(model, frames: int, sparse: bool):+ """The same input schedule through the transformer and the reference emulator."""+ g = example_game()+ rom = (GAMES / g["file"]).read_bytes()+ quirks = ",".join(g["quirks"])+ game = model.play(rom, quirks=quirks, ipf=g["ipf"], keep_tokens=True, sparse=sparse)+ ref = Machine(rom, quirks)+ keys = g["keys"]+ for f in range(frames):+ held = [keys["a"]] if f % 30 == 2 else [keys["left"]] if 5 <= f % 30 < 9 else \+ [keys["down"]] if f % 30 >= 20 else []+ game.frame(held)+ for k in range(16):+ ref.set_key(k, k in held)+ ref.tick()+ for _ in range(g["ipf"]):+ ref.step()+ return game, ref+++def test_loads_as_causal_lm(model):+ assert type(model).__name__ == "ChipformerForCausalLM"+ assert next(model.parameters()).dtype == torch.float64+ assert sum(int((p != 0).sum()) for p in model.parameters()) == dense_weights()[1]["nonzero_params"]+++def test_matches_numpy_dense_forward(model):+ w, config = dense_weights()+ tokens = Machine((GAMES / example_game()["file"]).read_bytes()).tokens[:400]+ _, want = forward(w, config, tokens)+ model.sparse(False)+ with torch.inference_mode():+ got = model(torch.tensor([tokens])).logits[0].numpy()+ np.testing.assert_allclose(got, want, rtol=1e-12, atol=1e-6)+ assert (got.argmax(1) == want.argmax(1)).all()++[email protected]("sparse", [True, False])+def test_game_trace_is_exact(model, sparse):+ game, ref = run_both(model, 40 if sparse else 8, sparse)+ assert len(game.tokens) > len(ref.tokens[:1]) and game.tokens == ref.tokens+ assert game.cells == ref.emu.cells+++def test_rollover_keeps_running(model):+ model.config.rollover_at = 2500+ try:+ game, ref = run_both(model, 60, True)+ finally:+ model.config.rollover_at = 250_000+ assert game.rollovers >= 1+ assert game.cells == ref.emu.cellsdiff --git a/tests/test_q35_chip8.py b/tests/test_q35_chip8.pynew file mode 100644index 0000000..a45e7c7--- /dev/null+++ b/tests/test_q35_chip8.py@@ -0,0 +1,87 @@+"""CHIP-8 as a q35 circuit, and as a stock Qwen3.5 checkpoint.++The circuit is checked against the reference trace token by token with the exact IR evaluator+(which also asserts bf16 exactness of every intermediate). The checkpoint test needs+``python -m chipformer.q35.tiny`` (dist/q35-tiny) and loads it with AutoModelForCausalLM.+"""++import json+from pathlib import Path++import numpy as np+import pytest++from chipformer.check import run_machine, verify_trace+from chipformer.fuzz import random_program, random_quirks+from chipformer.q35 import check as Q+from chipformer.q35 import chip8_q35++ROOT = Path(__file__).resolve().parent.parent+TINY = ROOT / "dist" / "q35-tiny"+GAMES = {g["id"]: g for g in json.loads((ROOT / "games" / "manifest.json").read_text(+ encoding="utf-8"))["games"]}++[email protected](scope="module")+def circuit():+ return chip8_q35.build()+++def game(gid, steps, seed=2):+ g = GAMES[gid]+ rom = (ROOT / "games" / g["file"]).read_bytes()+ return run_machine(rom, steps, ipf=g["ipf"], seed=seed, key_rate=0.3, quirks=tuple(g["quirks"]))++[email protected]("rom", ["3-corax+.ch8", "4-flags.ch8", "5-quirks.ch8"])[email protected]("quirks", ["octo", "chip48", "cosmac"])+def test_test_roms(circuit, rom, quirks):+ m = run_machine((ROOT / "roms" / "test" / rom).read_bytes(), 300, ipf=10, seed=1,+ quirks=quirks)+ n, bad = Q.check(circuit, m)+ assert n > 1000 and not bad, Q.describe(bad)++[email protected]("seed", range(4))+def test_fuzz(circuit, seed):+ m = run_machine(random_program(seed), 300, ipf=7, seed=seed, key_rate=0.3,+ quirks=random_quirks(seed))+ _, bad = Q.check(circuit, m)+ assert not bad, Q.describe(bad)++[email protected](not TINY.exists(), reason="run python -m chipformer.q35.tiny first")[email protected]("dtype", ["bfloat16", "float32"])+def test_checkpoint_prefill(dtype):+ torch = pytest.importorskip("torch")+ from transformers import AutoModelForCausalLM++ model = AutoModelForCausalLM.from_pretrained(TINY, dtype=getattr(torch, dtype)).eval()+ m = game("brix", 150)+ with torch.no_grad():+ pred = model(torch.tensor([m.tokens])).logits[0, :, :8708].argmax(-1).numpy()+ want = np.array(m.tokens[1:] + [-1])+ gen = np.array(m.generated[1:] + [False])+ assert gen.sum() > 500+ assert not (gen & (pred != want)).any()++[email protected](not TINY.exists(), reason="run python -m chipformer.q35.tiny first")+def test_checkpoint_free_running():+ torch = pytest.importorskip("torch")+ from transformers import AutoModelForCausalLM++ from chipformer.q35 import host++ model = AutoModelForCausalLM.from_pretrained(TINY, dtype=torch.bfloat16).eval()+ g = GAMES["tetris"]+ s = host.session(model, (ROOT / "games" / g["file"]).read_bytes(), tuple(g["quirks"]))+ for i in range(24):+ if i % g["ipf"] == 0:+ s.tick()+ if i == 8:+ s.set_key(5, True)+ s.step()+ n, err = verify_trace(s.tokens)+ assert err is None, err+ assert n >= 28diff --git a/tests/test_q35_expr.py b/tests/test_q35_expr.pynew file mode 100644index 0000000..d4b43f8--- /dev/null+++ b/tests/test_q35_expr.py@@ -0,0 +1,217 @@+"""Whole expressions (v18): the tape capture, the BIOS evaluator eval, and the circuit."""++import random+import re+from fractions import Fraction++import pytest++from chipformer.q35 import bridge as B+from chipformer.q35 import check as Q+from chipformer.q35 import chip8_q35+from chipformer.q35.lib import bcd+from test_q35_inline import TIMES, classes, printed++INV = {v: k for k, v in B.EXPANSION.items()}+++def tape_of(text: str) -> list[int]:+ """A string of symbols as tape codes, using fused codes where they exist."""+ out, i = [], 0+ multi = sorted(B.MULTI, key=len, reverse=True)+ while i < len(text):+ if text[i].isdigit():+ out.append(int(text[i]))+ i += 1+ continue+ m = next((m for m in multi if text.startswith(m, i)), text[i])+ out.append(INV[m])+ i += len(m)+ return out+++def python_value(text: str):+ """The expression's exact value by Python (^ is **, a bracket after an operand multiplies)."""+ text = re.sub(r"([\d)])\(", r"\1*(", text).replace("^", "**")+ text = re.sub(r"\d+(\.\d*)?", lambda m: f"Fraction('{m[0]}')", text)+ try:+ return eval(text, {"Fraction": Fraction}) # noqa: S307+ except ZeroDivisionError:+ return None+++def run_eval(tape, mode=1):+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(2)+ n0 = len(bm.tokens)+ bm._ecall(tape, mode)+ steps = 0+ while not bm.halted:+ bm.step()+ steps += 1+ assert steps < 20000, "eval does not halt"+ out = [t for t, g in zip(bm.tokens[n0:], bm.generated[n0:]) if B.is_text(t)]+ shown = {**{h: str(i) for i, h in enumerate(B.HEX)}, B.PERIOD: ".", B.ELLIPSIS: "...",+ B.DIV: "/"}+ # inline: " -" first; a tool response: " -" instead of the newline after <tool_response>+ return B.MINUS in out[:3], "".join(shown.get(t, "") for t in out[1:-1]), len(bm.tokens) - n0+++def random_expr(rng, d=0):+ def num():+ n = str(rng.randrange(0, 10 ** rng.randint(1, 4)))+ if rng.random() < 0.15:+ n += "." + str(rng.randrange(0, 1000))+ if rng.random() < 0.25:+ n = rng.choice(["(-" + n + ")", "-" + n])+ return n++ def operand():+ return num() if d > 2 or rng.random() < 0.65 else "(" + random_expr(rng, d + 1) + ")"++ parts = [operand()]+ for _ in range(rng.randint(1, 3)):+ op = rng.choice("+-*/" * 3 + "^")+ rhs = operand()+ if op == "^":+ rhs = str(rng.randrange(0, 5)) if rng.random() < 0.8 else "-" + str(rng.randrange(1, 3))+ parts += [op, rhs]+ return "".join(parts)+++FIXED = [("3+4*5", "23"), ("(3+4)*5", "35"), ("2^10-5", "1019"), ("((-8)/(-60))*329", "658/15"),+ ("20*(-8)/(-60)*329*9/((-36)/2)*7*(-6)/49", "376"), ("1/3", "1/3"),+ ("-2^2", "-4"), ("2^-2", "0.25"), ("0.5*4", "2"), ("1.25+2.5", "3.75"), ("7-10", "-3"),+ ("-(3-5)", "2"), ("2(3+4)", "14"), ("10/4", "2.5"), ("1/0", "..."), ("0*(-5)/7", "0"),+ ("(((9/12)/((-2)/232*-6))/((244521/6456)/303))/(((-2)/(-2))/1)", "116"),+ ("3+*4", "..."), ("2^123", "..."), ("99999999999999999999*99999", "...")]++[email protected]("text,want", FIXED)+def test_eval_fixed(text, want):+ neg, body, _ = run_eval(tape_of(text))+ assert ("-" if neg else "") + body == want+ assert bcd.eval_text(tape_of(text)) == (want.startswith("-"), want.lstrip("-"))+++def test_eval_random():+ """eval (the BIOS on the reference machine) == bcd.eval_text == Python's exact value."""+ rng = random.Random(18)+ n = errors = 0+ lengths = []+ for _ in range(400):+ text = random_expr(rng)+ tape = tape_of(text)+ if len(tape) > B.TAPE_MAX:+ continue+ want = bcd.eval_text(tape)+ neg, body, n_tok = run_eval(tape)+ assert (neg, body) == want, (text, want, (neg, body))+ lengths.append(n_tok)+ v = python_value(text)+ if want[1] == "...":+ errors += 1+ continue+ n += 1+ got = Fraction(body.rstrip(".")) * (-1 if neg else 1)+ if body.endswith("..."): # 10 decimals, truncated+ assert abs(v - got) < Fraction(1, 10**10), (text, v, body)+ else:+ assert v == got, (text, v, body)+ assert n > 250 and errors < n // 4, (n, errors)+ lengths.sort()+ assert lengths[len(lengths) // 2] < 8000, lengths[len(lengths) // 2]+++def scan(tokens):+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(3)+ calls = bm.qwen(classes(tokens))+ return bm, calls+++BS = chr(92)+SCAN = [+ ([" ", "(", "3", " +", " ", "4", ")", *TIMES, "12", " /", " ", "7", " ="], "(3+4)*12/7"),+ ([" ", "2", "^{", "10", "}", " -", " ", "5", " ="], "2^(10)-5"),+ ([" ", "((-", "8", ")/", "(-", "60", "))*", "329", " ="], "((-8)/(-60))*329"),+ ([" $", " ", "1", ".", "25", " +", " ", "2", ".", "5", " ="], "1.25+2.5"),+ ([" (", "1", ",", "234", " +", " ", "5", ")", " ", "*", " ", "2", " ="], "(1234+5)*2"),+ ([" =", " (", "12", " -", " ", "3", ")", "*", "45", " ="], "(12-3)*45"), # fresh after "="+ ([" ", "3", " +", " ", "4", " +", " ", "5", " ="], None), # too few digits+ (["x", " -", " ", "3", " +", " ", "45", " ="], None), # starts after a letter+ (["f", "(", "3", " +", " ", "45", ")", " ="], None),+ ([" ", "12", " +", " ", "3", " apples", " +", " ", "45", " ="], None), # broken window+ ([" ", "1234", *TIMES, "5678", " ="], "fast"), # one operator: the single-op path+ ([" ", "^", "12", " +", " ", "345", " ="], None), # an operator can't start it+ # a "." that ends a number breaks the window: no 3.11 - 8, and 12+345+6 alone+ ([" ", "3", ".", " $", "11", " -", " ", "8", " ="], None),+ ([" ", "1", ".", " ", "12", "+", "345", "+", "6", " ="], "12+345+6"),+]++[email protected]("tokens,want", SCAN)+def test_scan(tokens, want):+ _bm, calls = scan(tokens)+ if want is None:+ assert calls == []+ elif want == "fast":+ assert [c[0] for c in calls] == ["mul"]+ else:+ assert len(calls) == 1 and calls[0][0] == "eval"+ assert "".join(B.EXPANSION[c] for c in calls[0][1]) == want+++def test_prints():+ bm, _ = scan([" $", " ", "(", "3", " -", " ", "40", ")", " ", "*", " ", "2", " =", "$"])+ assert printed(bm) == " -74$"+ bm, _ = scan([" ", "100", " /", " ", "(", "3", " +", " ", "4", ")", " ="])+ assert printed(bm) == " 100/7."+ bm, _ = scan([" ", "(", "12", " /", " ", "0", ")", " +", " ", "5", " ="])+ assert printed(bm) == " ...." # can't: "..."++[email protected](scope="module")+def circuit():+ return chip8_q35.build()+++def check(circuit, bm):+ n, bad, bad_owner = Q.check_bridge(circuit, bm.tokens, bm.generated, B.owner_expected(bm))+ assert not bad and not bad_owner, (bad[:5], bad_owner[:5])+ return n++[email protected]("tokens,want", SCAN[:4] + SCAN[5:7] + SCAN[-2:])+def test_circuit_scan(circuit, tokens, want):+ bm, _ = scan(tokens)+ bm.text(2)+ check(circuit, bm)+++def test_circuit_prints(circuit):+ """A negative result and "..." in one session."""+ bm, calls = scan([" $", " ", "(", "3", " -", " ", "40", ")", " ", "*", " ", "2", " =", "$",+ " ", "and", " ", "(", "12", " /", " ", "0", ")", " +", " ", "5", " ="])+ assert [c[0] for c in calls] == ["eval", "eval"] and printed(bm) == " -74$ ...."+ bm.text(2)+ check(circuit, bm)+++def test_circuit_tool(circuit):+ """The tool calc(expression): the tape is the call's argument, the result a tool response."""+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(3)+ bm.calc_call(classes(["(", "3", "+", "4", ")*", "5"]))+ while not bm.halted:+ bm.step()+ bm.text(2)+ assert printed(bm) == "???35" + "?" * (3 + len(B.TOOL_TAIL))+ check(circuit, bm)+++def test_tool_negative():+ """A negative value prints with "-", inline and (since v16) as a tool response."""+ assert run_eval(tape_of("3-10"), mode=0)[:2] == (True, "7")+ assert run_eval(tape_of("3-10"), mode=1)[:2] == (True, "7")+ assert run_eval(tape_of("10-3*2"), mode=0)[:2] == (False, "4")diff --git a/tests/test_q35_inline.py b/tests/test_q35_inline.pynew file mode 100644index 0000000..88bcb1b--- /dev/null+++ b/tests/test_q35_inline.py@@ -0,0 +1,271 @@+"""Inline arithmetic: the circuit answers "A op B =" in Qwen's plain text."""++import random++import pytest++from chipformer.q35 import bridge as B+from chipformer.q35 import check as Q+from chipformer.q35 import chip8_q35+from chipformer.q35.lib import bcd++BS = chr(92)+WORDS = {" ": B.SPACE, " =": B.EQ, "=": B.EQ, " +": B.PLUS, "+": B.PLUS, " -": B.MINUS,+ "-": B.MINUS2, "−": B.MINUS2, " ×": B.TIMES, "*": B.TIMES, " *": B.TIMES, " " + BS: B.BSL, "times": B.TIMESW,+ ",": B.COMMA, "$": B.DOLLAR, " $": B.SDOLLAR, "$$": B.DDOLLAR, " $$": B.SDDOLLAR, "(": B.LPAREN, "\n": B.NL, "\n\n": B.WS,+ "/": B.DIV, " /": B.DIV2, "÷": B.DIV2, "div": B.FN_DIV, ".": B.PERIOD,+ "(-": B.LPMINUS, " (-": B.SLPMINUS, ")": B.RPAREN, ")/": B.RPDIV,+ " mod": B.MODOP, " %": B.MODOP, "mod": B.FN_MOD, "b": B.HEX[11], "gcd": B.FN_GCD,+ " gcd": B.FN_GCD}+++def classes(tokens: list[str]) -> list[int]:+ """Qwen tokens (digits split, as its tokenizer does) to text classes."""+ out = []+ for t in tokens:+ if t.isdigit():+ out += [B.HEX[int(ch)] for ch in t]+ else:+ out.append(WORDS.get(t, B.CLASS_OF.get(t, B.OTHER)))+ return out+++def run(tokens):+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(3)+ calls = bm.qwen(classes(tokens))+ bm.text(3)+ return bm, calls+++def printed(bm) -> str:+ out = []+ for t, g in zip(bm.tokens, bm.generated):+ if g and B.is_text(t):+ close = {B.SPACE: " ", B.MINUS: " -", B.PERIOD: ".", B.DOLLAR: "$", B.DDOLLAR: "$$",+ B.ELLIPSIS: "...", B.DIV: "/"}+ out.append(close[t] if t in close else str(B.HEX.index(t)) if t in B.HEX else "?")+ return "".join(out)+++TIMES = [" " + BS, "times", " "]+EV = "eval" # a whole expression (test_q35_expr.py)+CASES = [+ (["$", "12", *TIMES, "34", " =", " "], [("mul", "12", "34")]),+ ([" ", "1", ",", "234", ",", "567", *TIMES, "8", " ="], [("mul", "1234567", "8")]),+ ([" ", "37", " ×", " ", "43", " ="], [("mul", "37", "43")]),+ ([" ", "37", "*", "43", "="], [("mul", "37", "43")]),+ ([" ", " $", " ", "3000", " +", " ", "400", " ="], [("add", "3000", "400")]),+ ([" ", " $", " ", "300", " +", " ", "4", " ="], []), # small add / sub: Qwen's own+ ([" ", "10000", " -", " ", "30", " ="], [("sub", "10000", "30")]),+ ([" ", "100", " -", " ", "30", " ="], []),+ ([" ", "25", " -", " ", "1", ",", "786", ",", "338", " ="], [("sub", "25", "1786338")]),+ ([" ", "2500", "−", "17860", " ="], [("sub", "2500", "17860")]),+ (["\n\n", "42450", " *", " ", "8376", " ="], [("mul", "42450", "8376")]),+ ([" ", "2", ".", "55", *TIMES, "300", " ="], [EV]), # decimal: an expression+ (["x", "^", "22", " +", " ", "100", " ="], []), # not a plain first number+ ([" ", "20", " +", " ", "30", " +", " ", "40", " ="], [EV]), # chains: expressions+ ([" ", "20", *TIMES, "30", " +", " ", "40", " ="], [EV]),+ (["}", *TIMES, "30", " +", " ", "40", " ="], []), # operator before the first number+ (["-", "50", *TIMES, "30", " ="], []), # "-" after a letter: an operator+ # signed numbers: the single-op path (the expression path stays quiet when it fires)+ ([" ", "50", *TIMES, "-", "30", " ="], [("mul", "50", "30")]),+ (["\n", "-", "5000", *TIMES, "30", " ="], [("mul", "5000", "30")]),+ (["\n", "-", " ", "50", *TIMES, "30", " ="], [("mul", "50", "30")]), # a list's "- "+ (["x", " -", " ", "12", *TIMES, "34", " ="], []), # x - 12 * 34: "-" is an operator+ ([" ", "12", " -", "34", *TIMES, "5", " ="], [EV]), # 12 - 34 * 5: an expression+ (["\n", "-", "1418261", " /", " (-", "408", ")", " ="], [("div", "1418261", "408")]),+ (["$", "(-", "1418261", ")/", "(-", "408", ")", " ="], [("div", "1418261", "408")]),+ (["f", "(-", "3000", ")", *TIMES, "4", " ="], []), # f(-3000): not a number's sign+ ([" ", "=", " -", "5", " +", " ", "1234567", " ="], []), # add / sub: unsigned only+ ([" ", "1234567", " -", " -", "5", " ="], [EV]), # (the expression path takes it)+ (["\n", "-", "1234", " mod", " ", "56", " ="], []),+ ([" ", "1785", " mod", " ", "13", " ="], [("mod", "1785", "13")]),+ ([" ", "1785", " %", " ", "13", " ="], [("mod", "1785", "13")]),+ ([" ", "1785", " " + BS, "mod", " ", "13", " ="], [("mod", "1785", "13")]),+ ([" ", "1785", " " + BS, "b", "mod", " ", "13", " ="], [("mod", "1785", "13")]),+ ([" ", "1785", " ", "mod", " ", "13", " ="], []), # "mod" without a backslash+ ([" ", "gcd", "(", "1785", ",", " ", "13201915", ")", " ="], [("gcd", "1785", "13201915")]),+ (["$", BS, "gcd", "(", "1785", ",", " ", "-", "35", ")", " =", "$"], [("gcd", "1785", "35")]),+ ([" ", "1785", " gcd", "(", "12", ",", " ", "35", ")", " ="], []), # gcd after a number+ ([" ", "gcd", "(", "1785", ",", "35", ")", " ="], []), # "1785,35" is one number+ (["(", "50", " +", " ", "30", ")", " ="], []),+ ([" ", "12", *TIMES, "3", " ="], []), # small: Qwen's own+ ([" ", "5", " +", " ", "7", " ="], []),+ ([" ", "50", " +", " ", "30", " apples", " ="], []),+ ([" ", "30", " ", "40", *TIMES, "50", " ="], []), # two numbers before the operator+ ([" ", "50", " =", " ", "30", *TIMES, "40", " ="], [("mul", "30", "40")]), # fresh after =+ (["x", "^", "2", " =", " ", "4818396", *TIMES, "7", " ="], [("mul", "4818396", "7")]),+ ([" ", "50", " =", "30", *TIMES, "40", " ="], []), # a number right after "=" is not counted+ (["x", "4245", *TIMES, "60", " ="], []), # a letter right before the first number+ ([" ", "1686", "/", "6", " ="], [("div", "1686", "6")]),+ (["$", "259749", " /", " ", "3162", " ="], [("div", "259749", "3162")]),+ ([" ", "720", " " + BS, "div", " ", "45", " ="], [("div", "720", "45")]),+ ([" ", "100", "÷", "7", " ="], [("div", "100", "7")]),+ ([" ", "1", "/", "2", " ="], []), # small+ ([" ", "km", "/", "1234", " ="], []),+ ([" ", "1234", " ", "div", " ", "5", " ="], []), # "div" without a backslash+]++[email protected]("tokens,want", CASES)+def test_scan(tokens, want):+ _bm, calls = run(tokens)+ assert [EV if c[0] == EV else c for c in calls] == want+++def test_scan_prints_and_chains():+ bm, calls = run(["$", "12", *TIMES, "34", " =", "$", " ", "and", " ", "408", *TIMES, "2", " ="])+ assert calls == [("mul", "12", "34"), ("mul", "408", "2")]+ assert printed(bm) == " 408$ 816." # "$" closes the math it opened in, else "."+ # after a "." closer the single-op path does not go on from the printed result, but a whole+ # expression does: "408." is a number (v18)+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(3)+ calls = bm.qwen(classes([" ", "12", *TIMES, "34", " ="]))+ calls += bm.qwen(classes([*TIMES, "2", " ="]))+ assert [c[0] for c in calls] == ["mul", "eval"] and printed(bm) == " 408. 816."+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(3)+ calls = bm.qwen(classes([" $$", " ", "3000", " +", " ", "4000", " ="]))+ assert calls == [("add", "3000", "4000")] and printed(bm) == " 7000$$"+++def test_div_prints():+ for a, b, want in [("1686", "6", " 281."), ("1000", "8", " 125."), ("10", "16", " 0.625."),+ ("100", "7", " 100/7."), ("1", "3000", " 1/3000."),+ ("99999999", "9999", " 10001."),+ ("5", "125000000000000", " 1/25000000000000."), ("1234", "56", " 617/28.")]:+ bm, calls = run([" ", a, "/", b, " ="])+ assert calls == [("div", a, b)] and printed(bm) == want, (a, b, printed(bm))+ bm, _ = run(["$", "355", "/", "113", " ="])+ assert printed(bm) == " 355/113$"+ bm, _ = run([" ", "1234", "/", "0", " ="]) # division by zero: "..."+ assert printed(bm) == " ...."+++def test_signed_mod_gcd_prints():+ for tokens, want in [+ (["\n", "-", "1418261", " /", " (-", "408", ")", " ="], " 1418261/408."),+ (["$", "-", "2548", " /", " ", "52", " ="], " -49$"),+ ([" ", "12", *TIMES, "-", "34", " ="], " -408."),+ ([" (-", "1234", ")", *TIMES, "(-", "5", ")", " ="], " 6170."),+ (["\n", "-", "1", " /", " ", "3000", " ="], " -1/3000."),+ (["\n", "-", "1234", " /", " ", "0", " ="], " -...."), # by zero: "..." (and the sign)+ ([" ", "1785", " mod", " ", "13", " ="], " 4."),+ ([" ", "1785", " mod", " ", "1785", " ="], " 0."),+ ([" ", "17", " mod", " ", "5000", " ="], " 17."),+ ([" ", "00017", " mod", " ", "5000", " ="], " 17."),+ ([" ", "1785", " mod", " ", "0", " ="], " ...."),+ ([" ", "gcd", "(", "1785", ",", " ", "13201915", ")", " ="], " 5."),+ ([" ", "gcd", "(", "13201915", ",", " ", "1785", ")", " ="], " 5."),+ ([" ", "gcd", "(", "1234", ",", " ", "0", ")", " ="], " 1234."),+ ([" ", "gcd", "(", "0", ",", " ", "1234", ")", " ="], " 1234."),+ ([" ", "gcd", "(", "123456789012", ",", " ", "987654321098", ")", " ="], " 2."),+ ([" ", "gcd", "(", "2", "0" * 23, ",", " ", "4", "0" * 20, ")", " ="],+ " 4" + "0" * 20 + ".")]:+ bm, calls = run(tokens)+ assert len(calls) == 1 and printed(bm) == want, (tokens, printed(bm))+++def test_negative_sub_prints_minus():+ bm, calls = run([" ", "3000", " -", " ", "5000", " ="])+ assert calls == [("sub", "3000", "5000")] and printed(bm) == " -2000."+ bm, calls = run(["$", "21", " -", " ", "97000", " ="])+ assert calls == [("sub", "21", "97000")] and printed(bm) == " -96979$"+ bm, calls = run([" ", "4567", " -", " ", "4567", " ="])+ assert printed(bm) == " 0."++[email protected](scope="module")+def circuit():+ return chip8_q35.build()+++def check(circuit, bm):+ n, bad, bad_owner = Q.check_bridge(circuit, bm.tokens, bm.generated, B.owner_expected(bm))+ assert not bad and not bad_owner, (bad[:5], bad_owner[:5])+ return n++[email protected]("tokens,want", CASES)+def test_circuit_cases(circuit, tokens, want):+ bm, _ = run(tokens)+ check(circuit, bm)+++def test_circuit_with_tool_calls(circuit):+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(4)+ bm.qwen(classes(["$", "48", *TIMES, "12", " =", "$", " ", "then"]))+ bm.tool_call("add", "576", "99")+ while not bm.halted:+ bm.step()+ bm.qwen(classes([" ", "675", " -", " ", "5000", " =", "."]))+ tail = "?" * (3 + len(B.TOOL_TAIL)) # "\n</tool_response>\n" and the empty thinking block+ assert printed(bm) == f" 576$???675{tail} -4325." # ? = the tool response's tags and newlines+ bm.tool_call("div", "355", "113")+ while not bm.halted:+ bm.step()+ assert printed(bm).endswith("???355/113" + tail)+ check(circuit, bm)+ bm.tool_call("sub", "5", "675") # below 0: " -" instead of the newline+ while not bm.halted:+ bm.step()+ assert printed(bm).endswith("355/113" + tail + "??" + " -670" + tail)+ for op, x, y, want in [("div", "-1418261", "-408", "1418261/408"),+ ("mul", "-63075", "8808", " -555564600"), ("div", "-5", "0", " -..."),+ ("mod", "13201915", "1785", "55"), ("gcd", "1785", "13201915", "5"),+ ("gcd", "-12", "18", "6")]:+ bm.tool_call(op, x, y)+ while not bm.halted:+ bm.step()+ assert printed(bm).endswith(want + tail), (op, x, y, printed(bm)[-40:])+ check(circuit, bm)+++def test_template_tool_tags_do_not_fire(circuit):+ """Qwen's tool instructions show an empty "<tool_call></tool_call>": after inline operators in+ earlier text it must not start a call (the routine is cleared at <tool_call>)."""+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(3)+ calls = bm.qwen(classes([" ", "-", " ", "x", " +", " ", "y", " ", "*", " "])+ + [B.TC_OPEN, B.TC_CLOSE, B.NL] + classes([" ", "1234", " +"]) + [B.OTHER] * 3)+ assert calls == []+ check(circuit, bm)+ bm.tool_call("mul", "37", "43") # and a real call afterwards still works+ while not bm.halted:+ bm.step()+ bm.text(3)+ check(circuit, bm)+++def test_circuit_fuzz(circuit):+ """Random streams of the classes the capture cares about: the circuit fires exactly where+ the reference does, and the records and prints that follow are exact."""+ rng = random.Random(5)+ pool = ([B.HEX[d] for d in range(10)] * 4 + [B.SPACE] * 4+ + [B.EQ, B.PLUS, B.MINUS, B.MINUS2, B.DIV, B.TIMES, B.BSL, B.TIMESW, B.COMMA, B.DOLLAR, B.LPAREN,+ B.NL, B.OTHER, B.OTHER, B.LPMINUS, B.SLPMINUS, B.RPAREN, B.MODOP, B.FN_MOD,+ B.FN_GCD, B.HEX[11]])+ fired = 0+ for _ in range(12):+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(2)+ stream = []+ for _ in range(60):+ if rng.random() < 0.15: # plant an expression+ a, b = str(rng.randrange(1, 10**rng.randint(1, 5))), str(rng.randrange(10**rng.randint(1, 4)))+ op = rng.choice([" +", " -", "-", "*", TIMES, "/", [" " + chr(92), "div", " "],+ " mod", [" " + chr(92), "b", "mod", " "], "gcd"])+ sa, sb = rng.choice([[], ["-"], [" (-"]]), rng.choice([[], ["-"], ["(-"]])+ if op == "gcd":+ stream += classes([" ", "gcd", "(", *sa, a, ",", " ", *sb, b, ")", " ="])+ else:+ stream += [B.SPACE, *classes([*sa, a]),+ *classes(op if isinstance(op, list) else [op]),+ B.SPACE, *classes([*sb, b]), B.EQ]+ stream.append(rng.choice(pool))+ fired += len(bm.qwen(stream))+ check(circuit, bm)+ assert fired >= 8diff --git a/tests/test_q35_ir.py b/tests/test_q35_ir.pynew file mode 100644index 0000000..2eb9c4e--- /dev/null+++ b/tests/test_q35_ir.py@@ -0,0 +1,155 @@+"""The q35 IR, emitter and tiny stock-Qwen model agree on a toy memory machine.++Tokens: VAL v (0..255), ADDR a (0..255), READ a, CLS, NOP. ADDR a, VAL v writes cell a; READ a+predicts VAL(cell a), every other token predicts NOP; CLS clears memory and restarts a counter of+VAL tokens, decoded to [w > m] for m < 40.+"""++import numpy as np+import pytest++torch = pytest.importorskip("torch")+pytest.importorskip("transformers")++from chipformer.q35 import ir, tiny+from chipformer.q35.ir import A_BIT, A_REF, Chan, Graph, K++VAL, ADDR, READ, CLS, NOP = 0, 256, 512, 768, 769+VOCAB = list(range(770))+NM = 40+CODE = 10 # bits of the token id+++def build():+ g = Graph()+ one = g.one+ f = g.token_feature+ is_val = f("is_val", lambda t: int(t < 256))+ vb = [f(f"vb{i}", lambda t, i=i: (t >> i) & 1 if t < 256 else 0) for i in range(8)]+ an = [[f(f"an{j}[{x}]", lambda t, j=j, x=x: int(256 <= t < 512 and (t - 256) >> 4 * j & 15 == x))+ for x in range(16)] for j in range(2)]+ rn = [[f(f"rn{j}[{x}]", lambda t, j=j, x=x: int(512 <= t < 768 and (t - 512) >> 4 * j & 15 == x))+ for x in range(16)] for j in range(2)]+ is_cls = f("is_cls", lambda t: int(t == CLS))+ is_read = f("is_read", lambda t: int(512 <= t < 768))+ code = [f(f"code{i}", lambda t, i=i: 128 * (2 * ((t >> i) & 1) - 1)) for i in range(CODE)]++ found, bits = [], [[] for _ in range(8)]+ for h in range(2):+ key = {s: Chan(K * (an[0][s & 15] + an[1][(s >> 4) + 8 * h] - one), (0, 1, 0, 0))+ for s in range(128)}+ query = {s: K * (rn[0][s & 15] + rn[1][(s >> 4) + 8 * h] - one) for s in range(128)}+ val = {0: A_REF * one, 1: A_REF * one, **{2 + i: A_BIT * vb[i] for i in range(8)}}+ outs = [(0, 2.0**-10)] + [(2 + i, 2.0**7) for i in range(8)]+ z = {c: K * one for c, _ in outs}+ res = g.head(f"bank{h}", query, key, val, 238 * is_cls, z, outs)+ found.append(res[0])+ for i in range(8):+ bits[i].append(res[1 + i])++ (x,) = g.head("counter",+ {0: K * (one - is_cls), 1: K * (one - is_cls)},+ {0: K * is_cls, 1: K * (one - is_cls)},+ {0: A_REF * (one - is_cls), 1: A_REF * (one - is_cls), 2: A_BIT * is_cls},+ 110 * is_val + 238 * is_cls, {2: K * one}, [(2, 2.0**7)])+ dec = [g.ffn([(1, one, 7 * one - 8 * 2**m * x), (-1, one, 6 * one - 8 * 2**m * x)],+ f"w>{m}") for m in range(NM)]+ hit = g.ffn([(1, one, found[0] + found[1] + bits[0][0] + bits[0][1] - one)], "found&b0")++ rb = [b[0] + b[1] for b in bits]+ nxt = []+ for i in range(CODE):+ terms = [(1, one, one - is_read)] if (NOP >> i) & 1 else []+ if i < 8:+ terms.append((1, one, is_read + rb[i] - one))+ nxt.append(g.ffn(terms, f"next{i}") if terms else 0 * one)+ S = 2**12+ for i in range(CODE):+ g.ffn([(2 * S, one, nxt[i]), (-S, one, one), (-1, one, code[i]), (1, one, -1 * code[i])],+ f"code{i}", into=code[i])+ return g, {"found": found, "bits": bits, "x": x, "dec": dec, "hit": hit, "code": code}+++def make_tokens(n, seed=0):+ rng = np.random.default_rng(seed)+ toks, cells, w = [CLS], {}, 0+ want = {"x": [0.0], "read": [None], "next": [NOP]}+ while len(toks) < n:+ r = rng.random()+ if r < 0.45:+ a, v = int(rng.integers(256)), int(rng.integers(256))+ toks += [ADDR + a, VAL + v]+ cells[a] = v+ want["read"] += [None, None]+ want["next"] += [NOP, NOP]+ w += 1+ want["x"] += [2.0**-(w - 1), 2.0**-w]+ elif r < 0.9:+ a = int(rng.integers(256))+ toks.append(READ + a)+ want["read"].append(cells.get(a))+ want["next"].append(VAL + cells.get(a, 0))+ want["x"].append(2.0**-w)+ elif r < 0.95:+ toks.append(CLS)+ cells, w = {}, 0+ want["read"].append(None)+ want["next"].append(NOP)+ want["x"].append(0.0)+ else:+ toks.append(-1) # a text token+ want["read"].append(None)+ want["next"].append(None)+ want["x"].append(None)+ return toks, want++[email protected](scope="module")+def graph():+ return build()+++def test_ir_semantics(graph):+ g, d = graph+ toks, want = make_tokens(3000, seed=1)+ m = ir.evaluate(g, toks)+ for t, tok in enumerate(toks):+ if tok < 0:+ continue+ if want["x"][t] is not None and t and want["x"][t] >= 2.0**-100:+ x = ir.ev(d["x"], m[t : t + 1])[0]+ assert x == want["x"][t], (t, x, want["x"][t])+ w = round(-np.log2(x))+ for mm, e in enumerate(d["dec"]):+ assert ir.ev(e, m[t : t + 1])[0] == float(w > mm), (t, mm, w)+ if want["read"][t] is not None:+ v = sum(int(ir.ev(b[0] + b[1], m[t : t + 1])[0]) << i for i, b in enumerate(d["bits"]))+ assert v == want["read"][t]++[email protected]("dtype", [torch.bfloat16, torch.float32], ids=["bf16", "fp32"])+def test_tiny_model_matches_ir(graph, dtype):+ g, d = graph+ model = tiny.build(g, VOCAB, [next(iter(c.terms)) for c in d["code"]], dtype=dtype)+ lay = model.layout+ toks, want = make_tokens(1200, seed=2)+ ids = torch.tensor([[t if t >= 0 else len(VOCAB) for t in toks]]) # padding row = text+ m = ir.evaluate(g, toks)+ caught = []+ n_chip = g.n_layers()+ hook = model.model.layers[n_chip - 1].register_forward_hook(lambda mod, i, o: caught.append(o))+ with torch.no_grad():+ logits = model(ids).logits[0].float()+ hook.remove()+ res = caught[0][0].float().numpy().astype(np.float64)+ cols = [lay.dim(i) for i in range(1, len(g.dims))]+ got, ref = res[:, cols], m[:, 1:]+ chip = np.array(toks) >= 0+ if dtype == torch.bfloat16:+ assert np.array_equal(got[chip], ref[chip])+ else:+ assert np.abs(got[chip] - ref[chip]).max() < 1e-3+ pred = logits[:, : len(VOCAB)].argmax(-1).numpy()+ for t, w in enumerate(want["next"]):+ if w is not None:+ assert pred[t] == w, (t, pred[t], w)diff --git a/tests/test_q35_ir_vheads.py b/tests/test_q35_ir_vheads.pynew file mode 100644index 0000000..1b57695--- /dev/null+++ b/tests/test_q35_ir_vheads.py@@ -0,0 +1,85 @@+"""The IR's newer head features in real HF modules: a key head's second value head with its own+write gate, value conv taps, and outputs added onto shared dims. The residual stream of a stock+Qwen3_5ForCausalLM equals the IR evaluator's, exactly."""++import random++import numpy as np+import torch++from chipformer.dsl.graph import Expr+from chipformer.q35 import ir+from chipformer.q35.emit import embedding_rows, emit_layers+from chipformer.q35.ir import Chan, Target+from chipformer.q35.prims import Circ+from chipformer.q35.store_q35 import thead+from chipformer.q35.tiny import BASE, pick_layout++N_TOK = 28 # tokens 0..15: data (4 bits); 16..19: write slot s; 20..23 / 24..27: read slot s+++def graph():+ c = Circ(Target(key_heads=2, value_heads=4, intermediate=256))+ f = c.token_feature+ bit = [f(f"b{i}", lambda t, i=i: int(t < 16 and t >> i & 1)) for i in range(4)]+ wr = [f(f"w{s}", lambda t, s=s: int(t == 16 + s)) for s in range(4)]+ rd = [f(f"r{s}", lambda t, s=s: int(t == 20 + s)) for s in range(4)]+ rd2 = [f(f"r2.{s}", lambda t, s=s: int(t == 24 + s)) for s in range(4)]+ hi = f("hi", lambda t: int(t in (18, 19, 22, 23, 26, 27))) # slots 2, 3: second value head+ one = c.one+ # rows = slot s; a write token stores the bits of the two data tokens before it (lags 1, 2)+ data = {2 + 4 * j + i: Chan(bit[i], (0, 1, 0, 0) if j == 0 else (0, 0, 1, 0))+ for j in range(2) for i in range(4)}+ keys = {s: wr[s] for s in range(4)}+ qs = {s: rd[s] for s in range(4)}+ wa = c.and_([sum(wr, Expr()), one - hi], "wa")+ wb = c.and_([sum(wr, Expr()), hi], "wb")+ outs = thead(c, "t", keys, qs, [+ {"data": data, "ref": one, "beta": wa, "gate": one - hi},+ {"data": data, "ref": one, "beta": wb, "gate": hi, "into": "main"}])+ read = outs[0]+ # a second head (one value head, all slots) adds onto the same dims, read by its own tokens+ keys2 = {s: wr[s] for s in range(4)}+ qs2 = {s: rd2[s] for s in range(4)}+ thead(c, "t2", keys2, qs2, [{"data": data, "ref": one, "into": read}])+ out = [c.node(c.t_and([r_]), f"out{i}") for i, r_ in enumerate(read)]+ return c, out+++def hf(c):+ from transformers import Qwen3_5ForCausalLM, Qwen3_5TextConfig++ lay = pick_layout(len(c.dims))+ n = c.n_layers()+ tc = dict(BASE, hidden_size=lay.hidden, vocab_size=64, num_hidden_layers=n + 1,+ layer_types=["linear_attention"] * n + ["full_attention"],+ linear_num_key_heads=2, linear_num_value_heads=4, intermediate_size=256)+ sd = emit_layers(c, lay, tc, 0, "model.")+ emb = torch.zeros(64, lay.hidden)+ emb[:N_TOK] = embedding_rows(c, lay, list(range(N_TOK)), lay.hidden)+ sd["model.embed_tokens.weight"] = emb+ model = Qwen3_5ForCausalLM(Qwen3_5TextConfig(**tc))+ for p in model.parameters():+ p.data.zero_()+ model.load_state_dict(sd, strict=False)+ return model.to(torch.bfloat16).eval(), lay+++def test_vheads_in_hf():+ c, out = graph()+ model, lay = hf(c)+ rng = random.Random(0)+ toks = []+ for _ in range(40):+ toks += [rng.randrange(16), rng.randrange(16), 16 + rng.randrange(4)]+ if rng.random() < 0.6:+ toks.append(rng.choice([20, 24]) + rng.randrange(4))+ m = ir.evaluate(c, toks)+ with torch.no_grad():+ hs = model(input_ids=torch.tensor([toks]), output_hidden_states=True).hidden_states+ h = hs[c.n_layers()][0].double().numpy()+ for e in out:+ (d,) = e.terms+ want = m[:, d]+ assert np.isin(want, [0.0, 1.0]).all() and want.any()+ assert (h[:, lay.dim(d)] == want).all(), (d, np.nonzero(h[:, lay.dim(d)] != want)[0][:5])diff --git a/tests/test_q35_macros.py b/tests/test_q35_macros.pynew file mode 100644index 0000000..0c166a0--- /dev/null+++ b/tests/test_q35_macros.py@@ -0,0 +1,135 @@+"""The looping bignum macros (BMUL0 / BMUL, BTRIM, BRSUB, BADD / BSUB with jump targets):+reference results, trace lengths, and the circuit against the reference."""++import random++import pytest++from chipformer.q35 import bridge as B+from chipformer.q35 import check as Q+from chipformer.q35 import chip8_q35+from chipformer.q35.lib import bcd+from chipformer.q35.lib.macros import macro_writes++CASES = [("mul", "7", "8"), ("mul", "123456", "7"), ("mul", "7", "123456"), ("mul", "0", "4567"),+ ("mul", "99999999", "99999999"), ("mul", "1000", "1000"), ("add", "0", "0"),+ ("add", "999999", "1"), ("sub", "10000", "9999"), ("sub", "4567", "4567"),+ ("sub", "1234", "98765"), ("sub", "5", "6"), ("sub", "0", "0")]+++def run_call(op, a, b, mode):+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(3)+ n0 = len(bm.tokens)+ bm._call(op, a, b, mode=mode)+ while not bm.halted:+ bm.step()+ return bm, n0+++def shown(bm, n0):+ sym = {B.SPACE: " ", B.MINUS: " -", B.PERIOD: "."}+ return "".join(sym.get(t, str(B.HEX.index(t)) if t in B.HEX else "")+ for t, g in zip(bm.tokens[n0:], bm.generated[n0:]) if g and B.is_text(t))++[email protected]("op,a,b", CASES)+def test_reference_results(op, a, b):+ v = {"add": int(a) + int(b), "sub": int(a) - int(b), "mul": int(a) * int(b)}[op]+ bm, n0 = run_call(op, a, b, 1)+ assert shown(bm, n0) == (" -" if v < 0 else " ") + str(abs(v)) + "."+ bm, n0 = run_call(op, a, b, 0) # a tool call: " -" instead of the newline below 0+ assert shown(bm, n0) == (" -" + str(-v) if v < 0 else str(v))+++def test_trace_lengths():+ """Tokens from the call to the end of the print (the v13 BIOS took 209 / 383 / 84 / 576)."""+ for (op, a, b), most in [(("mul", "123456", "7890"), 140), (("mul", "12345678", "87654321"), 265),+ (("add", "1234", "567"), 60), (("sub", "1234", "5678"), 85)]:+ bm, n0 = run_call(op, a, b, 1)+ assert len(bm.tokens) - n0 <= most, (op, a, b, len(bm.tokens) - n0)+++def test_btrim_only_reads_the_result():+ from chipformer import spec++ cells = [0] * 0x2000+ for i, v0, trims in [(bcd.R_ADDR, 3, True), (bcd.R_ADDR, 1, False), (0x300, 3, False),+ (bcd.R_ADDR + 2 * bcd.MAX_DIGITS, 3, False)]:+ cells[spec.I], cells[spec.V], cells[spec.PC] = i, v0, 0x210+ w = macro_writes(cells, 0x5008)+ want = [(spec.V, v0 - 1), (spec.I, i + 1)] if trims else [(spec.PC, 0x212)] # stays: no PC+ assert w == want, (i, v0, w)++[email protected](scope="module")+def circuit():+ return chip8_q35.build()+++OPW = {"add": B.PLUS, "sub": B.MINUS, "mul": B.TIMES}+++def inline(bm, op, a, b, opener=()):+ """Qwen writes " a op b =" (the circuit fires if the grammar allows it)."""+ return bm.qwen([*opener, B.SPACE, *(B.HEX[int(ch)] for ch in a), OPW[op], B.SPACE,+ *(B.HEX[int(ch)] for ch in b), B.EQ])+++def test_circuit_macros(circuit):+ """Every case, inline and as a tool call, in one session: records and prints exact."""+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(3)+ fired = 0+ for op, a, b in CASES:+ fired += len(inline(bm, op, a, b))+ bm.text(2)+ bm.tool_call(op, a, b)+ while not bm.halted:+ bm.step()+ bm.text(2)+ assert fired >= 8+ _n, bad, bad_owner = Q.check_bridge(circuit, bm.tokens, bm.generated, B.owner_expected(bm))+ assert not bad and not bad_owner, (bad[:5], bad_owner[:5])+++def test_circuit_random_sessions(circuit):+ rng = random.Random(7)+ fired = 0+ for _ in range(4):+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(2)+ for _ in range(3):+ op = rng.choice(["add", "sub", "mul"])+ la, lb = (rng.randint(1, 9), rng.randint(1, 9)) if op == "mul" else (+ rng.randint(1, 20), rng.randint(1, 20))+ a, b = (str(rng.randrange(10 ** n)) for n in (la, lb))+ if rng.random() < 0.5:+ fired += len(inline(bm, op, a, b, rng.choice([(), (B.DOLLAR,), (B.DDOLLAR,)])))+ else:+ bm.tool_call(op, a, b)+ while not bm.halted:+ bm.step()+ bm.text(3)+ _n, bad, bad_owner = Q.check_bridge(circuit, bm.tokens, bm.generated, B.owner_expected(bm))+ assert not bad and not bad_owner, (bad[:5], bad_owner[:5])+ assert fired >= 2+++def test_ops_are_fixed():+ assert {k: v for k, v in bcd.OPS.items() if k in ("add", "sub", "mul")} == {+ "add": 0x202, "sub": 0x206, "mul": 0x20A}+++def test_bld_bsta():+ """BLD copies VX cells from I into A (B), with the length; BSTA stores A at I."""+ from chipformer import spec++ cells = [0] * spec.NUM_CELLS+ cells[0x900:0x905] = [3, 1, 4, 1, 5]+ cells[spec.I], cells[spec.V + 2], cells[spec.PC] = 0x900, 4, 0x300+ w = macro_writes(cells, 0x521B) # bld b, v2+ assert w == [(bcd.B_ADDR + j, v) for j, v in enumerate([3, 1, 4, 1])] + [+ (bcd.LEN + 1, 4), (spec.PC, 0x302)]+ cells[bcd.A_ADDR:bcd.A_ADDR + 3], cells[bcd.LEN] = [9, 8, 7], 3+ assert macro_writes(cells, 0x500C) == [(0x900, 9), (0x901, 8), (0x902, 7), (spec.PC, 0x302)]diff --git a/tests/test_q35_mute.py b/tests/test_q35_mute.pynew file mode 100644index 0000000..df56c78--- /dev/null+++ b/tests/test_q35_mute.py@@ -0,0 +1,106 @@+"""The mute patch: on chip tokens Qwen's own layers keep their state and write nothing.++A small random Qwen3.5 (text + vision) is widened like a real merge, with ballast dims after+Qwen's. Chip-like tokens carry the ballast; text tokens don't.+"""++import pytest++torch = pytest.importorskip("torch")+pytest.importorskip("transformers")++from chipformer.q35 import merge+from chipformer.q35.emit import Layout+from chipformer.q35.mute import mute++D0, HIDDEN, M = 128, 256, 1 # 1 * 2^80 / 256 = 4^36+VOCAB = 64+++def small_model():+ from transformers import Qwen3_5Config, Qwen3_5ForConditionalGeneration++ torch.manual_seed(0)+ cfg = Qwen3_5Config(+ text_config={"vocab_size": VOCAB, "hidden_size": D0, "intermediate_size": 256,+ "num_hidden_layers": 4, "num_attention_heads": 2, "num_key_value_heads": 1,+ "head_dim": 256, "linear_num_key_heads": 2, "linear_num_value_heads": 2,+ "linear_key_head_dim": 64, "linear_value_head_dim": 64,+ "tie_word_embeddings": True},+ vision_config={"depth": 1, "hidden_size": 32, "intermediate_size": 64, "num_heads": 2,+ "out_hidden_size": D0},+ tie_word_embeddings=True)+ model = Qwen3_5ForConditionalGeneration(cfg).eval()+ for n, p in model.named_parameters():+ if n.endswith("norm.weight"):+ p.data.normal_(0, 0.1) # nonzero zero-centred gains, like a trained model+ return model++[email protected](scope="module")+def models():+ base = small_model()+ cfg = base.config.to_dict()+ sd = {k: v.detach().clone() for k, v in base.state_dict().items() if k != "lm_head.weight"}+ cfg2, sd2 = merge.widen(cfg, sd, HIDDEN)+ plain = merge.instantiate(cfg2, {k: v.clone() for k, v in sd2.items()}, torch.float32)+ lay = Layout(hidden=HIDDEN, off=D0, m=M)+ mute(sd2, cfg2["text_config"], 4, lay.off, lay.ballast * lay.scale)+ muted = merge.instantiate(cfg2, sd2, torch.float32)+ return plain, muted, lay+++def embed(model, lay, toks, chip):+ e = model.get_input_embeddings()(torch.tensor([toks]))+ e = e.clone()+ for i, is_chip in enumerate(chip):+ if is_chip:+ e[0, i, :D0] = 0+ e[0, i, lay.off] = lay.ballast+ return e+++def run(model, e):+ states = []+ with torch.no_grad():+ out = model(inputs_embeds=e, use_cache=True, output_hidden_states=True)+ cache = out.past_key_values+ for i, layer in enumerate(cache.layers):+ if hasattr(layer, "recurrent_states") and layer.recurrent_states:+ states.append(layer.recurrent_states[0].clone())+ return out, states+++def test_text_is_bit_identical(models):+ plain, muted, lay = models+ toks = list(range(3, 40))+ e = embed(plain, lay, toks, [False] * len(toks))+ a, _ = run(plain, e)+ b, _ = run(muted, e)+ assert torch.equal(a.logits, b.logits)+++def test_chip_tokens_keep_qwen_state(models):+ _, muted, lay = models+ text = list(range(3, 23))+ e_text = embed(muted, lay, text, [False] * len(text))+ e_chip = embed(muted, lay, [5] * 30, [True] * 30)+ out, s_text = run(muted, e_text)+ # decode the chip tokens one by one: the recurrent kernel leaves the state bit-identical+ cache = out.past_key_values+ with torch.no_grad():+ for i in range(e_chip.shape[1]):+ o = muted(inputs_embeds=e_chip[:, i : i + 1], past_key_values=cache, use_cache=True,+ output_hidden_states=True)+ cache = o.past_key_values+ for h in o.hidden_states: # chip positions add nothing to Qwen's dims+ assert h[0, :, :D0].abs().max() == 0+ s_dec = [layer.recurrent_states[0] for layer in cache.layers+ if hasattr(layer, "recurrent_states") and layer.recurrent_states]+ assert s_text and len(s_dec) == len(s_text)+ for x, y in zip(s_text, s_dec):+ assert torch.equal(x, y), "chip tokens changed a Qwen DeltaNet state"+ # one chunked prefill over text + chip: equal up to the kernel's rounding+ _, s_both = run(muted, torch.cat([e_text, e_chip], 1))+ for x, y in zip(s_text, s_both):+ assert torch.allclose(x, y, rtol=1e-5, atol=1e-7)diff --git a/tests/test_q35_numerics.py b/tests/test_q35_numerics.pynew file mode 100644index 0000000..99991a7--- /dev/null+++ b/tests/test_q35_numerics.py@@ -0,0 +1,51 @@+"""Phase 0: the Qwen3.5 primitives the embedded CHIP-8 relies on, in the real HF modules."""++import pytest++torch = pytest.importorskip("torch")+pytest.importorskip("transformers")++from chipformer.q35 import spike as S++# fp32 is checked for token exactness (small, bounded error); bf16 must be bit-exact+TOL = {torch.bfloat16: 0.0, torch.float32: 1e-4}++[email protected](scope="module")+def seq():+ x, want = S.make_sequence(1500, seed=3)+ x[:, S.GARBAGE :] *= 1e3 # Qwen-dim garbage up to ~1e6 must not move the norm+ return x, want++[email protected]("dtype", [torch.bfloat16, torch.float32], ids=["bf16", "fp32"])[email protected]("mode", ["prefill", "decode"])+def test_ram_counter_mlp(seq, dtype, mode):+ x, want = seq+ model = S.build(dtype)+ res = S.run(model, x, mode, split=700)+ err, leak, n = S.compare(res, want, x)+ assert n > 10_000+ assert leak == 0.0, "text tokens changed chip dims"+ if dtype == torch.bfloat16:+ assert err == 0.0+ else:+ # relative check on the counter, absolute elsewhere+ for t, exp in want.items():+ for d, v in exp.items():+ tol = TOL[dtype] * (v if d == S.OUT_CNT and v else 1.0)+ assert abs(float(res[t, d]) - v) <= tol, (t, d, float(res[t, d]), v)+++def test_scalar_thresholds():+ for dtype in (torch.bfloat16, torch.float32):+ n = torch.arange(-6, 7, dtype=dtype)+ silu = torch.nn.functional.silu(128 * n)+ assert torch.equal(silu, torch.clamp(128 * n, min=0)), dtype+ sig = torch.sigmoid(torch.tensor([128.0, 0.0], dtype=dtype))+ assert sig.tolist() == [1.0, 0.5]+ sp = torch.nn.functional.softplus(torch.tensor([-110.0, 0.0, 128.0]))+ assert sp[0] == 0 and sp[2] == 128+ assert torch.exp(torch.tensor(-128.0)) == 0+ for k in range(-40, 41): # rsqrt of powers of 4 is exact+ assert torch.rsqrt(torch.tensor(4.0**k)) == 2.0**-kdiff --git a/tests/test_q35_overlay.py b/tests/test_q35_overlay.pynew file mode 100644index 0000000..08d6086--- /dev/null+++ b/tests/test_q35_overlay.py@@ -0,0 +1,155 @@+"""The overlay (overlay.py): circuit dims that only chip tokens need live in Qwen's dims.++IR level: the switched-off circuit predicts exactly what the reference does, and random Qwen+values in the overlaid dims at text positions change nothing else. Weights level: a merge of a+small random Qwen3.5 (the real tokenizer, 0.8B shapes, two layers) keeps Qwen's dims of every+text token bit-identical through the chip layers, clears them on chip tokens, and prints the+reference trace.+"""++import json+import random+from pathlib import Path++import numpy as np+import pytest++from chipformer.q35 import bridge as B+from chipformer.q35 import check as Q+from chipformer.q35 import chip8_q35, overlay+from chipformer.q35.lib import bcd++TOKENIZERS = sorted(Path.home().glob(".cache/huggingface/hub/models--Qwen--Qwen3.5-*/snapshots/*/"+ "tokenizer.json"))+++def traces() -> list[B.BridgeMachine]:+ from test_q35_expr import SCAN, scan+ from test_q35_inline import CASES, classes, run++ out = [run(toks)[0] for toks, _ in CASES[::3]]+ out += [scan(toks)[0] for toks, want in SCAN[:4] if want] # v18: whole expressions+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(4)+ for op, a, b in [("mul", "37", "43"), ("add", "48213", "9921"), ("div", "355", "113"),+ ("sub", "5", "675"), ("div", "-1418261", "-408"),+ ("mod", "13201915", "1785"), ("gcd", "1785", "13201915")]:+ bm.tool_call(op, a, b)+ while not bm.halted:+ bm.step()+ bm.text(3)+ bm.calc_call(classes(["(", "3", "+", "4", ")*", "5"])) # v18: calc(expression)+ while not bm.halted:+ bm.step()+ bm.text(3)+ out.append(bm)+ rng = random.Random(3)+ pool = [B.HEX[d] for d in range(10)] * 3 + [B.SPACE, B.EQ, B.TIMES, B.DIV, B.PLUS, B.OTHER]+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(2)+ for _ in range(6):+ bm.qwen([B.SPACE, B.HEX[4], B.HEX[2], B.HEX[1], B.HEX[7], B.TIMES, B.HEX[6], B.HEX[3],+ B.EQ] + [rng.choice(pool) for _ in range(12)])+ out.append(bm)+ return out++[email protected](scope="module")+def overlaid():+ c = chip8_q35.build()+ p = overlay.build(c, set(c.keep), overlay.corpus(docs=12))+ return c, p+++def test_plan(overlaid):+ c, p = overlaid+ assert len(p.dims) > len(c.dims) // 2+ assert not p.dims & set(c.keep)+ assert p.q_bound > 100 # far above any Qwen embedding value+++def test_circuit_is_unchanged(overlaid):+ c, _ = overlaid+ for bm in traces():+ n, bad, bad_owner = Q.check_bridge(c, bm.tokens, bm.generated, B.owner_expected(bm))+ assert not bad and not bad_owner, (n, bad, bad_owner)++[email protected]("scale", [1.0, 100.0])+def test_qwen_values_change_nothing(overlaid, scale):+ c, p = overlaid+ rng = np.random.default_rng(int(scale))+ for bm in traces():+ overlay.check_noisy(c, p, bm.tokens, rng, scale)+++def test_allocation_uses_qwen_dims(overlaid):+ from chipformer.q35.emit import allocate++ c, p = overlaid+ keep = set(c.keep)+ wide = allocate(chip8_q35.build(), None, keep, c.target.intermediate)+ a = allocate(c, None, keep, c.target.intermediate, overlay=p.dims, overlay_cap=1024)+ assert len(a.final_over) <= 1024 and set(a.over) <= p.dims+ assert a.n_slots < wide.n_slots - 400+++def small_qwen(dst: Path, tokenizer: Path) -> None:+ """A random Qwen3.5 with 0.8B shapes (the circuit's target) and two layers."""+ import shutil++ import torch+ from transformers import Qwen3_5Config, Qwen3_5ForConditionalGeneration++ cfg = json.loads((tokenizer.parent / "config.json").read_text())+ tc = cfg["text_config"]+ tc.update(hidden_size=1024, intermediate_size=3584, linear_num_value_heads=16,+ num_hidden_layers=2, layer_types=["linear_attention", "full_attention"],+ full_attention_interval=2, mtp_num_hidden_layers=0, num_attention_heads=8,+ num_key_value_heads=2, tie_word_embeddings=True)+ cfg["vision_config"].update(depth=1, hidden_size=64, intermediate_size=128, num_heads=2,+ out_hidden_size=1024)+ cfg["tie_word_embeddings"] = True+ torch.manual_seed(0)+ config = Qwen3_5Config(**{k: v for k, v in cfg.items() if k not in ("architectures",+ "model_type")})+ model = Qwen3_5ForConditionalGeneration(config).to(torch.bfloat16)+ model.save_pretrained(dst)+ for f in ("tokenizer.json", "tokenizer_config.json", "chat_template.jinja"):+ if (tokenizer.parent / f).exists():+ shutil.copy(tokenizer.parent / f, dst / f)++[email protected](not TOKENIZERS, reason="needs a Qwen3.5 tokenizer in the HF cache")+def test_merged_checkpoint(tmp_path):+ torch = pytest.importorskip("torch")+ from chipformer.q35 import merge++ src, dst = tmp_path / "src", tmp_path / "merged"+ small_qwen(src, TOKENIZERS[-1])+ c, lay = merge.build_merged(src, dst, mute_qwen=False)+ meta = json.loads((dst / "chipformer.json").read_text())+ assert meta["overlay_slots"] > 0 and lay.hidden < 1024 + meta["slots"] + 2 * lay.m + 400+ cfg, sd = merge.load(dst)+ model = merge.instantiate(cfg, sd, torch.float32)+ lm = model.model.language_model+ v0, n_chip, d0 = meta["chip_token_offset"], meta["chip_layers"], meta["chip_dims_offset"]+ ids = {int(k): v for k, v in meta["text_ids"].items()}+ other = next(i for i in range(1000, 2000) if i not in ids.values())+ for bm in traces()[-2:]:+ toks = [v0 + t if not B.is_text(t) else ids.get(t, other) for t in bm.tokens]+ caught = []+ hook = lm.layers[n_chip - 1].register_forward_hook(+ lambda m, i, o: caught.append(o[0] if isinstance(o, tuple) else o))+ with torch.no_grad():+ logits = model(input_ids=torch.tensor([toks])).logits[0]+ hook.remove()+ res = caught[0][0]+ emb = lm.embed_tokens.weight[torch.tensor(toks)]+ txt = torch.tensor([B.is_text(t) for t in bm.tokens])+ assert torch.equal(res[txt, :d0], emb[txt, :d0]) # Qwen's input, bit for bit+ assert (res[~txt, :d0] == 0).all() # chip tokens: overlay slots cleared+ gen = np.array(bm.generated[1:] + [False])+ want = np.array(toks[1:] + [-1])+ pred = logits.argmax(-1).numpy()+ assert not (gen & (pred != want)).any(), np.nonzero(gen & (pred != want))[0][:5]diff --git a/tests/test_q35_store.py b/tests/test_q35_store.pynew file mode 100644index 0000000..62908f0--- /dev/null+++ b/tests/test_q35_store.py@@ -0,0 +1,323 @@+"""The object store: reference semantics, and the circuit against the reference token by token."""++import random++import pytest++from chipformer.q35 import bridge as B+from chipformer.q35 import chip8_q35+from chipformer.q35 import store as S+from chipformer.q35 import store_check as SC+from chipformer.q35.ir import Target+from chipformer.q35.lib import bcd++TARGETS = {"4b": Target(16, 32, 9216), "0.8b": Target(16, 16, 3584)}++[email protected](scope="module")+def vocab():+ return S.Vocab()++[email protected](scope="module")+def circuit(vocab):+ return chip8_q35.build(TARGETS["4b"], store=vocab)+++def test_token_ids(vocab):+ """The ids the circuit looks for are Qwen3.5's."""+ tok = vocab.tok+ one = {s: tok.encode(s, add_special_tokens=False).ids for s in+ ["\n", "=", "function", "parameter", "</", "<", ">", "ok", "not", " found", "empty",+ "<tool_response>", "</tool_response>", "<tool_call>", "</tool_call>"]}+ assert one["\n"] == [S.Q_NL] and one["="] == [S.Q_EQ] and one["function"] == [S.Q_FUNCTION]+ assert one["parameter"] == [S.Q_PARAMETER] and one["</"] == [S.Q_LTSLASH]+ assert one["<"] == [S.Q_LT] and one[">"] == [S.Q_GT]+ assert one["ok"] == S.MSG["ok"] and one["not"] + one[" found"] == S.MSG["not_found"]+ assert one["empty"] == S.MSG["empty"]+ for cls, q in S.QWEN_CLASS_IDS.items():+ assert tok.encode(B.QWEN_TEXT[cls], add_special_tokens=False).ids == [q]+ for op, q in S.OP_BARE.items():+ assert tok.encode(op, add_special_tokens=False).ids == [q]+ for op, q in S.OP_JOINED.items():+ assert tok.encode("=" + op, add_special_tokens=False).ids == [q]+ for p, q in S.PARAM_BARE.items():+ assert tok.encode(p, add_special_tokens=False).ids == [q]+ for p, q in S.PARAM_JOINED.items():+ assert tok.encode("=" + p, add_special_tokens=False).ids == [q]+ # the template spells a call the way StoreMachine.call does+ text = ("<tool_call>\n<function=put>\n<parameter=name>\nnotes\n</parameter>\n"+ "<parameter=text>\nhi there\n</parameter>\n</function>\n</tool_call>")+ sm = S.StoreMachine(vocab)+ n = len(sm.tokens)+ sm.call("put", vocab.tok_ids("notes"), vocab.tok_ids("hi there"))+ qids = [S.qid_of(t) for t in sm.tokens[n:]]+ assert qids[:qids.index(248059) + 1] == tok.encode(text, add_special_tokens=False).ids+++def answers(vocab, sm, start):+ """The texts the store printed after ``start``, one per <tool_response>."""+ out, cur = [], None+ for t, g in zip(sm.tokens[start:], sm.generated[start:]):+ q = S.qid_of(t)+ if not g or q is None:+ continue+ if q == 248066:+ cur = []+ elif q == 248067:+ out.append("".join(cur)[1:]) # (the newline after <tool_response>)+ cur = None+ elif cur is not None:+ cur.append(vocab.strings.get(q, "?"))+ return out+++class Oracle:+ """What a filesystem of named texts should answer."""++ def __init__(self):+ self.files = {} # name key -> text (in creation order: dicts keep it)++ def run(self, op, key, text):+ if op == "put":+ if key in self.files and False:+ pass+ self.files[key] = text+ return "ok\n"+ if op == "append":+ self.files[key] = self.files.get(key, "") + text+ return "ok\n"+ if op == "get":+ return self.files[key] if key in self.files else "not found\n"+ if op == "delete":+ return "ok\n" if self.files.pop(key, None) is not None else "not found\n"+ names = [k for k in self.files]+ return "".join(names) if names else "empty\n"+++def test_reference_semantics(vocab):+ """Random calls: the reference answers what a filesystem should (names keep their+ trailing newline, appends start on a new line, a re-created name goes to the end)."""+ rng = random.Random(1)+ words = ["notes", "plan", "id", "key", "todo", "a", "b", "user_7", "x/y/z"]+ for trial in range(4):+ sm = S.StoreMachine(vocab)+ orc = Oracle()+ order = [] # directory order of live names (deleted ones keep their slot; re-put+ # of a deleted name appends a new entry)+ for _ in range(30):+ op = rng.choice(["put", "put", "append", "get", "get", "delete", "list"])+ name = rng.choice(words)+ body = " ".join(rng.choice(words) for _ in range(rng.randint(0, 6)))+ start = len(sm.tokens)+ nq = vocab.tok_ids(name)+ tq = vocab.tok_ids(body) if body else []+ if op == "list":+ sm.call("list")+ elif op in ("get", "delete"):+ sm.call(op, nq)+ else:+ sm.call(op, nq, tq)+ got = answers(vocab, sm, start)+ assert len(got) == 1+ key = name + "\n"+ text = (body + "\n") if body else "\n"+ if op == "list":+ live = [k for k in order if k in orc.files]+ want = "".join(live) if order else "empty\n"+ if not order:+ assert got[0] == "empty\n"+ else:+ assert got[0] == want, (got, want)+ continue+ want = orc.run(op, key, text)+ if op in ("put", "append") and key not in order:+ order.append(key)+ if op == "delete" and want == "ok\n":+ order.remove(key)+ assert got[0] == want, (trial, op, name, got, want)+++def run_circuit(circuit, vocab, sm, chunk=None):+ n, bad, bad_owner = SC.check(circuit, vocab, sm, chunk=chunk)+ assert not bad and not bad_owner, ([(p, SC.name(w), SC.name(g), SC.name(sm.tokens[p]))+ for p, w, g in bad[:5]], bad_owner[:5])+ return n+++def test_circuit_all_ops(circuit, vocab):+ sm = S.StoreMachine(vocab)+ sm.qwen_text(vocab.tok_ids("Hello there."))+ nm = vocab.tok_ids("notes/plan")+ sm.call("list") # empty+ sm.call("get", nm) # not found+ sm.call("put", nm, vocab.tok_ids("Buy milk, then call Bob at 5pm."))+ sm.call("get", nm)+ sm.call("append", nm, vocab.tok_ids(" Also eggs."))+ sm.call("append", nm, vocab.tok_ids("and bread"))+ sm.call("get", nm)+ sm.call("put", vocab.tok_ids("id"), vocab.tok_ids("7f3a-99b2"))+ sm.call("list")+ sm.call("delete", nm)+ sm.call("get", nm)+ sm.call("list")+ sm.call("delete", nm) # not found any more+ sm.call("put", nm, []) # an empty text+ sm.call("get", nm)+ sm.qwen_text(vocab.tok_ids("Done."))+ assert run_circuit(circuit, vocab, sm) > 200+++def random_text(rng, vocab, n, plain=False):+ """n random Qwen tokens (plain: no digits, operators or tags, so nothing else fires)."""+ out = []+ while len(out) < n:+ q = rng.randrange(S.SPECIAL)+ if plain and vocab.cls(q) != B.OTHER:+ continue+ out.append(q)+ return out+++def test_circuit_random(circuit, vocab):+ """Random calls with random tokens: overwrites, chains of appends, deletes, names that+ differ only after NMAX tokens, empty and over-long texts, any token at all."""+ rng = random.Random(7)+ base = [random_text(rng, vocab, rng.randint(1, 3)) for _ in range(4)]+ names = base + [base[0] + random_text(rng, vocab, 1)] # a longer name, same start+ names += [base[1][:1] * 7 + [5], base[1][:1] * 7 + [6]] # equal in their first NMAX tokens+ for trial in range(2):+ sm = S.StoreMachine(vocab)+ sm.qwen_text(random_text(rng, vocab, 20, plain=True))+ for _ in range(14):+ op = rng.choice(["put", "append", "append", "get", "get", "delete", "list"])+ nm = rng.choice(names)+ k = rng.choice([0, 1, 5, 30, S.CMAX - 1, S.CMAX, S.CMAX + 7])+ if op == "list":+ sm.call("list")+ elif op in ("get", "delete"):+ sm.call(op, nm)+ else:+ sm.call(op, nm, random_text(rng, vocab, k))+ sm.qwen_text(random_text(rng, vocab, rng.randint(0, 8), plain=True))+ run_circuit(circuit, vocab, sm)+++def test_circuit_with_arithmetic(circuit, vocab):+ """The arithmetic still works next to the store, and text the store prints never fires it."""+ sm = S.StoreMachine(vocab)+ sm.text(3)+ sm.tool_call("mul", "37", "43")+ while not sm.halted:+ sm.step()+ nm = vocab.tok_ids("calc")+ sm.call("put", nm, vocab.tok_ids(" 1234 \\times 5678 = then 99999 + 1 ="))+ sm.call("get", nm) # the printed "=" must not start an inline call+ sm.tool_call("add", "576", "99")+ while not sm.halted:+ sm.step()+ sm.call("get", nm)+ run_circuit(circuit, vocab, sm)+++def test_store_off_is_unchanged():+ """Without the store the circuit is the plain one (pinned: paired RAM, rescheduled, fused+ exit, docs/q35-size.md)."""+ c = chip8_q35.build(TARGETS["4b"])+ # (q35-int: v16 signed / mod / gcd, v18 expressions, the tool-response tail)+ assert c.stats() == {"layers": 12, "dims": 3220, "heads": 84, "ffn_units": 28253,+ "max_units": 8640, "max_heads": 16}+ assert c.store is None+++def test_big_dims_not_subtracted(circuit):+ """The counters' 2^(26 - n) outputs are never retired by a subtraction in the MLP that writes+ their slot's next value: 1 - 2^26 rounds in fp32 (it broke arithmetic in HF at the 4B+ placement, which the IR cannot see: handovers are the emitter's)."""+ from chipformer.q35.emit import allocate++ assert circuit.big+ a = allocate(circuit, None, set(circuit.keep), circuit.target.intermediate)+ assert not [o for o in a.subtract.values() if o in circuit.big]+++def test_store_fits(vocab):+ """At most 16 chip layers on Qwen3.5-4B's and 0.8B's head shapes."""+ for t in TARGETS.values():+ assert chip8_q35.build(t, store=vocab).n_layers() <= 16++++def test_circuit_long_text(circuit, vocab):+ """The files outlive 100k tokens of other text (and every counter saturating meanwhile)."""+ rng = random.Random(3)+ sm = S.StoreMachine(vocab)+ names = [random_text(rng, vocab, 2) for _ in range(6)]+ texts = [random_text(rng, vocab, rng.randint(5, S.CMAX)) for _ in names]+ for nm, tx in zip(names, texts):+ sm.call("put", nm, tx)+ for i in range(10):+ sm.qwen_text(random_text(rng, vocab, 10_500, plain=True))+ nm = names[i % len(names)]+ if i == 4:+ sm.call("append", nm, random_text(rng, vocab, 40))+ sm.call("get", nm)+ sm.call("list")+ assert len(sm.tokens) > 100_000+ run_circuit(circuit, vocab, sm, chunk=4096)+++def test_snapshot_restore(circuit, vocab):+ """Session 2 starts from nothing but the store's heads of session 1: its gets are exact."""+ from chipformer.q35 import ir+ from chipformer.q35 import store_snapshot as snap++ s1 = S.StoreMachine(vocab)+ s1.qwen_text(vocab.tok_ids("Remember these."))+ files = {"plan": "Ship v2 on Friday, then rest.", "key": "sk-91f2-AB77-x0", "todo": "eggs"}+ for k, t in files.items():+ s1.call("put", vocab.tok_ids(k), vocab.tok_ids(t))+ s1.call("append", vocab.tok_ids("todo"), vocab.tok_ids("milk"))+ s1.call("delete", vocab.tok_ids("key"))+ carry = {}+ ir.evaluate(circuit, s1.tokens, True, carry)+ state = snap.ir_snapshot(circuit, carry)+ s2 = S.StoreMachine.resume(s1)+ s2.qwen_text(vocab.tok_ids("A new day."))+ for k in ("plan", "todo", "key"):+ s2.call("get", vocab.tok_ids(k))+ s2.call("list")+ s2.call("put", vocab.tok_ids("new"), vocab.tok_ids("fresh"))+ s2.call("list")+ got_text = answers(vocab, s2, 0)+ assert got_text[:3] == ["Ship v2 on Friday, then rest.\n", "eggs\nmilk\n", "not found\n"]+ assert got_text[5] == "plan\ntodo\nnew\n"+ pred, _owner, _ = SC._predict(circuit, vocab, s2.tokens, True, dict(state))++ def norm(t):+ t = int(t)+ return S.cls_of(t) if S.cls_of(t) in S.QWEN_CLASS_IDS and B.is_text(t) else t++ bad = [p for p, g in enumerate(s2.generated[1:])+ if g and norm(pred[p]) != norm(s2.tokens[p + 1])]+ assert not bad, bad[:5]+++def test_disk_image(circuit, vocab):+ """Replaying the disk image in a fresh session rebuilds the files exactly."""+ from chipformer.q35 import store_snapshot as snap++ s1 = S.StoreMachine(vocab)+ s1.call("put", vocab.tok_ids("a"), vocab.tok_ids("first file"))+ s1.call("put", vocab.tok_ids("b"), vocab.tok_ids("second"))+ s1.call("append", vocab.tok_ids("b"), vocab.tok_ids("more of it"))+ s1.call("delete", vocab.tok_ids("a"))+ s1.call("put", vocab.tok_ids("c"), vocab.tok_ids("third"))+ image = snap.disk_image(s1)+ s2 = S.StoreMachine(vocab)+ snap.replay(s2, image)+ n = len(s2.tokens)+ s2.call("get", vocab.tok_ids("b"))+ s2.call("list")+ assert answers(vocab, s2, n) == ["second\nmore of it\n", "b\nc\n"]+ run_circuit(circuit, vocab, s2)diff --git a/tests/test_q35_v2.py b/tests/test_q35_v2.pynew file mode 100644index 0000000..d60daf4--- /dev/null+++ b/tests/test_q35_v2.py@@ -0,0 +1,143 @@+"""v2 circuit: prints (00FD), the owner bit, and the exit that clears the chip dims."""++from pathlib import Path++import numpy as np+import pytest++from chipformer.q35 import bridge as B+from chipformer.q35 import check as Q+from chipformer.q35 import chip8_q35++ROOT = Path(__file__).resolve().parent.parent+TINY = ROOT / "dist" / "q35-tiny"+++def print_rom(digits: list[int]) -> bytes:+ return bytes([0xA2, 0x0A, 0x60, len(digits), 0x00, 0xFD, 0x12, 0x06, 0, 0] + digits)+++def print_trace(digits, before=5, after=8):+ bm = B.BridgeMachine(print_rom(digits))+ while not bm.halted:+ bm.step()+ bm.text(after)+ toks = [B.OTHER] * before + bm.tokens+ gen = [False] * before + bm.generated+ owner = [0] * before + B.owner_expected(bm)+ return toks, gen, owner++[email protected](scope="module")+def circuit():+ return chip8_q35.build()++[email protected]("digits", [[4, 8, 1, 7], [], list(range(16)), [9] * B.MAX_PRINT, [9] * 58, [3] * 60])+def test_print(circuit, digits):+ toks, gen, owner = print_trace(digits)+ n, bad, bad_owner = Q.check_bridge(circuit, toks, gen, owner)+ assert n >= 7 and not bad and not bad_owner, (bad, bad_owner)+++def test_print_text(circuit):+ toks, *_ = print_trace([4, 8, 1, 7])+ printed = [t for t in toks if B.is_text(t) and t != B.OTHER]+ assert printed == [B.NL, B.TR_OPEN, B.NL, *(B.HEX[d] for d in (4, 8, 1, 7)),+ B.NL, B.TR_CLOSE, B.NL, *B.TOOL_TAIL]++[email protected](not TINY.exists(), reason="run python -m chipformer.q35.tiny first")[email protected]("dtype", ["bfloat16", "float32"])+def test_checkpoint_print_and_exit(circuit, dtype):+ torch = pytest.importorskip("torch")+ from transformers import AutoModelForCausalLM++ from chipformer.q35 import tiny+ from chipformer.q35.emit import allocate+ from chipformer.q35.ir import Head+ from chipformer.q35.tiny import pick_layout++ model = AutoModelForCausalLM.from_pretrained(TINY, dtype=getattr(torch, dtype)).eval()+ toks, gen, _ = print_trace(list(range(12)))+ caught = []+ last = model.model.layers[circuit.n_layers() - 1]+ hook = last.register_forward_hook(lambda m, i, o: caught.append(o[0] if isinstance(o, tuple)+ else o))+ with torch.no_grad():+ logits = model(torch.tensor([toks])).logits[0].float()+ hook.remove()+ pred = logits[:, :B.TEXT_BASE + B.N_TEXT].argmax(-1).numpy()+ want = np.array(toks[1:] + [-1])+ g = np.array(gen[1:] + [False])+ assert not (g & (pred != want)).any()+ # after the exit only mu, the output code and tau remain in the chip region+ alloc = allocate(circuit, None, set(circuit.keep), tiny.BASE["intermediate_size"])+ lay = pick_layout(alloc.n_slots + 1)+ lay.slots = alloc.slots+ res = caught[0][0].float()+ kept = {lay.dim(d) for d in circuit.keep}+ # the counter's halving rows reach 2^-j: below the head's ``tiny`` (2^-90) the exit's+ # clear cannot subtract them exactly (the normed value underflows), and the IR already+ # lets such outputs round; nothing reads those chip dims after the exit+ tiny_dims = {lay.dim(d): n.tiny for n in circuit.nodes if isinstance(n, Head) and n.tiny+ for vh in n.vheads() for _, d, _ in vh.out if d in alloc.slots}+ others = [i for i in range(lay.hidden) if i not in kept and i not in tiny_dims]+ assert res[:, others].abs().max() == 0+ for i, t in tiny_dims.items():+ assert res[:, i].abs().max() < t++[email protected]("calls", [[("mul", "37", "43")], [("add", "48213", "9921"),+ ("mul", "12", "34")]])+def test_tool_call(circuit, calls):+ from chipformer.q35.lib import bcd++ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(6)+ for op, a, b in calls:+ bm.tool_call(op, a, b)+ while not bm.halted:+ bm.step()+ bm.text(5)+ n, bad, bad_owner = Q.check_bridge(circuit, bm.tokens, bm.generated, B.owner_expected(bm))+ assert n > 40 and not bad and not bad_owner, (bad, bad_owner)+++def test_bios_arithmetic():+ import random++ from chipformer.q35.lib import bcd+ from chipformer.ref.emu import apply_writes++ rng = random.Random(1)+ for _ in range(100):+ op = rng.choice(["add", "mul"])+ a, b = str(rng.randrange(10**rng.randint(1, 10))), str(rng.randrange(10**rng.randint(1, 10)))+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ apply_writes(bm.emu.cells, bcd.call_writes(op, a, b))+ while not bm.halted:+ bm.step()+ got = "".join("0123456789abcdef"[B.HEX.index(t)] for t in bm.tokens if t in B.HEX)+ assert got == str(int(a) + int(b) if op == "add" else int(a) * int(b))+++def test_bignum_macros(circuit):+ """Random multi-call sessions: BADD / BZERO / BMAC records are predicted exactly."""+ import random++ from chipformer.q35.lib import bcd++ rng = random.Random(11)+ for _ in range(6):+ bm = B.BridgeMachine(bcd.ROM, bcd.QUIRKS)+ bm.text(3)+ for _ in range(2):+ op = rng.choice(["add", "mul"])+ a, b = str(rng.randrange(10**rng.randint(1, 12))), str(rng.randrange(10**rng.randint(1, 12)))+ bm.tool_call(op, a, b)+ while not bm.halted:+ bm.step()+ bm.text(4)+ _n, bad, bad_owner = Q.check_bridge(circuit, bm.tokens, bm.generated, B.owner_expected(bm))+ assert not bad and not bad_owner, (bad, bad_owner)diff --git a/tests/test_ref.py b/tests/test_ref.pynew file mode 100644index 0000000..4be6e33--- /dev/null+++ b/tests/test_ref.py@@ -0,0 +1,161 @@+import random+from pathlib import Path++import pytest++from chipformer import spec+from chipformer.ref.emu import Chip8+from chipformer.spec import DISPLAY, KEYS, PC, SP, STACK, I, V+from chipformer.trace import Machine, parse, replay++ROMS = Path(__file__).resolve().parent.parent / "roms"+++def prog(*words: int) -> bytes:+ return b"".join(w.to_bytes(2, "big") for w in words)+++def run(rom: bytes, steps: int, quirks="chip48") -> Chip8:+ emu = Chip8(rom, quirks)+ for _ in range(steps):+ emu.step()+ return emu+++def test_quirk_shift():+ code = prog(0x6003, 0x6181, 0x8016) # V0=3, V1=0x81, V0 >>= (V1 or V0)+ assert run(code, 3, "shift").cells[V:V + 2] == [1, 0x81]+ emu = run(code, 3, "octo")+ assert emu.cells[V] == 0x40 and emu.cells[V + 0xF] == 1+++def test_quirk_loadstore():+ code = prog(0xA300, 0xF255)+ assert run(code, 2, "loadstore").cells[I] == 0x300+ assert run(code, 2, "octo").cells[I] == 0x303+++def test_quirk_clip_vs_wrap():+ code = prog(0x603E, 0x611E, 0xA050, 0xD015) # font '0' at x=62, y=30+ clipped = run(code, 4, "clip").cells+ wrapped = run(code, 4, "octo").cells+ assert clipped[DISPLAY + 0] == 0 and clipped[DISPLAY + 30 * 8 + 7] == 0x03+ assert wrapped[DISPLAY + 30 * 8 + 7] == 0x03 and wrapped[DISPLAY + 30 * 8] == 0xC0 # wraps right+ assert wrapped[DISPLAY + 0 * 8 + 7] == 0x02 # row 32 wraps to row 0+++def test_quirk_logic_jump_vforder():+ assert run(prog(0x6F07, 0x8011), 2, "logic").cells[V + 0xF] == 0+ assert run(prog(0x6F07, 0x8011), 2, "octo").cells[V + 0xF] == 7+ assert run(prog(0x6104, 0xB120), 2, "jump").cells[PC] == 0x124 # B1NN: 0x120 + V1+ assert run(prog(0x6104, 0xB120), 2, "octo").cells[PC] == 0x120 # BNNN: 0x120 + V0+ code = prog(0x6FFF, 0x6102, 0x8F14) # VF = 0xFF + 2: result 1, carry 1+ assert run(code, 3, "octo").cells[V + 0xF] == 1 # flag written last+ assert run(code, 3, "vforder").cells[V + 0xF] == 1 # result (0x01) written last+ code = prog(0x6FFF, 0x6103, 0x8F14) # result 2, carry 1+ assert run(code, 3, "octo").cells[V + 0xF] == 1+ assert run(code, 3, "vforder").cells[V + 0xF] == 2+++def test_lfsr_is_maximal():+ s, seen = spec.RNG_SEED, set()+ while s not in seen:+ seen.add(s)+ s = spec.lfsr_step(s)+ assert len(seen) == 4095 and 0 not in seen+++def test_vocab_disjoint():+ assert spec.addr_token(0) == spec.val_token(spec.MAX_VALUE) + 1+ assert spec.STEP == spec.addr_token(spec.NUM_CELLS - 1) + 1+ assert spec.VOCAB_SIZE == spec.BOOT + 1+++def test_alu_and_flags():+ emu = run(prog(0x60FF, 0x6102, 0x8014), 3) # V0=FF, V1=2, V0+=V1+ assert emu.cells[V + 0] == 0x01 and emu.cells[V + 0xF] == 1+ emu = run(prog(0x6005, 0x6107, 0x8015), 3) # 5-7 borrows+ assert emu.cells[V + 0] == 0xFE and emu.cells[V + 0xF] == 0+ emu = run(prog(0x6F05, 0x6107, 0x8F15), 3) # flag wins when X=F+ assert emu.cells[V + 0xF] == 0+ emu = run(prog(0x6081, 0x800E), 2)+ assert emu.cells[V + 0] == 0x02 and emu.cells[V + 0xF] == 1+++def test_call_return_and_skips():+ # 200: call 206 | 202: 6A01 | 204: jump 204 | 206: 3000 (skip) | 208: 6B01 | 20A: ret+ emu = run(prog(0x2206, 0x6A01, 0x1204, 0x3000, 0x6B01, 0x00EE), 3) # call, skip, ret+ assert emu.cells[PC] == 0x202 and emu.cells[SP] == 0 and emu.cells[STACK] == 0x202+ assert emu.cells[V + 0xB] == 0+ emu.step()+ assert emu.cells[V + 0xA] == 1+++def test_bcd_store_load():+ emu = run(prog(0x60EA, 0xA300, 0xF033, 0xF265), 4) # 234 -> 2,3,4+ assert emu.cells[0x300:0x303] == [2, 3, 4]+ assert emu.cells[V:V + 3] == [2, 3, 4] and emu.cells[I] == 0x300+++def test_draw_straddle_collision_and_clip():+ # draw font '0' at x=4,y=0 twice -> second draw erases and collides+ emu = run(prog(0x6004, 0x6100, 0xF129,0xD015, 0xD015), 4)+ assert emu.cells[V + 0xF] == 0+ assert emu.cells[DISPLAY] == 0x0F and emu.cells[DISPLAY + 1] == 0x00 # F0 >> 4+ emu.step()+ assert emu.cells[V + 0xF] == 1 and not any(emu.cells[DISPLAY:DISPLAY + 256])+ # clipped at bottom-right: x=62, y=30, 5 rows -> only 2 rows, 2 pixels each visible+ emu = run(prog(0x603E, 0x611E, 0xA050, 0xD015), 4)+ assert emu.cells[DISPLAY + 30 * 8 + 7] == 0x03 and emu.cells[DISPLAY + 31 * 8 + 7] == 0x02+++def test_keys_and_wait():+ emu = Chip8(prog(0xF30A, 0x1202))+ emu.step()+ assert emu.cells[PC] == 0x200 # no key: stays+ emu.set_key(7, True)+ emu.step()+ assert emu.cells[PC] == 0x202 and emu.cells[V + 3] == 7+ assert emu.cells[KEYS + 7] == 1+++def random_trace(rom: bytes, steps: int, seed: int = 0) -> Machine:+ rng = random.Random(seed)+ m = Machine(rom)+ for i in range(steps):+ if i % 10 == 0:+ m.tick()+ m.set_key(rng.randrange(16), rng.random() < 0.5)+ m.step()+ return m+++def test_trace_replay_matches_emulator():+ rng = random.Random(1)+ rom = bytes(rng.randrange(256) for _ in range(1024)) # random code exercises every opcode+ m = random_trace(rom, 3000)+ parse(m.tokens)+ assert replay(m.tokens) == m.emu.cells+++def test_trace_replay_every_record():+ m = Machine(prog(0x6004, 0x6100, 0xA050, 0xD015, 0x7001, 0x1206))+ for _ in range(50):+ m.step()+ assert replay(m.tokens) == m.emu.cells+++def test_boot_prompt_restores_state():+ m = random_trace(bytes(random.Random(2).randrange(256) for _ in range(512)), 500)+ restart = Machine(b"")+ from chipformer.trace import boot_tokens++ assert replay(boot_tokens(m.emu.cells)) == m.emu.cells+ assert restart.emu.cells != m.emu.cells++[email protected]("rom", sorted((ROMS / "test").glob("*.ch8")) if (ROMS / "test").exists()+ else [], ids=lambda p: p.name)+def test_rom_trace_roundtrip(rom: Path):+ m = random_trace(rom.read_bytes(), 20_000)+ assert replay(m.tokens) == m.emu.cellsdiff --git a/uv.lock b/uv.locknew file mode 100644index 0000000..0f81384--- /dev/null+++ b/uv.lock@@ -0,0 +1,1388 @@+version = 1+revision = 3+requires-python = ">=3.11"+resolution-markers = [+ "python_full_version >= '3.12'",+ "python_full_version < '3.12'",+]++[[package]]+name = "annotated-doc"+version = "0.0.5"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/5a/8e/38aa427ed5402449e226975b649c5dc73ccadfefeb95e6aecb8f8ea4b6b6/annotated_doc-0.0.5.tar.gz", hash = "sha256:c7e58ce09192557605d8bbd92836d7e1d520ac9580096042c0bfd197efacf1bb", size = 10758, upload-time = "2026-07-28T13:50:58.129Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/3e/30/e900b21425a860e195f32e37657aa1f7c7f2b1bfb26f03ca209b90933c06/annotated_doc-0.0.5-py3-none-any.whl", hash = "sha256:117bac03a25ede5df5440e855b32d556049ca169ead221505badf432fed4b101", size = 5302, upload-time = "2026-07-28T13:50:57.239Z" },+]++[[package]]+name = "anyio"+version = "4.15.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "idna" },+ { name = "typing-extensions", marker = "python_full_version < '3.15'" },+]+sdist = { url = "https://files.pythonhosted.org/packages/a9/d2/f4d173e22df740bc37b1db102b386ba719b66e95b0f0d751f556b387e6d2/anyio-4.15.1.tar.gz", hash = "sha256:9f28306018cbd6d329e64a36d58256edff76dd996fe423bc957326e578b82a94", size = 276966, upload-time = "2026-09-05T10:42:39.44Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/12/b8/4bd346e22b28902df4d651910f5242c28d84e4a5c2435ca5c3f797ed7e2e/anyio-4.15.1-py3-none-any.whl", hash = "sha256:6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101", size = 132079, upload-time = "2026-09-05T10:42:37.923Z" },+]++[[package]]+name = "chipformer"+version = "0.1.0"+source = { editable = "." }+dependencies = [+ { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },+ { name = "numpy", version = "2.5.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },+ { name = "safetensors" },+]++[package.optional-dependencies]+hf = [+ { name = "torch" },+ { name = "transformers" },+]+play = [+ { name = "pygame" },+]++[package.dev-dependencies]+dev = [+ { name = "pillow" },+ { name = "pytest" },+ { name = "ruff" },+]++[package.metadata]+requires-dist = [+ { name = "numpy", specifier = ">=1.26" },+ { name = "pygame", marker = "extra == 'play'", specifier = ">=2.5" },+ { name = "safetensors", specifier = ">=0.8.0" },+ { name = "torch", marker = "extra == 'hf'", specifier = ">=2.14.1" },+ { name = "transformers", marker = "extra == 'hf'", specifier = ">=5.19.0" },+]+provides-extras = ["play", "hf"]++[package.metadata.requires-dev]+dev = [+ { name = "pillow", specifier = ">=12.3.0" },+ { name = "pytest", specifier = ">=8" },+ { name = "ruff", specifier = ">=0.6" },+]++[[package]]+name = "click"+version = "8.5.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/c7/0e/7fa0ef50764b67090eca4114772a2abf8b6148198475e54c660b97caeee6/click-8.5.0.tar.gz", hash = "sha256:ba0d2089de75ea0310e2dde03160e6ca10009947fb95a182f9b54021bb272e34", size = 382235, upload-time = "2026-08-26T13:33:14.56Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/58/50/6c0d534c5f134586a8e1ba4e330569e32f057e33372ae556463212fb4cd3/click-8.5.0-py3-none-any.whl", hash = "sha256:255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360", size = 125251, upload-time = "2026-08-26T13:33:12.928Z" },+]++[[package]]+name = "colorama"+version = "0.4.6"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },+]++[[package]]+name = "cuda-bindings"+version = "13.4.4"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "cuda-pathfinder" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/a1/15/0277326dd1a2d619ea8a85b790fc9e43cb67897d3a3377eb783f5bc2a33a/cuda_bindings-13.4.4-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b8b2e4235b552edb42cdbe39de7a9259ceec631e21c7344b76177db9cc6c58b4", size = 6494096, upload-time = "2026-10-14T12:35:11.314Z" },+ { url = "https://files.pythonhosted.org/packages/97/36/3bfb7b74817a0a78f80340fae742a27d166e8fcdffa459b79d5be66e077a/cuda_bindings-13.4.4-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e1cef1eaf5a08cc3bf3675ca9d3f97f3fc7085998556c848b4053e74ba4ba392", size = 7160763, upload-time = "2026-10-14T12:35:13.773Z" },+ { url = "https://files.pythonhosted.org/packages/d7/38/87909f62598d02d9e5c536326aed1755f6bfa190285507b962bb601e0005/cuda_bindings-13.4.4-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b346bfe1dda49537537c06cc4727b2210d949d8aee883cb591bca46581629d00", size = 6488222, upload-time = "2026-10-14T12:35:20.703Z" },+ { url = "https://files.pythonhosted.org/packages/c2/f8/332443cf969734551891cdbd93657eaeaf747ac2b7dd73feb3422701b02b/cuda_bindings-13.4.4-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:433efa31e33c2868621b5f6036bd3bf5de9c45c5546cb1361a5f6ff3415b8bb5", size = 7172184, upload-time = "2026-10-14T12:35:22.817Z" },+ { url = "https://files.pythonhosted.org/packages/a7/97/7d75105a92ee18b7764f5cf60672d6a966842348a2a049e663fe690902ff/cuda_bindings-13.4.4-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0c3f4cc887eac0d5edb5d7396cc78b336bee151a97e2ba7acf59e8b9380132d0", size = 6316532, upload-time = "2026-10-14T12:35:30.291Z" },+ { url = "https://files.pythonhosted.org/packages/ba/8f/aedec367fea64267594a26ede3704f241e1bee465962f3a92d6186c774b7/cuda_bindings-13.4.4-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5d6adc44f098472d8ac2d6211923eea6081830521852dc77f8c9237bf9320a59", size = 6929120, upload-time = "2026-10-14T12:35:32.508Z" },+ { url = "https://files.pythonhosted.org/packages/e6/f0/d71d1dd45caa358a39c6898dcc9f1bd22a9f22b92376b967835c047b50bc/cuda_bindings-13.4.4-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4a429132f06d0c3a853aa419aabe4c0a7bfc3f3886fb4c336d81729f63754d31", size = 6407946, upload-time = "2026-10-14T12:35:39.826Z" },+ { url = "https://files.pythonhosted.org/packages/15/65/0b8f2f2509e5f1a8e556a2e083ac0287fe4af79afdba3153454229306f39/cuda_bindings-13.4.4-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:680e70dcb8b61fdae16d3bb4876c41d55aadc9fe8b3825ea567f2f2c85606b0a", size = 6978954, upload-time = "2026-10-14T12:35:42.892Z" },+ { url = "https://files.pythonhosted.org/packages/f1/dd/81903887199ffd90f9ee29eee8ecd497b734007e78b80689e0dd2f904bd8/cuda_bindings-13.4.4-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:03a8fd2a25530dbae26ecd4462d418156028e35b3f3b081440ff6368506e10eb", size = 6314269, upload-time = "2026-10-14T12:35:50.607Z" },+ { url = "https://files.pythonhosted.org/packages/90/44/f6325a6061f1fc619e216dd6c7cbd9a0b475d3937bb32296c84cc27e6eeb/cuda_bindings-13.4.4-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0ce9208c7cf03a947ac11553b08ee2fca594ac63d34360639a9060980f546a62", size = 6865978, upload-time = "2026-10-14T12:35:52.874Z" },+ { url = "https://files.pythonhosted.org/packages/c2/26/00d628420ac318a6a6d559fe6602e08a0156e7f4b5bf9dc6ea20c63789bc/cuda_bindings-13.4.4-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:50bc70ef3b225f4794f5b374773629836bbfdf4a21fc975ac25e7daad67494b8", size = 6414169, upload-time = "2026-10-14T14:12:29.952Z" },+ { url = "https://files.pythonhosted.org/packages/63/88/5b50b13cef9ae8baadcca03a47cf0e9146c0b52ca6aeabaf50fc17e18445/cuda_bindings-13.4.4-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6781fe08ed5f4b5163a3c6577e079bc28955811382d35995c916bf052ee0994e", size = 7012626, upload-time = "2026-10-14T14:12:32.224Z" },+ { url = "https://files.pythonhosted.org/packages/63/cb/013d9fd501dc1b05c3f467f28ad3adeed9e4d39318a4668a7a75fc5b19da/cuda_bindings-13.4.4-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a4c97f8430d7f7c4a4d0f245510d26647dd81ea3fc65941f7a815ad395cbc30e", size = 6302110, upload-time = "2026-10-14T14:12:39.47Z" },+ { url = "https://files.pythonhosted.org/packages/c8/5d/ca0d5c1d9c09a9de15f474c1fa6abe801761e75cdc1be9a03431063f2af5/cuda_bindings-13.4.4-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:baa0f5f1f9a05032e7deb97051e85aa18b4e3c6db914f1c813bb2f02e5c72db4", size = 6878099, upload-time = "2026-10-14T14:12:41.504Z" },+]++[[package]]+name = "cuda-pathfinder"+version = "1.8.3"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b9/fb/f8e1890428f9f590b4beebd63b068aac1ce32a3331510c847b9f9a78f261/cuda_pathfinder-1.8.3-py3-none-any.whl", hash = "sha256:e29e59829c297a7a5233bd9cc71094fc5bddbd076951482670178f9eade39b1f", size = 62561, upload-time = "2026-10-02T03:20:23.712Z" },+]++[[package]]+name = "cuda-toolkit"+version = "13.0.3.0"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d1/c7/a79086a62c98befcdb8349656c6f114e2db3b8b2422f6e25c97a7f2a9a3c/cuda_toolkit-13.0.3.0-py2.py3-none-any.whl", hash = "sha256:d693caaa261214ddd7dbb60d68e71cbed884e68c2be7509778f3051da0b91c3f", size = 2512, upload-time = "2026-04-14T00:50:08.173Z" },+]++[package.optional-dependencies]+cublas = [+ { name = "nvidia-cublas", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-cuda-nvrtc", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cudart = [+ { name = "nvidia-cuda-runtime", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cufft = [+ { name = "nvidia-cufft", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-nvjitlink", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cufile = [+ { name = "nvidia-cufile", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cupti = [+ { name = "nvidia-cuda-cupti", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+curand = [+ { name = "nvidia-curand", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cusolver = [+ { name = "nvidia-cublas", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-cusolver", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-cusparse", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-nvjitlink", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+cusparse = [+ { name = "nvidia-cusparse", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+ { name = "nvidia-nvjitlink", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+nvjitlink = [+ { name = "nvidia-nvjitlink", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+nvrtc = [+ { name = "nvidia-cuda-nvrtc", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]+nvtx = [+ { name = "nvidia-nvtx", marker = "platform_machine == 'aarch64' or platform_machine == 'x86_64'" },+]++[[package]]+name = "filelock"+version = "4.2.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/f4/a9/1af41b37c3279712b22cdc63aac78a52432202b6fe1f9666a2a3d2831fb4/filelock-4.2.0.tar.gz", hash = "sha256:7a60906c75227cf04d0c273afadc8219400f11aeb13cc69591d4f6cdc6c8036e", size = 569667, upload-time = "2026-10-14T20:57:13.11Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/8e/a3/9bc26acff301fe1aaea1cc3d82a1d57e0a34df3e1cadbfa91ac2dbcdde5c/filelock-4.2.0-py3-none-any.whl", hash = "sha256:2ff5690882e8cdb00ef31fb3d01a3094c29f30985426c59495afb1733f3b7238", size = 135032, upload-time = "2026-10-14T20:57:11.349Z" },+]++[[package]]+name = "fsspec"+version = "2026.9.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/77/cd/9be253869fc42e764de7f3dedd6969af7d44ff9c3375214a3442a6f3fc08/fsspec-2026.9.0.tar.gz", hash = "sha256:0f08147951c8cb31d844c3547d631053b127863b60be04cf06e121333ee0e2fe", size = 333545, upload-time = "2026-09-18T17:50:42.825Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/6c/c0/a98505f18594f1bce828bb159cec0fcf9860562f1a2c85913409fc8f3d9e/fsspec-2026.9.0-py3-none-any.whl", hash = "sha256:8dd6e646e99ea382bd85f97a45e6b526a442d79423a7dc673f1e2756d05fcb5f", size = 221738, upload-time = "2026-09-18T17:50:41.341Z" },+]++[[package]]+name = "h11"+version = "0.16.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },+]++[[package]]+name = "hf-xet"+version = "1.7.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/9e/27/06d899ea7bd721d272f84aac98bdb238de98af4cc767a69056d967d68c71/hf_xet-1.7.0.tar.gz", hash = "sha256:d406ec79053c0871817f700c2ac8c36ba0d87f9c34b7458b0f0063bb218b0466", size = 985689, upload-time = "2026-10-06T20:18:43.89Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/9f/7c/3e45174942e6793adde6cba4daa7fb037275cf02a944d9eadfcf9ff33b86/hf_xet-1.7.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:fa029678be1ba7f953c409b0b27bf15cc69cd1c9b3a674fbd78856ebefca1052", size = 3803919, upload-time = "2026-10-06T20:18:09.844Z" },+ { url = "https://files.pythonhosted.org/packages/ff/3a/5e8b363391adcbb002e191dbf924dab31464ea9c45adfeb73502afc36d35/hf_xet-1.7.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:57bc157b8b7fe3bee9dcb9af7f3da8de41801c3b31a9ef68a77a33c6a6be382f", size = 3553588, upload-time = "2026-10-06T20:18:13.376Z" },+ { url = "https://files.pythonhosted.org/packages/e5/c2/0d1eaa5da13bbf9c896badc7f380601c7d973a87a6ffb4d100267c4536c1/hf_xet-1.7.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:87dab080f8f7d32781c2586904e3603f4e60d09bfc727706c3ae419e0829beeb", size = 4201962, upload-time = "2026-10-06T20:18:16.11Z" },+ { url = "https://files.pythonhosted.org/packages/23/2d/225d5b11a9ca7d31b9470a57f2b2be1a5cef8b84325a2146aeb4589e226c/hf_xet-1.7.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:b01fe18dbbd151a2403d2c64ed30dc6547b00d6babab9a617d77c7acdb81ee66", size = 3982978, upload-time = "2026-10-06T20:18:18.092Z" },+ { url = "https://files.pythonhosted.org/packages/93/34/9d681f0e3dac0b5dae0d7dea748429266f24e52415446523f464fbaa828e/hf_xet-1.7.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:4ee5e05a627f5ab5bad7a86582277d645556ea1e199903aae19e033a392aa13a", size = 4181558, upload-time = "2026-10-06T20:18:20.082Z" },+ { url = "https://files.pythonhosted.org/packages/de/f0/277f039b7d72027bc2ed277f1b62a2f70f740a5aac2a3e7243e5b6854c5d/hf_xet-1.7.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:19c0e64f14175ccb6a1aff69e0d2ab9ec5269a560e6687abaf2b3fa4f73de7cd", size = 4411546, upload-time = "2026-10-06T20:18:21.999Z" },+ { url = "https://files.pythonhosted.org/packages/3d/7f/832d3ddb49326114175b7bcc50daea8565c09fd21ac03a02b211c09fefb7/hf_xet-1.7.0-cp314-cp314t-win_amd64.whl", hash = "sha256:757168feb5679647c0bb13ee5d0faebe799c4dff9051419885a566ebd79f949d", size = 3812809, upload-time = "2026-10-06T20:18:24.288Z" },+ { url = "https://files.pythonhosted.org/packages/3d/c4/310c3c29e5beae7c049e63947bd1923d597883b41c9ec4718589920812c4/hf_xet-1.7.0-cp314-cp314t-win_arm64.whl", hash = "sha256:b91569d5f1b61c34b043687da02c05dd3604f3d329e7868510bf3f7971599006", size = 3646174, upload-time = "2026-10-06T20:18:26.279Z" },+ { url = "https://files.pythonhosted.org/packages/9c/0b/b03be21ffaada749ba0d3197d8aefbf1aa698bac149580421c15239b299e/hf_xet-1.7.0-cp38-abi3-macosx_10_12_x86_64.whl", hash = "sha256:e3e88a7a75d7d95cbee1f37dc31341d6201124cf21c6c4b1dfab8ccba9b09e0f", size = 3796096, upload-time = "2026-10-06T20:18:28.43Z" },+ { url = "https://files.pythonhosted.org/packages/c3/47/a26ebdce7056a61e931f228439bc0ab08cbec239d1690f965e5e637cba79/hf_xet-1.7.0-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:59fba37039233c7fcbe196817d6cdcf1b40dfb17b410f229d85b0cf0a1848da4", size = 3560352, upload-time = "2026-10-06T20:18:30.365Z" },+ { url = "https://files.pythonhosted.org/packages/a3/4c/2bf3b66c215d409655f28de1622393dde04c9461280d48c7924bb3b2decd/hf_xet-1.7.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:2814a6e999d13464c4d679b788cc5d784eb5a4edfc638a31f10e9a11ab531ef8", size = 4212180, upload-time = "2026-10-06T20:18:32.292Z" },+ { url = "https://files.pythonhosted.org/packages/49/0c/a2f703a5a78267556e89e03316fa0805c86b72b50829bc67665746e8ebf0/hf_xet-1.7.0-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:fcfd6c22418e57dd5b3aea649e813b2e2cfb2aebf317b210d90f1fe4b3018b52", size = 3990011, upload-time = "2026-10-06T20:18:34.21Z" },+ { url = "https://files.pythonhosted.org/packages/a4/77/e52e4201b1cbf571530a61cc57f70182045a39a230089ee5f1df182a4de2/hf_xet-1.7.0-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:80f79dae613ce9e0ea1fd1ae15616ca9ac74aed4c770aabc199c4f03ebecc863", size = 4190628, upload-time = "2026-10-06T20:18:36.062Z" },+ { url = "https://files.pythonhosted.org/packages/6c/dc/03a21b89f118664a0926ff25b0f8e44a519bf22724a6a8fc7a9abbc188b6/hf_xet-1.7.0-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:0a9e802f33bf50c851abe45fc5380e61f959e2d369647d6742b79ad9d6c27cab", size = 4418814, upload-time = "2026-10-06T20:18:37.888Z" },+ { url = "https://files.pythonhosted.org/packages/4d/59/b35106dfa71b6eef605dc88bd038fe99c7f86fb132a15b60d0bf2f235b2c/hf_xet-1.7.0-cp38-abi3-win_amd64.whl", hash = "sha256:2b7bb5727889b0f2436dbaaad8fc4c3e66b8240d992716989e0c086b4278b1bc", size = 3822644, upload-time = "2026-10-06T20:18:40.052Z" },+ { url = "https://files.pythonhosted.org/packages/48/cd/072313585f74fe9d441e2eb5e0a4703c30586cd709810ea369675f61b74e/hf_xet-1.7.0-cp38-abi3-win_arm64.whl", hash = "sha256:acc3851cf2576a8fb2ae926da863f4efabe21303cf292e9a44332802ab0dcc6a", size = 3662436, upload-time = "2026-10-06T20:18:42.205Z" },+]++[[package]]+name = "httpcore2"+version = "2.13.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "h11" },+ { name = "truststore" },+]+sdist = { url = "https://files.pythonhosted.org/packages/cb/f3/1db7aa2bc2524062192bb0e0323969492d1883152a232fe36eea65f4e35c/httpcore2-2.13.1.tar.gz", hash = "sha256:e0aa977abe17e69a3b820a24542a6fa88702676d83880b8d194dcd18408e5103", size = 68071, upload-time = "2026-09-23T07:47:22.372Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/09/ba/a4568248771ce81957bfb7cc600264a40fbcda092391ee1c415c50be4bea/httpcore2-2.13.1-py3-none-any.whl", hash = "sha256:e1e05d4f25f7d7d496bfb96748f6f4b67657b03da069b3a68c36069f3db73d0a", size = 83423, upload-time = "2026-09-23T07:47:19.365Z" },+]++[[package]]+name = "httpx2"+version = "2.13.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "anyio", marker = "sys_platform != 'emscripten'" },+ { name = "httpcore2", marker = "sys_platform != 'emscripten'" },+ { name = "httpx2-jsfetch", marker = "python_full_version >= '3.12' and sys_platform == 'emscripten'" },+ { name = "idna" },+ { name = "truststore", marker = "sys_platform != 'emscripten'" },+ { name = "typing-extensions", marker = "python_full_version < '3.13'" },+]+sdist = { url = "https://files.pythonhosted.org/packages/d5/44/474bef2a0e9d90f1715d32cb98b0738695ca17ba324095fb2497ed7fbd59/httpx2-2.13.1.tar.gz", hash = "sha256:e48744a19e3af5ee48313d0ce5fe941d5422fae5705ea922a4aabf94d7800dfa", size = 100405, upload-time = "2026-09-23T07:47:23.052Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d8/9c/6fe8931fd9f381042a9e4c7d5a7b4cbf7016b252bec0c99a49fce42c3326/httpx2-2.13.1-py3-none-any.whl", hash = "sha256:6dff50fabc270ee5fd25d845d0b078ed20564579744d6d962850975996d2f9a4", size = 95597, upload-time = "2026-09-23T07:47:20.995Z" },+]++[[package]]+name = "httpx2-jsfetch"+version = "1.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/cd/c4/0e5636363151a2a1795e0a77617168b9ca438e1748ec05fc9b5687f93d64/httpx2_jsfetch-1.0.tar.gz", hash = "sha256:70a0e3eabfef7cce5ad9c629f7d01ca05e418f586646f4ddf14782e4c1454c60", size = 6872, upload-time = "2026-08-07T00:13:07.492Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/9b/43/832f631d32e4f1211caa2ba368317739fe71f0b8530e4c9d15dc454bac2a/httpx2_jsfetch-1.0-py3-none-any.whl", hash = "sha256:cb916b707601e69a07721aabc8f3f6659be3a6893bc1ff5c6f9e02241df2da32", size = 6382, upload-time = "2026-08-07T00:13:06.567Z" },+]++[[package]]+name = "huggingface-hub"+version = "2.2.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "click" },+ { name = "filelock" },+ { name = "fsspec" },+ { name = "hf-xet", marker = "platform_machine == 'AMD64' or platform_machine == 'ARM64' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'" },+ { name = "httpx2" },+ { name = "packaging" },+ { name = "pyyaml" },+ { name = "tqdm" },+ { name = "typing-extensions" },+]+sdist = { url = "https://files.pythonhosted.org/packages/12/47/6858d63643e66fb4f6585c3cfd4029c0b2bc1ae21688cee9b3335f20a10d/huggingface_hub-2.2.0.tar.gz", hash = "sha256:5d1b47537394e4215cb858aa12fd493d0f7ef7f58990f5dcd24bc173107b2871", size = 1041026, upload-time = "2026-10-08T15:30:59.971Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/eb/b0/0f7b430fd100b3a3b037fdbb314878200241082e607b3383c63d91a13a72/huggingface_hub-2.2.0-py3-none-any.whl", hash = "sha256:1667f145dc56dc210d60966069397df9ecfca9607a5d43db88b308c89dae56b3", size = 839884, upload-time = "2026-10-08T15:30:57.914Z" },+]++[[package]]+name = "idna"+version = "3.20"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/f5/08/8eea9d4b8302028f3abb2c0813953f7aec26d33b7a8960ed760e65ff29fa/idna-3.20.tar.gz", hash = "sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44", size = 216463, upload-time = "2026-09-17T14:11:04.752Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/58/a2/bb081bab032533a855d44de1d56f8e8426114ff1ba5d1f07a438a0a654f8/idna-3.20-py3-none-any.whl", hash = "sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c", size = 69583, upload-time = "2026-09-17T14:11:03.168Z" },+]++[[package]]+name = "iniconfig"+version = "2.3.1"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/01/e1/2069291243c926a2ff1cd706c7f3eeb9b62144bf60f77c9fb9ff2fb26bd3/iniconfig-2.3.1.tar.gz", hash = "sha256:67f4b9c50da0dedf52af349e7749a80a9057a5031199791b906c3bb3ae878960", size = 21209, upload-time = "2026-10-06T22:48:38.076Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/56/43/4ca9e49d27a1fcf6bece6f6aec0ea46bb9112489b93d4b688fb415457bdb/iniconfig-2.3.1-py3-none-any.whl", hash = "sha256:9121e2c1fdb355232495be3194c8dfe87ccc2d5dee45947b78e68f499790d7a7", size = 7552, upload-time = "2026-10-06T22:48:36.959Z" },+]++[[package]]+name = "jinja2"+version = "3.1.6"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "markupsafe" },+]+sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },+]++[[package]]+name = "markdown-it-py"+version = "4.2.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "mdurl" },+]+sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" },+]++[[package]]+name = "markupsafe"+version = "3.0.4"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/38/9b/e422a865e1d5d57d0e509b4e0bf1c1a70a7f6382c29a5aa428df994c8bc8/markupsafe-3.0.4.tar.gz", hash = "sha256:2e9ad7dd851bf45fab9f75cbff4cb493fee9979e8d8c7c9c3ee119022518edd6", size = 153777, upload-time = "2026-10-02T23:07:22.29Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/32/55/18dbb4778b30ada5ce071608503cc3edc9e14e13d868c17a6d178fc30f7a/markupsafe-3.0.4-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:9e25feb9e330b63edb0278a0acdf85e50d0cb0fbf49c3084abbe4e24ae195346", size = 11520, upload-time = "2026-10-02T23:04:36.299Z" },+ { url = "https://files.pythonhosted.org/packages/6c/14/0b05f79b4733e264a18d08fe08fa1df7347630ff32a6cb82180d9dccec55/markupsafe-3.0.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:7d3391b2188d18737cb2fa147028b1096236eaa7e156446c650a489fa2cadc91", size = 12063, upload-time = "2026-10-02T23:04:37.291Z" },+ { url = "https://files.pythonhosted.org/packages/ca/3a/63ba10b6c1463216b3e4df669a9f0e5a3b0c3071557d2e8229e3968c79fb/markupsafe-3.0.4-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:849dd2bb0e5e4ab2b71c7191726a4a8d5aa8a610daa584728cbee0b710ddc4ef", size = 24321, upload-time = "2026-10-02T23:04:38.262Z" },+ { url = "https://files.pythonhosted.org/packages/1a/2e/5f015261b76ad633d187ef6f388b413aedd64a8773c4df59e530a0be5525/markupsafe-3.0.4-cp311-cp311-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:befb4158af32106b9a93db8d6d1d1cbbd418c0d5aca0cabb7b1780abf0c89169", size = 28845, upload-time = "2026-10-02T23:04:39.295Z" },+ { url = "https://files.pythonhosted.org/packages/33/cf/26e594b26be40c2f1fec63ccf8a8b99d0335a5b2cbe84835c7d82a994375/markupsafe-3.0.4-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:71f88e749ea29f67f21f3b36433c1dc54c7729ed2a6d9e2da2e0d9e0d7b224eb", size = 24701, upload-time = "2026-10-02T23:04:40.348Z" },+ { url = "https://files.pythonhosted.org/packages/81/a5/a513b76c139a3915b43404324e55c0b7979ae4f0d39eb6f075b0282e90a8/markupsafe-3.0.4-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6da83a088f8ef93b2d483a8232a4dbf4d69d3d8496b568a03c56becac43e1808", size = 22961, upload-time = "2026-10-02T23:04:41.372Z" },+ { url = "https://files.pythonhosted.org/packages/46/cf/4c66192c100b4542bcbe392ae06696b670f66927be3ac38a213234778ff9/markupsafe-3.0.4-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:8f0fac8b13d14bb06c68195f849371924ae53dd7b1c00fed24650f704383b692", size = 21920, upload-time = "2026-10-02T23:04:42.429Z" },+ { url = "https://files.pythonhosted.org/packages/cb/17/ac3662678bfbad649893117ada2ba44dc30bf56884e84f13154a792b10f1/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:4a7cdc2a420ca01058182da4253329764d4bfa055564d1eced90e6ba1e8b1d3d", size = 23726, upload-time = "2026-10-02T23:04:43.531Z" },+ { url = "https://files.pythonhosted.org/packages/f7/af/fe47cee339180a69ebca3c57fb3483d0f5cbd1e8337d1871fb1d9c1aebee/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:83b3944fea42a8400edf92fd1770fb8d0d4f7de651353bd2d8525a92dba69a21", size = 22539, upload-time = "2026-10-02T23:04:44.618Z" },+ { url = "https://files.pythonhosted.org/packages/12/32/d55440ba140442800e02d799c9cb5ab597bf6ebdb1177b5ea39a11f797bd/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:8138eb83940ec7299024d92d4dee45f601b9e6c5ffde9d25f4e35e326203c707", size = 23963, upload-time = "2026-10-02T23:04:45.656Z" },+ { url = "https://files.pythonhosted.org/packages/50/9d/9c86042cb364c2ad4c971e6d1247929effd25f714cd7ee11b05e6316445b/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:811d02d5122171c1941357efd8f9bf4ffe907b7f0a1a4e729a880e4be3f46e3e", size = 21506, upload-time = "2026-10-02T23:04:46.671Z" },+ { url = "https://files.pythonhosted.org/packages/75/ef/5b824f03ba40c3b3652b6272d083d2fc4fcdd440de519a3a39ba2c3e7262/markupsafe-3.0.4-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:50b5bedc9ed8a94fc8857a42ef4f84a81ea88f8d4f05dc8705fb23ee6d8dcca7", size = 22955, upload-time = "2026-10-02T23:04:47.698Z" },+ { url = "https://files.pythonhosted.org/packages/1e/e8/44cfcb5ea40e5e43cec7793ef90704ed0c475280839076c4345757eb8e59/markupsafe-3.0.4-cp311-cp311-win32.whl", hash = "sha256:2e5a7cd7fdd14fcb1ae5d7d8bf23d24fbd1daefd1fbca2580132e1ea75f098b5", size = 14094, upload-time = "2026-10-02T23:04:48.788Z" },+ { url = "https://files.pythonhosted.org/packages/91/89/f2b509f7bf79352e40117824c1070dbeafd4df67031d3fa98165a3134228/markupsafe-3.0.4-cp311-cp311-win_amd64.whl", hash = "sha256:fdb4ca07ab75ffadab4a8b135ad59cdbb3156b99310f3d565370da74a15d6bd3", size = 14328, upload-time = "2026-10-02T23:04:49.801Z" },+ { url = "https://files.pythonhosted.org/packages/2b/5a/ccf22672a0f64dc682306e288f0dabcb06c7a201f3bb6e6cbf86d9e8ad03/markupsafe-3.0.4-cp311-cp311-win_arm64.whl", hash = "sha256:569d65055d367e3dcdf30c3f41119467b73d9ee9faf332bdf40402644f5ac08e", size = 14188, upload-time = "2026-10-02T23:04:50.847Z" },+ { url = "https://files.pythonhosted.org/packages/81/09/4c59d56b8461ae8eb0d8ba34bb25b7e618547044679d58a82ef9b2479fc1/markupsafe-3.0.4-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:61631e08084be9e21a8967ec3139c7616ed7c5e9368e05c86d1b39562c8a57b6", size = 11658, upload-time = "2026-10-02T23:04:51.876Z" },+ { url = "https://files.pythonhosted.org/packages/a2/f0/d6613774d86fbf6d145751d43c59875e47a6f9f17daee0aef173bd36d90e/markupsafe-3.0.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0930db9bdc62d22944e10b066448bb65dc9abe9112880c7cab8da54db4284d5f", size = 12050, upload-time = "2026-10-02T23:04:52.931Z" },+ { url = "https://files.pythonhosted.org/packages/0d/f2/8f18e0b806eb13c1f8d07d917a720831ead54253a6dec011fbc78098a6f8/markupsafe-3.0.4-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6a45c3d514f2436064db00d7fc8778d888f0236ebfed649b53d13a59e69ad51b", size = 24363, upload-time = "2026-10-02T23:04:53.895Z" },+ { url = "https://files.pythonhosted.org/packages/60/ce/fa07dbe8a5675558fa36dea033e19995bc783de2dec5f540ccb9030b06aa/markupsafe-3.0.4-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:1e1451fab512d1bcc3dc26988ec1edb0b82c2db909132872cd9356070a6b63df", size = 28525, upload-time = "2026-10-02T23:04:54.905Z" },+ { url = "https://files.pythonhosted.org/packages/85/40/be87c01f3868ec217f8a2015089d71c22c8c5a75324822e5ed1cdd87210d/markupsafe-3.0.4-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bd3ce56ae2cbae3ba82b683bc425cd7e48d2ed8b10f3e818186b6f5646d9271c", size = 24733, upload-time = "2026-10-02T23:04:56.229Z" },+ { url = "https://files.pythonhosted.org/packages/4f/a7/aeedb5140afa41fc74c225e9184ab96723a6e873b6ee1c9fede7283456d8/markupsafe-3.0.4-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8e124f974786f831d6043728e38296969d3579db8896fe004682f5758e613581", size = 22985, upload-time = "2026-10-02T23:04:57.521Z" },+ { url = "https://files.pythonhosted.org/packages/c3/fc/e91352bb08c6a59da3ef0909d457bf95a5f5908fbf151b30a06d9dbcfbb4/markupsafe-3.0.4-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c02e8f18bdedba082cef725942ac823b9b60656db07f7e265cb31618dfd00d77", size = 22001, upload-time = "2026-10-02T23:04:58.597Z" },+ { url = "https://files.pythonhosted.org/packages/5d/f8/bffee5e7d2a3deb59748a797650a48af7e672025cf641a79344a771ad106/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:9f098115c247e11d138ab83a28fa0323c77015007ea2df73ba5fd714dfefd67c", size = 23793, upload-time = "2026-10-02T23:04:59.686Z" },+ { url = "https://files.pythonhosted.org/packages/ed/59/b853d6628ecb4d658e1d637224846d5e9bb4adf4f8df97f3be9f29dce2ec/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:d5f93ebbeb8032d47e349328ec8662d973d9b05a70b3c35df1f91fe419b84749", size = 22640, upload-time = "2026-10-02T23:05:00.768Z" },+ { url = "https://files.pythonhosted.org/packages/09/b2/1506df394f0f075797c418d0301498f49e43be194e3ffcb49e6fe6ccf022/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:64511c54db4e4987aef4c41923235927428729e8174c5dba488429be70a998ed", size = 24074, upload-time = "2026-10-02T23:05:01.813Z" },+ { url = "https://files.pythonhosted.org/packages/c7/81/5ed69cda630ac69ef60d06c09ba5a7f84ff66a2e28cf986fd5614ab3c6e6/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:e1a622f13970d81f95d0c72f9dc090dce9085fccfa4c9f2174377ee32bd15786", size = 21563, upload-time = "2026-10-02T23:05:03.239Z" },+ { url = "https://files.pythonhosted.org/packages/0c/fe/fb1e79be0fea60aa32602ebefc9c35a82bb42b4df157285ab7dfec12341a/markupsafe-3.0.4-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:c9a7f43c0b202b334cc9184af09bb8f21d3a209e038efaf106936fb69e6b026e", size = 23048, upload-time = "2026-10-02T23:05:04.479Z" },+ { url = "https://files.pythonhosted.org/packages/c8/52/7632a53360671a9b750cdbabaf9cdd89f18b42248b8e4cb42c0b0296e459/markupsafe-3.0.4-cp312-cp312-win32.whl", hash = "sha256:f0ec3b750b59375eab5b0fb2b9254810c00a3375be6d789899f1055a1d556237", size = 14108, upload-time = "2026-10-02T23:05:05.513Z" },+ { url = "https://files.pythonhosted.org/packages/3f/bf/62495e180b7000aaf30000fff849e933f74264638057176cf46852500adc/markupsafe-3.0.4-cp312-cp312-win_amd64.whl", hash = "sha256:11935df9bf455ed0c04eb87bcd720f02b1fe5e02128a9430f23aed6f93336fc7", size = 14303, upload-time = "2026-10-02T23:05:06.538Z" },+ { url = "https://files.pythonhosted.org/packages/c5/8e/4c24208776a65878d656996945aacfbfe010d3720d1a98fc0eb8491fc03b/markupsafe-3.0.4-cp312-cp312-win_arm64.whl", hash = "sha256:a4bbd2d87dd233b9fc5812160c3d0ffbe42edc22a26ce0469f58479ede633fe9", size = 14174, upload-time = "2026-10-02T23:05:07.617Z" },+ { url = "https://files.pythonhosted.org/packages/6d/18/4bc5ba32499e87bb2b0ef5b3a9bb9c00a131fa961ddf0be548cb550f548b/markupsafe-3.0.4-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:de8b364c423ef0a4bad9069657d617f9a5d2b2062457a89b1fa16ee199c399c1", size = 12786, upload-time = "2026-10-02T23:05:08.709Z" },+ { url = "https://files.pythonhosted.org/packages/4e/6f/17f0c099bf25f3e31e63cc19244d9f6af861a9a4ab778c203997903cfdd0/markupsafe-3.0.4-cp313-cp313-android_24_x86_64.whl", hash = "sha256:34bdde374c5932765d7dc685c4a1d191a3207852d67e8e0a9eb6ea85156181f1", size = 12510, upload-time = "2026-10-02T23:05:09.93Z" },+ { url = "https://files.pythonhosted.org/packages/11/af/1a141081b905036ee904ec4bd945e1f70b4e1b32d33c4e59e8cf1d58b247/markupsafe-3.0.4-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:6bd9e1788e15bfcf6a9082de42e30387e7b85d211ab21e57a939bb8cfaaf8d96", size = 11583, upload-time = "2026-10-02T23:05:10.884Z" },+ { url = "https://files.pythonhosted.org/packages/e7/0a/a89385ae590232622a03e091805cff12f24fabe6c11e0e8bae096cece81c/markupsafe-3.0.4-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:5066b244f576f91afc8ee3ba029a89f99d39c79b1853fe9d39bea9f0afbec148", size = 11801, upload-time = "2026-10-02T23:05:11.913Z" },+ { url = "https://files.pythonhosted.org/packages/ed/85/ea548dc013962eb73653124bc595635fbf9e0fa41d1f181a967ccb784dfb/markupsafe-3.0.4-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:7a83aa6e4805df46fed18e989d3d16f86ef60cb50bbc8d9ce3a6be89165fbf6e", size = 11443, upload-time = "2026-10-02T23:05:12.887Z" },+ { url = "https://files.pythonhosted.org/packages/cc/72/15f2e5ec9cf2eb00d5cdfe968d94e4156a7bd7303832c3f3b2c403a36839/markupsafe-3.0.4-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:2d1b7d9308288661f56672b1b157d75fc536714d3638487bbea17b6318a78248", size = 11665, upload-time = "2026-10-02T23:05:13.829Z" },+ { url = "https://files.pythonhosted.org/packages/ca/e0/4030bea613677e333c8a2c901fd405055f657f9d06acba5b7357984b6ef7/markupsafe-3.0.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:73e77980c7207854f00fc4e71fb1626868d5740ab4012623d55c7a99ad122a72", size = 12061, upload-time = "2026-10-02T23:05:14.807Z" },+ { url = "https://files.pythonhosted.org/packages/f3/a5/28b76a7449eb702966b88bef599e2360b411fbb3afeee8fe560939be06ec/markupsafe-3.0.4-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7018d4af1cd272e847aa5917983ab5e83e4f6579f9dbfecd4a79c0ca80b144c2", size = 24395, upload-time = "2026-10-02T23:05:15.909Z" },+ { url = "https://files.pythonhosted.org/packages/07/6c/21232811afc3a063b5e934b1ae2efda52f46154ec382f585149c020e61fe/markupsafe-3.0.4-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:c90d5b3d4e944e065a301d741b3c1d784f6bd1f503aa68b4967e32b2ba313d85", size = 27183, upload-time = "2026-10-02T23:05:16.976Z" },+ { url = "https://files.pythonhosted.org/packages/14/38/6ccdfa5b59049cb36fb80cbc80aee9cf1fc9bb77d1335ad435f2070b08cf/markupsafe-3.0.4-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:18a801868a884f216e784d7d14db2a4077143ce7610440aee2ce8f734e7cfcde", size = 24772, upload-time = "2026-10-02T23:05:18.209Z" },+ { url = "https://files.pythonhosted.org/packages/63/e0/cec6865dfe88cb48fedd4b20aed6af5158e41092adcbf3e028bcc6ec2108/markupsafe-3.0.4-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:434139499bb20b502ed3baa1f169e618f924a97e7a777fea1a49446d80106cf6", size = 23017, upload-time = "2026-10-02T23:05:19.286Z" },+ { url = "https://files.pythonhosted.org/packages/ee/76/6ed4940bb7648a9aac457c14f870cfdd5105f139a0fb1f29cd61fafa47d1/markupsafe-3.0.4-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9e227f3dbe6bde7491cf0a9965d00b88c6b1a4a95d11480ddf88bb96d397c19f", size = 22036, upload-time = "2026-10-02T23:05:20.352Z" },+ { url = "https://files.pythonhosted.org/packages/a1/4f/ed476226d4fe46a09090a36025bf319296810028df55eb12f1253b540f3a/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:b8cd1f918b26fd7b1832ece557cc18f2d8747309ff8b3f0ef9d4250c5ad67a39", size = 23786, upload-time = "2026-10-02T23:05:21.576Z" },+ { url = "https://files.pythonhosted.org/packages/9a/35/66ff30450e35ef5fba9ebc930c9411747e537fd9447b65e44f5007e2b84d/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:a5fcffb37e602b0b3c1638a97746b9b96125caa9bcf6fa41d337a9261de231ee", size = 22672, upload-time = "2026-10-02T23:05:22.922Z" },+ { url = "https://files.pythonhosted.org/packages/32/0b/72f45ce4b4efcbca4b80cf1b06703eff0be8d37e82abb78f66c85a7ead1e/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:5989cb26b2e1efc6a42216a9f6b5ee495ce5ace2e5b352a9af489976b32d1ee2", size = 24107, upload-time = "2026-10-02T23:05:24.175Z" },+ { url = "https://files.pythonhosted.org/packages/d2/03/71776e5fdcba04614b384cc102e8a4198208579d896fd1394cb7cb9aa900/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:add96447a86d205ab616665d53b2950ee81083757f56e6ea833c8b2917646b46", size = 21615, upload-time = "2026-10-02T23:05:25.215Z" },+ { url = "https://files.pythonhosted.org/packages/ab/5f/801ce02a02e7aee0f784b1ec7843026178f6adeb9c93ac67eb1992a9a84d/markupsafe-3.0.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:2628d3a8cb648ecebb3c5d6b0a1052d400e4d8b7ac0fb786be8d285b50040d17", size = 23076, upload-time = "2026-10-02T23:05:26.423Z" },+ { url = "https://files.pythonhosted.org/packages/4a/85/c43776625428f3bb4a61e8633940400e3efe6409e3c6f5bff26de5e45618/markupsafe-3.0.4-cp313-cp313-win32.whl", hash = "sha256:672d207103e6b16ca098611b0f9efad6bc00afd47c03d6ef62186495ca677dc0", size = 14118, upload-time = "2026-10-02T23:05:27.716Z" },+ { url = "https://files.pythonhosted.org/packages/6f/36/163da64de88a13db79214ef75fa041be7fa13bdb42261cf5b7484de14bfb/markupsafe-3.0.4-cp313-cp313-win_amd64.whl", hash = "sha256:1f1f9477e174582b0a1b583d60b66e1f2cf5d3fe12cee985e4aedf44766600e5", size = 14313, upload-time = "2026-10-02T23:05:28.749Z" },+ { url = "https://files.pythonhosted.org/packages/9f/a8/9b662783ffaa1149221432a923cee562f78b9cbbb8baa3df9b3753e63e1e/markupsafe-3.0.4-cp313-cp313-win_arm64.whl", hash = "sha256:06de8ef6331f6e822c28d577dc8bf43fe398800477c49498f38fc38b67ff33fc", size = 14189, upload-time = "2026-10-02T23:05:29.917Z" },+ { url = "https://files.pythonhosted.org/packages/5c/c3/a944f3b0df22bd129e96915b9f4e98d2eeca6516687d7618304a966c3c74/markupsafe-3.0.4-cp314-cp314-android_24_arm64_v8a.whl", hash = "sha256:4ed644d75aa94a2baf7ec3a96eaa160ea58c742eb9d27c6506053c5c40fc84ed", size = 12787, upload-time = "2026-10-02T23:05:30.971Z" },+ { url = "https://files.pythonhosted.org/packages/d4/d6/a44863f69d88b6c7e27889108f70d47aed259edf89d5df3c5fca1eac87d6/markupsafe-3.0.4-cp314-cp314-android_24_x86_64.whl", hash = "sha256:6d2a9efe686f9de00d0d1ea32a4a5a86d558a2277501bd78d964214eab625e59", size = 12525, upload-time = "2026-10-02T23:05:32.263Z" },+ { url = "https://files.pythonhosted.org/packages/17/8f/168ba80e532dd6a93f96f8f706f1ad41d7990b6e1aeedc1cc0d211a33497/markupsafe-3.0.4-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:8781a792a070cf2bd1b86d3aa943894115faaba6e88122a7bf32d62072742453", size = 11589, upload-time = "2026-10-02T23:05:33.251Z" },+ { url = "https://files.pythonhosted.org/packages/32/b3/aa2c95a574d3af39403a469b295886eb9b6d448da568cbebb5a2cbfdc2e5/markupsafe-3.0.4-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:971a3bbb75d97ae4e2e8f7d4834236f86f85f0c85e04ab2e191db1123b04f80b", size = 11801, upload-time = "2026-10-02T23:05:34.315Z" },+ { url = "https://files.pythonhosted.org/packages/60/d0/34b810107d83840e768bf485de795893ebbae35b26ab061b487adfa0a692/markupsafe-3.0.4-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:8909c2f1c6dd65e054ac4b573a91c8384d1492281e55d82d159d653f7a13adf6", size = 11447, upload-time = "2026-10-02T23:05:35.302Z" },+ { url = "https://files.pythonhosted.org/packages/6c/ab/2f8488f0f817a39fca068d2b17daf446bf5cdb3eae28c3720af534d873b4/markupsafe-3.0.4-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:4cf3468d5ec187ffffcaca8e61929a37448f215dafc1386a12c750a72fe53634", size = 11722, upload-time = "2026-10-02T23:05:36.363Z" },+ { url = "https://files.pythonhosted.org/packages/ad/40/e2d117b048d47282ade906fbfd92814cbee5647afc13fda88a3406039372/markupsafe-3.0.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:52704c5d36eb6dda8866493decd61111fff86244c9b1ad225ca01b9e91e5970f", size = 12061, upload-time = "2026-10-02T23:05:37.397Z" },+ { url = "https://files.pythonhosted.org/packages/9a/a8/73a81135e85ba66217f5af7facb03bbb386807e1a729ab64532e4c802652/markupsafe-3.0.4-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1caa2fa5a6184fb233153b35f654e6687bd555476f6170f29d8ee9be1a8b0af9", size = 24434, upload-time = "2026-10-02T23:05:38.407Z" },+ { url = "https://files.pythonhosted.org/packages/ac/ca/fa9216dd01efee2dfdacafe7df32b4d0170fbac694b0c258a193d6e53999/markupsafe-3.0.4-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:387d8cd30e69b3f0a72877b9ae717033396404e19095b17fe89753a981fda44f", size = 27183, upload-time = "2026-10-02T23:05:39.581Z" },+ { url = "https://files.pythonhosted.org/packages/fa/4e/a469509e538d37af51103b17b073126973f2b1cbf197ff32c7ddf025cfe5/markupsafe-3.0.4-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:051417f74bcaaefa316276e0ff723f541616ca51043d070da00249d9bddd3e3c", size = 24813, upload-time = "2026-10-02T23:05:40.671Z" },+ { url = "https://files.pythonhosted.org/packages/8f/db/d7282caf7ab03af44d5d6fdbaa019b35c7d7f1c90588b839c07cba640d6a/markupsafe-3.0.4-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a8e9f292fcda89b324f2f5c91d13f1424a153e40fc2756f38ee23b15835ff300", size = 23049, upload-time = "2026-10-02T23:05:41.864Z" },+ { url = "https://files.pythonhosted.org/packages/30/f3/b6a425206e6964efda6acee544d0eb01d1501784d0b8e2dcc74986f33b17/markupsafe-3.0.4-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:df1ae86ff54725a01fa1a0510b914ca53a161b7050be74f6204e24aded5971d0", size = 22088, upload-time = "2026-10-02T23:05:43.014Z" },+ { url = "https://files.pythonhosted.org/packages/ea/8a/84d3582fc1f0d5bd466cdf2eebf175e172158a6e70701aacec1de1b35430/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:8965520ac587c94a4ac48b729be3d8b8de00af39699b17585dfb599babe77977", size = 23846, upload-time = "2026-10-02T23:05:44.098Z" },+ { url = "https://files.pythonhosted.org/packages/1c/65/db101cce51b7ba4864ac491a9859d297dd1adf0e55b103fee9db9c47c527/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:340cbb1957ba99929cbf19a75626d36ba1ae21d1730b287d1cf7f824a20c4fc7", size = 22674, upload-time = "2026-10-02T23:05:45.23Z" },+ { url = "https://files.pythonhosted.org/packages/e0/49/ddee9813d71db0c7a5c9d97c832125e6758a0c844777f1cf076569bb0e22/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:3a93d9616ddecfb393727a0041a562cf0b15a244e20f2bd25efc7949be4c4f17", size = 24104, upload-time = "2026-10-02T23:05:46.398Z" },+ { url = "https://files.pythonhosted.org/packages/aa/0e/7d8518d726726870a2399d69fd30d0fa36c5e57a2132c336b58d7c491073/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:d2e56fd3b00222722abfb3f5f0759ddbae4b90811b5ad4343c64030ad1bde70c", size = 21637, upload-time = "2026-10-02T23:05:47.48Z" },+ { url = "https://files.pythonhosted.org/packages/b4/b0/b505e8a361ba557dbf3b3aa7331ea39b00d2022a26e925ff8463b9714bb3/markupsafe-3.0.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:0d9c47709875fdb321452056622e930c52afbc07a7d780762fbb8b4d91ce6fa4", size = 23081, upload-time = "2026-10-02T23:05:48.611Z" },+ { url = "https://files.pythonhosted.org/packages/1c/ea/9cc3cea873f980c75cbdb6f4277ce30ee955de38be0b3d02f14c108e0698/markupsafe-3.0.4-cp314-cp314-win32.whl", hash = "sha256:38fc55594dab834470b6733dead2ee9e3f657fb0608c769dcafa0ba5ab52f45c", size = 14272, upload-time = "2026-10-02T23:05:49.707Z" },+ { url = "https://files.pythonhosted.org/packages/80/f0/5792ff768a410f93ee3f84fc19345295ffc352d2c936b424cb37e514714c/markupsafe-3.0.4-cp314-cp314-win_amd64.whl", hash = "sha256:c1bc67752d5f21013cfe430df4062441714eab79f65a6a05e01505957e9c35fe", size = 14501, upload-time = "2026-10-02T23:05:50.788Z" },+ { url = "https://files.pythonhosted.org/packages/5f/cf/3d074a8edffcc6899355232ff2543ae8d929733239596423b7db79698bc9/markupsafe-3.0.4-cp314-cp314-win_arm64.whl", hash = "sha256:7e1636da3d8dfc220b6dd10264db5f2b165e4888c4518594898fbe381049af8a", size = 14364, upload-time = "2026-10-02T23:05:51.857Z" },+ { url = "https://files.pythonhosted.org/packages/d9/31/87ce42159aae2163cf3bbbd0c44bc87780510eecab1ea3859099aed95dcb/markupsafe-3.0.4-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:805c8b84534fa10891890f0e4be39f3a99e94615d93e8836bf9fa1fdca2feeb2", size = 11759, upload-time = "2026-10-02T23:05:52.951Z" },+ { url = "https://files.pythonhosted.org/packages/5f/53/b047207eeb7752e960aca3eb1df5fb7eefa7dd4c62ac49bb156456c8a702/markupsafe-3.0.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:fa95848c929b6a75f6848d3c9793e59db365ee436776e57db835cdbfa79ba977", size = 12096, upload-time = "2026-10-02T23:05:54.066Z" },+ { url = "https://files.pythonhosted.org/packages/ee/51/4326c88a13c7b755657d44b4bb986f8c3d9843ecba7e22d98661d87f9a57/markupsafe-3.0.4-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e916035e3e9930cbdfdd10abf48861340221857f45509565898e012263f7b289", size = 25699, upload-time = "2026-10-02T23:05:55.15Z" },+ { url = "https://files.pythonhosted.org/packages/f2/bb/990581b7474bfcf2cf34bed6ba5ea23bd87adb9d671213d68e88620e7a6b/markupsafe-3.0.4-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:b4d12837e0203bbace818ff4a7461afdcd78bcd782351cea148139180d7bcffe", size = 27804, upload-time = "2026-10-02T23:05:56.29Z" },+ { url = "https://files.pythonhosted.org/packages/6b/89/89491878c28e8291f5aa2fffe2c2d57230d10ae366d55dd810b840513d78/markupsafe-3.0.4-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:5086f9975abb1ab531ee6afca1761e4b59a19b446f3f6522ed776963228cfe5a", size = 25912, upload-time = "2026-10-02T23:05:57.416Z" },+ { url = "https://files.pythonhosted.org/packages/30/77/680998b54efdea06fc114565cd739b6d059f826a0279219b218dfa750d29/markupsafe-3.0.4-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b4a635a0487774f841cb1fb62e907e7195cc95bc761e053184b8acc3ceb20733", size = 23623, upload-time = "2026-10-02T23:05:58.557Z" },+ { url = "https://files.pythonhosted.org/packages/ae/75/2709f5ac5de9467b40b10e2bb8f89cc63dfb74582e09aa734b1124a217de/markupsafe-3.0.4-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:cb96e6e088d6cf71c1ea977510948320234824cf226e32f6f6e044f7a9c82b34", size = 23301, upload-time = "2026-10-02T23:05:59.94Z" },+ { url = "https://files.pythonhosted.org/packages/a0/c8/39eadc6c5b14c9c7679bfb98f4d4c6a97863b5beb91839aca4d2d6e16e55/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8b5d563170ff8ba3181caa967c99a3c804d1dedb702c7cb93a6a7c32247da978", size = 24950, upload-time = "2026-10-02T23:06:01.289Z" },+ { url = "https://files.pythonhosted.org/packages/1a/5e/01037f8a43e8ccb0bffb4fbdc5212db05bf080fdd7286cd392332d58128a/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:396ec4e65cc889f69786b3b89478b471cee5a3bcf468b9d9bb03e1a30fb291fc", size = 23330, upload-time = "2026-10-02T23:06:02.441Z" },+ { url = "https://files.pythonhosted.org/packages/d4/f4/23e83ce0596bb0cbe670502d31df8f757bbd01a392aa486fa3b40d1ed399/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:15ba9e28640feef770374b116a6f019c21f52404aeabe516aa7f800587b98cfc", size = 25239, upload-time = "2026-10-02T23:06:03.579Z" },+ { url = "https://files.pythonhosted.org/packages/88/5b/3708897368073cc683d524750474f41a77d2986152c380dcc55b20fdf340/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:d920abdfa61279ba1a2ef9484aab07bf03331f8c08a10120fa332353d06e6932", size = 22767, upload-time = "2026-10-02T23:06:04.699Z" },+ { url = "https://files.pythonhosted.org/packages/c6/61/ebda1307864b409e6b3115757a3d4a09cca46cfb6cc65191b5de226b424b/markupsafe-3.0.4-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:a9f54054101545a9a9cccefddf54316aa6e4491611fcbef9e91b3b6bebec04f6", size = 23668, upload-time = "2026-10-02T23:06:05.9Z" },+ { url = "https://files.pythonhosted.org/packages/09/15/98075cceac3b5ba0dbb8e4762a847be967d2befc349a2cf2d0ac77f62c9d/markupsafe-3.0.4-cp314-cp314t-win32.whl", hash = "sha256:12a606a492de952afcb43b59a14aaaaad120e708d3663dd0fdf2d738d427a691", size = 14301, upload-time = "2026-10-02T23:06:07.109Z" },+ { url = "https://files.pythonhosted.org/packages/0b/a3/768b560fcc4156685cb563d922b217810cfa7bc135773367f62f1f9d2078/markupsafe-3.0.4-cp314-cp314t-win_amd64.whl", hash = "sha256:a18f38cafc329bac5e3c2b96c765b4c96d3d103421ed22ab7988c1e3fce27464", size = 14526, upload-time = "2026-10-02T23:06:08.276Z" },+ { url = "https://files.pythonhosted.org/packages/93/63/da554b4c97a6b0ea3229ca7fe8cbfb620be81613d517f482e85958550537/markupsafe-3.0.4-cp314-cp314t-win_arm64.whl", hash = "sha256:eba154571c16e032112afac0dc2dfe9e63c2ceb7aedd07bb7eecf2ce26d4dd4c", size = 14421, upload-time = "2026-10-02T23:06:09.402Z" },+ { url = "https://files.pythonhosted.org/packages/a9/30/54d11c8ca027114898cab97421fb39e4ffd9ddf47cdbc44df2ec76722da9/markupsafe-3.0.4-cp315-cp315-android_24_arm64_v8a.whl", hash = "sha256:737c9c3981998eba27f11786f84fddcbabc74068b72a4a1f454ea02094b57b65", size = 12791, upload-time = "2026-10-02T23:06:10.485Z" },+ { url = "https://files.pythonhosted.org/packages/10/6d/97c913e253a14bd3cd0e15a5c56d13203b823fa7ee32498342896a072dc4/markupsafe-3.0.4-cp315-cp315-android_24_x86_64.whl", hash = "sha256:489505b03f692c3f376394e49194fa7a7f9e8558d6e293a7056a0032b0c38163", size = 12527, upload-time = "2026-10-02T23:06:11.834Z" },+ { url = "https://files.pythonhosted.org/packages/26/f9/b86d032042a4d597d9e1997f0e5f63a3eedaf11258e0a05760b0a0a826ea/markupsafe-3.0.4-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:077293e425f28ec737dbcad442a71752e28f8ae27cde3d68acd1fb212091cd92", size = 11602, upload-time = "2026-10-02T23:06:13.122Z" },+ { url = "https://files.pythonhosted.org/packages/f2/dc/73c14c1eedf0ac5fa3292ba43435e6c49d2c2050f33cebde541f8f4807f1/markupsafe-3.0.4-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:9348cbb300d224fe3b89793262cb093504d4ae927004468463f745188a193e4a", size = 11817, upload-time = "2026-10-02T23:06:14.227Z" },+ { url = "https://files.pythonhosted.org/packages/8f/69/2c2fcaa5fcee22d72c7819c0d536fd181c74a688e6143845419579cd2863/markupsafe-3.0.4-cp315-cp315-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:b807e598953730f82e4eae3bd30f6a122cf6b31c398c6b504c0e04c13c170429", size = 11446, upload-time = "2026-10-02T23:06:15.574Z" },+ { url = "https://files.pythonhosted.org/packages/88/54/9e5ec76c62e6e2834d5a93623018c943e8b3bb41d663e3fd4c03303b9b85/markupsafe-3.0.4-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:799c39bdf5e2f1292fedd3009f7b3c9e760f10b2420cb9638d56920840ff6db8", size = 11719, upload-time = "2026-10-02T23:06:16.701Z" },+ { url = "https://files.pythonhosted.org/packages/96/24/3ec292b44064c16229e064d770b2625bd8ea941aa61f44905a9fa44942c0/markupsafe-3.0.4-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:ae9dcb8fbe244cb82f8a6458b455b927a03685e383d9bacf1ea5ce180b96dc97", size = 12075, upload-time = "2026-10-02T23:06:17.855Z" },+ { url = "https://files.pythonhosted.org/packages/aa/85/b64fdb1f304848518742136983c24e96d967bfb59a0ea160e92736901ab0/markupsafe-3.0.4-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4bced6e2a6dba6a28f7dd3c6ce14df1b2dd495923f16ea484cad03decd463b2b", size = 24656, upload-time = "2026-10-02T23:06:18.963Z" },+ { url = "https://files.pythonhosted.org/packages/9c/18/23997d4c65b355da6390d61cd56e0ab3befd6ba8dda25cb40c602bd0fa6b/markupsafe-3.0.4-cp315-cp315-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3882fb412298575bae3b9c46868251f15cc69307359f87bb1b382e53d6e5a2c9", size = 27737, upload-time = "2026-10-02T23:06:20.117Z" },+ { url = "https://files.pythonhosted.org/packages/d4/36/35998dead3c6af88c38265a56e58100211f036234ab88eb2283fd4cbce44/markupsafe-3.0.4-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:04e7902ba80ee4bac1d50a549606527a1dcf0476cd81403db41099d3b60ec653", size = 25003, upload-time = "2026-10-02T23:06:21.284Z" },+ { url = "https://files.pythonhosted.org/packages/82/96/ef49135ce260db4ca4a12b119ed468449cd248db6b1468e2112b546d7a2e/markupsafe-3.0.4-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:925f929d6b59a8b3f8b8c6ac363cd0af7eecc81efb3071770b3c6717c450a369", size = 23111, upload-time = "2026-10-02T23:06:22.524Z" },+ { url = "https://files.pythonhosted.org/packages/50/7d/83126e338bd88c17a220668235368ad719fd4638e426739858cbb8508f77/markupsafe-3.0.4-cp315-cp315-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f68edfc67aabac33708941f26f22a7b8e9f81429bc0cf249fcf7d66b23af8d19", size = 22403, upload-time = "2026-10-02T23:06:23.785Z" },+ { url = "https://files.pythonhosted.org/packages/83/dd/daf7e420de23c8206c365204e7b85e1251d8e19d34196a56336f316e5ed2/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:e5c802729725bd07e2bc3ab7b76dc7e0bbfc53129d8f1eb1c002c24cf774717e", size = 24083, upload-time = "2026-10-02T23:06:25.037Z" },+ { url = "https://files.pythonhosted.org/packages/19/3c/11eecdc06bc44ad5570350085b572ebf049e8f9a38d1ece6d76640b739cd/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_armv7l.whl", hash = "sha256:55ffd6ce583d97dc71dc92e930324c8c0d25aea7e3ade6ae54ef77cedb096811", size = 22931, upload-time = "2026-10-02T23:06:26.328Z" },+ { url = "https://files.pythonhosted.org/packages/0d/9e/ac0fd77f2a726e56ecc3ca0235d095feace1358d1b822406c2a2ef26a4dc/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:2cb3dd71fc6be918ad4264346a8ed69485f9b7ed7bf35495d8e22807cd6b8bea", size = 24299, upload-time = "2026-10-02T23:06:27.742Z" },+ { url = "https://files.pythonhosted.org/packages/d7/09/c6bd842ad58ff5b3bc76eeed7e9a42a6f11adc5d090ec697b72c9672731e/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:94f5407f7bc64fa6463906b896f9904beeeb7dd8dc116ee8e9056c8714ff9916", size = 21948, upload-time = "2026-10-02T23:06:29.274Z" },+ { url = "https://files.pythonhosted.org/packages/a3/46/82f586711fed61e86faa1ee1bc317d68cd45a10c8bdbe3f7d1fdf9026ad8/markupsafe-3.0.4-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:2dad610540cb2e6272855c178f08ae9a1c7ac258a7fb71660553a5f104b42741", size = 23150, upload-time = "2026-10-02T23:06:30.583Z" },+ { url = "https://files.pythonhosted.org/packages/19/2d/2dfdce99318abbfa26925195fbc17db188c46a1ec6457be121b6f9cfeb42/markupsafe-3.0.4-cp315-cp315-win32.whl", hash = "sha256:03470d1a8268e692ecf79ecd565593e59d44219377a7ead61f1f1b94c1f7ff6b", size = 14269, upload-time = "2026-10-02T23:06:31.949Z" },+ { url = "https://files.pythonhosted.org/packages/5b/ec/6000fd82e8791e58fcd0456ec20f098957e2b03d5ed02eb73241a577c0ba/markupsafe-3.0.4-cp315-cp315-win_amd64.whl", hash = "sha256:d882a373d8093c2941e01291b7ced96e9cbe4781da9a7751ca7e6c70385e5214", size = 14496, upload-time = "2026-10-02T23:06:33.258Z" },+ { url = "https://files.pythonhosted.org/packages/bc/66/e73bd5016421d5d6e2fb6de7dd609f9de020942ac8c626526bd8c6eeaf82/markupsafe-3.0.4-cp315-cp315-win_arm64.whl", hash = "sha256:353bd63081912ab8cfa6a0c7d185934cdf8426f04c618bba6bc4b394f2069b67", size = 14361, upload-time = "2026-10-02T23:06:34.539Z" },+ { url = "https://files.pythonhosted.org/packages/90/df/cb8c3dc98d313a951df2f8968f44e4cb5643df6d3cab749a530ce2f7d972/markupsafe-3.0.4-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:c61750fadcd119d0825bcb7d7d675dd264dcc89cc05292aab5be68ebdbb374ad", size = 11760, upload-time = "2026-10-02T23:06:35.807Z" },+ { url = "https://files.pythonhosted.org/packages/d6/bb/4af9b3ca0753d654ac75f9531d5bd741bb77ca6e696f36807c475ffc099a/markupsafe-3.0.4-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:1c0df495a977d10460a94941799c72d5b5ab03d3858d949b55b5a66c8f371c99", size = 12108, upload-time = "2026-10-02T23:06:37.089Z" },+ { url = "https://files.pythonhosted.org/packages/3f/d4/b56429313aee5fd59b079c3df5615299959e25e7113eb6d8caadbdd7d38a/markupsafe-3.0.4-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:02fa4acbc6a3fc5c693c34d4dd8c1130b7fe99cc915181b0ddd6f72aeb296002", size = 25762, upload-time = "2026-10-02T23:06:38.419Z" },+ { url = "https://files.pythonhosted.org/packages/65/f5/34c181e891aa4f7d59c918584672e0c5eb7fffe76c1387d1246008bf4081/markupsafe-3.0.4-cp315-cp315t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:05295589e619b9bed252a86b532b8e27350abc372d18ba89b59375325e91ec1e", size = 28400, upload-time = "2026-10-02T23:06:39.819Z" },+ { url = "https://files.pythonhosted.org/packages/ce/b5/ad14694fd0ac9a5ce30bc6498f2999378f418583dd1679cca5a1b512957e/markupsafe-3.0.4-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:be6cb0c799abb0e2ba3e618e6d28ddddf7e485f6c2ce938dfa237daf3905072c", size = 25946, upload-time = "2026-10-02T23:06:41.381Z" },+ { url = "https://files.pythonhosted.org/packages/d6/a8/26b606445387d0ceb1eb1f21840094b84e4e3c3c3983d80d10b89823b490/markupsafe-3.0.4-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:26e9867520db70d37f7fb421a7f0d8adb40171011fb84ce869afa1a83370dfa8", size = 23634, upload-time = "2026-10-02T23:06:42.748Z" },+ { url = "https://files.pythonhosted.org/packages/39/a2/b8814de672f1f0094d498bf646f2fec9d6356b503d28ef500b71c5095377/markupsafe-3.0.4-cp315-cp315t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f03460ff076f70ab595bb45a0205ccea1971443575b6920c52e755dec2b3fbfe", size = 23488, upload-time = "2026-10-02T23:06:44.176Z" },+ { url = "https://files.pythonhosted.org/packages/db/c7/287223376fb73335a3cc5d6eb22c6ab01358cf33945a9c39c06b9dac3f4b/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:436e3ffc6310d3c41878c601db29098102fe5d8a467c49da4a4125254e0980f2", size = 25037, upload-time = "2026-10-02T23:06:45.646Z" },+ { url = "https://files.pythonhosted.org/packages/f9/29/4df8355e313426d19e62ba33e0253c009ca12a0894ee77d67fa67255361c/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_armv7l.whl", hash = "sha256:4e2c4809c14559aa7ef426f27fb35afbb38104c349a903bf8f3600456764bb38", size = 23543, upload-time = "2026-10-02T23:06:47.264Z" },+ { url = "https://files.pythonhosted.org/packages/71/e5/8377731e8495668dcc768f645e717df18318c841edaf023a99395f6da9b4/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_ppc64le.whl", hash = "sha256:da2af0d7aebfc2074080d72efa6ab8317c62481ef1f896f65d9999c1c01f4494", size = 25282, upload-time = "2026-10-02T23:06:48.795Z" },+ { url = "https://files.pythonhosted.org/packages/ed/5f/373456e37ceb1478d657d6fe769cbe0a39f0a8dfc1548eeb19c471eefdd9/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_riscv64.whl", hash = "sha256:aa2c838cc024642cc04c6854232f32b43e5e22833dd11119c1766c7873b8370d", size = 22993, upload-time = "2026-10-02T23:06:50.31Z" },+ { url = "https://files.pythonhosted.org/packages/d7/93/2cbd5628435afb6f541bbaced4bce0c2edac4b09a142e6e928b8b0da9858/markupsafe-3.0.4-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:b91cc9d336957239ff200f30097e6fea2dc6d6fb3c81e853eaa09eac904fd894", size = 23712, upload-time = "2026-10-02T23:06:51.759Z" },+ { url = "https://files.pythonhosted.org/packages/81/99/157e10966b033b363aeda5263e82596ee232a0b1d082fdbf90aa417ff083/markupsafe-3.0.4-cp315-cp315t-win32.whl", hash = "sha256:e49fb0d1ce92cfa0cb198cc5b1b11cdf9d0638658e2a2db2687e39db7c87fc78", size = 14300, upload-time = "2026-10-02T23:06:53.241Z" },+ { url = "https://files.pythonhosted.org/packages/33/05/55884815414c9706a23deca150b72c25a62109e65b0b6ce232077802c719/markupsafe-3.0.4-cp315-cp315t-win_amd64.whl", hash = "sha256:4f6e0852a0283b1b1fd776eeb7b766a5f440b3e2bd31ab51af3b400585f3965c", size = 14522, upload-time = "2026-10-02T23:06:54.729Z" },+ { url = "https://files.pythonhosted.org/packages/92/f9/ecbde7149e95b8a0f18e16d5d747f7dc06049d5da2e4f77f6f5e4a1f46a8/markupsafe-3.0.4-cp315-cp315t-win_arm64.whl", hash = "sha256:39dbacefc411633db5b4378b066a9aca70a3d7e2922c9e578d825f844026eeba", size = 14417, upload-time = "2026-10-02T23:06:56.246Z" },+]++[[package]]+name = "mdurl"+version = "0.1.2"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" },+]++[[package]]+name = "mpmath"+version = "1.3.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/e0/47/dd32fa426cc72114383ac549964eecb20ecfd886d1e5ccf5340b55b02f57/mpmath-1.3.0.tar.gz", hash = "sha256:7a28eb2a9774d00c7bc92411c19a89209d5da7c4c9a9e227be8330a23a25b91f", size = 508106, upload-time = "2023-03-07T16:47:11.061Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl", hash = "sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c", size = 536198, upload-time = "2023-03-07T16:47:09.197Z" },+]++[[package]]+name = "networkx"+version = "3.6.1"+source = { registry = "https://pypi.org/simple" }+resolution-markers = [+ "python_full_version < '3.12'",+]+sdist = { url = "https://files.pythonhosted.org/packages/6a/51/63fe664f3908c97be9d2e4f1158eb633317598cfa6e1fc14af5383f17512/networkx-3.6.1.tar.gz", hash = "sha256:26b7c357accc0c8cde558ad486283728b65b6a95d85ee1cd66bafab4c8168509", size = 2517025, upload-time = "2025-12-08T17:02:39.908Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/9e/c9/b2622292ea83fbb4ec318f5b9ab867d0a28ab43c5717bb85b0a5f6b3b0a4/networkx-3.6.1-py3-none-any.whl", hash = "sha256:d47fbf302e7d9cbbb9e2555a0d267983d2aa476bac30e90dfbe5669bd57f3762", size = 2068504, upload-time = "2025-12-08T17:02:38.159Z" },+]++[[package]]+name = "networkx"+version = "3.7"+source = { registry = "https://pypi.org/simple" }+resolution-markers = [+ "python_full_version >= '3.12'",+]+sdist = { url = "https://files.pythonhosted.org/packages/dc/76/3af777226b63a5e64a6b36b1ec5855c14e2b94a37096d4760e595fc43511/networkx-3.7.tar.gz", hash = "sha256:fd77a511bd90f39f3d016351345b52cf5319b813bdca01de3f755d3cca62e96a", size = 1866482, upload-time = "2026-09-21T16:45:16.974Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/7e/cd/fe58041e9011f307c490e3e17dd48cc516448f7c698a3f2d9d9d65d7e6a8/networkx-3.7-py3-none-any.whl", hash = "sha256:e3fd2c13a7814cee3746340d8d7f8598a67f16a58bf47fb7f8793fab6efca1b0", size = 2142205, upload-time = "2026-09-21T16:45:14.609Z" },+]++[[package]]+name = "numpy"+version = "2.4.6"+source = { registry = "https://pypi.org/simple" }+resolution-markers = [+ "python_full_version < '3.12'",+]+sdist = { url = "https://files.pythonhosted.org/packages/d0/ad/fed0499ce6a338d2a03ebae59cd15093910c8875328855781952abf6c2fe/numpy-2.4.6.tar.gz", hash = "sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda", size = 20735807, upload-time = "2026-05-18T23:37:14.07Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b3/49/ec46835a70be8fa6446c495126ac84fdb28cb2558e1620ffb87a10c8b64c/numpy-2.4.6-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:0280e0356c0829a18d9de1cb7eee50ec22ca639878d7240307ca0943d73cd2c4", size = 16969194, upload-time = "2026-05-18T23:33:13.503Z" },+ { url = "https://files.pythonhosted.org/packages/0e/0d/f5957185c0ee2f3e12f78715aa9e3b353fd83633316c8532b38faa37e3f6/numpy-2.4.6-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:110f8b71aacb688ec69062bb7f6938a0f8acb01b7c1c4beb453c65b6d234584d", size = 14964111, upload-time = "2026-05-18T23:33:17.795Z" },+ { url = "https://files.pythonhosted.org/packages/ad/40/40a40ee0ddf7ceb782c49af278894b686e586d65d8c1889c8b5da01a3d7d/numpy-2.4.6-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:4cfe66903cc32a9921a6733d96b19bb6abf310397581bbad89c228f5abaf0ee8", size = 5469159, upload-time = "2026-05-18T23:33:20.654Z" },+ { url = "https://files.pythonhosted.org/packages/63/13/f9a8046535cb21deae82f8d03de9617e08882d274fad2539630761888228/numpy-2.4.6-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:8155154c7c691289fe18f510b5d4657c68c67989f293f0535a91360392ff6538", size = 6798936, upload-time = "2026-05-18T23:33:22.987Z" },+ { url = "https://files.pythonhosted.org/packages/33/a8/6fa8c1a345a8c85dbb21932c447bee07c30a2c2a3f31e369c0a84b300147/numpy-2.4.6-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ab0a9c4ffb1a6d95ef519fe4247dba8eb6b18ad93999f76b7f657039acabd47", size = 15966692, upload-time = "2026-05-18T23:33:26.62Z" },+ { url = "https://files.pythonhosted.org/packages/02/03/74fe2a4cb3817d94d86402f2506554130a2f01414e299b5a843e5a8a957f/numpy-2.4.6-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:89cd468399cfd2504718f0ba50e410dca55a170b61a02ad92bb18c8a65186e93", size = 16918164, upload-time = "2026-05-18T23:33:29.955Z" },+ { url = "https://files.pythonhosted.org/packages/c5/80/3615be3313f7e7696609bc194b9f0101da809df79e859bdb84e0cd043f46/numpy-2.4.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:c2d37ab77531417474168eb79d6d80b14f821a966818505d03013d0833edb7a8", size = 17322877, upload-time = "2026-05-18T23:33:34.724Z" },+ { url = "https://files.pythonhosted.org/packages/ca/ac/a691e0fe2675e370d0e08ff905adc49a1c8830e8cae03efe4477e92cd55d/numpy-2.4.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f407cb6b8e9d6d8c626bc73c945db1706035af8fd632295547bf1c9e46d092d6", size = 18651487, upload-time = "2026-05-18T23:33:38.217Z" },+ { url = "https://files.pythonhosted.org/packages/15/a7/9bc1cd626d7bf6869bfedf27b91b6ab5dd607758bf8e959d6fa80c6a59cb/numpy-2.4.6-cp311-cp311-win32.whl", hash = "sha256:ddea102b48f9e339f3948bf22040944184627a30fdf7f858667673b9c5f033c8", size = 6233945, upload-time = "2026-05-18T23:33:41.331Z" },+ { url = "https://files.pythonhosted.org/packages/c5/31/7fc6239c12bce7e931463251cca4426c465e1876ba3cc785402ef4dd8f4e/numpy-2.4.6-cp311-cp311-win_amd64.whl", hash = "sha256:1e254a00cdf42b1e4d5b3d68d33af63268d41340d8885df2ab6470f2e1500147", size = 12608406, upload-time = "2026-05-18T23:33:44.131Z" },+ { url = "https://files.pythonhosted.org/packages/27/83/140f85a466595a16382996a1bf06b2b54bcd597488921b0c9daaeeda72af/numpy-2.4.6-cp311-cp311-win_arm64.whl", hash = "sha256:ed9749eef4cbd126da3dc1d6bcb3a57f5eb7ac6a6484146bdbf743f552dfc577", size = 10479528, upload-time = "2026-05-18T23:33:50.725Z" },+ { url = "https://files.pythonhosted.org/packages/95/2a/3d7b5ac8aac24feaf9ad7ed58f45b0bbc06d37e4338ae84c9f2298b570f9/numpy-2.4.6-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:001fbb8e08d942dd57599e781f2472269ee7f2755fae407b4f67b2f0b17da3f1", size = 16689119, upload-time = "2026-05-18T23:33:54.065Z" },+ { url = "https://files.pythonhosted.org/packages/ea/12/92c4c131527599e8288d6918e888d88726f84d805d784b771f32408aeaef/numpy-2.4.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ebfb099f8dcf083deef3ac1ca4c1503f387cf76296fcb3816b66f5ecb5f54fdb", size = 14699246, upload-time = "2026-05-18T23:33:57.621Z" },+ { url = "https://files.pythonhosted.org/packages/ad/fe/c0a6b7b2ca128a8fb228575147073b660656734b8ebe4d76c8fd748dcc79/numpy-2.4.6-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:3213d622a0283a39a93d188f3cf72b26862df52fbb4ca3697f51705016523d41", size = 5204410, upload-time = "2026-05-18T23:34:00.302Z" },+ { url = "https://files.pythonhosted.org/packages/f3/d4/9770d14ba719432bb90a421bfd443872ed0f70f7264b64bec12ea363d5fd/numpy-2.4.6-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:357cc07a6d7b0b182ff02249616a03742827ebb1277546b5c7cd7f7620a45698", size = 6551240, upload-time = "2026-05-18T23:34:02.852Z" },+ { url = "https://files.pythonhosted.org/packages/c9/c6/50a46a6205feba2343f1d6d17438107c5dc491ed1c736e6ea68689fd906b/numpy-2.4.6-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f9fb9157b4ce2971008323afe46053787b526ef624fea915b261468a8421a0f", size = 15671012, upload-time = "2026-05-18T23:34:05.485Z" },+ { url = "https://files.pythonhosted.org/packages/99/60/14115e6364fa676c5397c2ad3004e527e9aa487abf5d0706ec81bbd08529/numpy-2.4.6-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:90f9849678c75fe7afa2d348ac842c168b0a4d3d61919687216dfc547976d853", size = 16645538, upload-time = "2026-05-18T23:34:09.265Z" },+ { url = "https://files.pythonhosted.org/packages/ae/c5/693cbe59e57db94d2231fa519ca3978dc9e19da5a8f088588f5c6e947ff2/numpy-2.4.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c1a2af6c6ef86344a6b0db6b97834208bf598db514f2b155042439b62605601a", size = 17020706, upload-time = "2026-05-18T23:34:13.053Z" },+ { url = "https://files.pythonhosted.org/packages/ef/fc/85b7c4eff9b4966ade25c2273cf7e7012e92366c032058653934b37de044/numpy-2.4.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e5805d5a22fd19c8ccff10a9561f9df94436b0545619ea579db2d3c35294bce2", size = 18368541, upload-time = "2026-05-18T23:34:17.024Z" },+ { url = "https://files.pythonhosted.org/packages/f6/81/e1b27545deedce7f4a0b348618c6b62d74e36a4dc9ccd42f3eb2f85eee32/numpy-2.4.6-cp312-cp312-win32.whl", hash = "sha256:e3eeb0aabd6bd5ce64faae67e9935203a6991b4bc2a485a767fbafb2c5125f45", size = 5962825, upload-time = "2026-05-18T23:34:20.3Z" },+ { url = "https://files.pythonhosted.org/packages/ab/ca/feab00bd44aa5fe1ad2c18f08b4d3bb92e26484b0b1d1443897809ed528c/numpy-2.4.6-cp312-cp312-win_amd64.whl", hash = "sha256:d8e8286dd7cea7895157318d1b91cdacac64c479f3cbc8dce548331728484751", size = 12321687, upload-time = "2026-05-18T23:34:23.095Z" },+ { url = "https://files.pythonhosted.org/packages/63/cf/5a6d34850a39d1093558564f77ee8e8e0bee5061151b8f05a55711001ec7/numpy-2.4.6-cp312-cp312-win_arm64.whl", hash = "sha256:4081eb135ac24158bd51cdfbef16f1c64df7063b1143f24731387137c092bec8", size = 10221482, upload-time = "2026-05-18T23:34:25.876Z" },+ { url = "https://files.pythonhosted.org/packages/fb/82/bdab26d7438c6791ca31b7c024ca37c1eab8b726ba236129005cd4a06e45/numpy-2.4.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0", size = 16684648, upload-time = "2026-05-18T23:34:29.41Z" },+ { url = "https://files.pythonhosted.org/packages/1b/30/a80189bcc7f5e4258b3fbc3968d909d1756f54d023299ecc39ad6fdb9ef8/numpy-2.4.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb", size = 14693902, upload-time = "2026-05-18T23:34:33.013Z" },+ { url = "https://files.pythonhosted.org/packages/97/12/70b5d0d7c15e1ebb8a6a84a8caa1d19e181d84fb58bb6d70aca29099dec1/numpy-2.4.6-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f", size = 5198992, upload-time = "2026-05-18T23:34:36.132Z" },+ { url = "https://files.pythonhosted.org/packages/ba/8c/ebd2a8f8a83541f8d38cc5667e8c2b69cecfd30da6e45693e8158857d44b/numpy-2.4.6-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3", size = 6546944, upload-time = "2026-05-18T23:34:38.484Z" },+ { url = "https://files.pythonhosted.org/packages/bb/c5/7b863a97a91671a0338f4253bd3b5a3d3852f0692dae91711c9f4a10e787/numpy-2.4.6-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b", size = 15669392, upload-time = "2026-05-18T23:34:41.257Z" },+ { url = "https://files.pythonhosted.org/packages/a5/9d/3584b9984ca4c047aea75214ce1a4c4c73d849bd71b604264b7f5653f8a8/numpy-2.4.6-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089", size = 16633220, upload-time = "2026-05-18T23:34:45.075Z" },+ { url = "https://files.pythonhosted.org/packages/05/ae/7c67fba23bd98caec7c99261f3a16072ade14813486b0282cb29846de832/numpy-2.4.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a", size = 17020800, upload-time = "2026-05-18T23:34:49.065Z" },+ { url = "https://files.pythonhosted.org/packages/d9/5d/3b6725cb31d983c5e66916f5d36f6d7e5521129e4c4404d64f918292a5b6/numpy-2.4.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605", size = 18357600, upload-time = "2026-05-18T23:34:52.709Z" },+ { url = "https://files.pythonhosted.org/packages/f7/da/2ccc6c2fe8898dee01d90c75c5f5f914a23daf99e3e0f59516a08760c8b5/numpy-2.4.6-cp313-cp313-win32.whl", hash = "sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91", size = 5961134, upload-time = "2026-05-18T23:34:55.618Z" },+ { url = "https://files.pythonhosted.org/packages/b5/cd/9cc4dc876fb065d5c220aae4d5e14826b2715331bb7618ce1fb07a679d99/numpy-2.4.6-cp313-cp313-win_amd64.whl", hash = "sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359", size = 12318598, upload-time = "2026-05-18T23:34:58.928Z" },+ { url = "https://files.pythonhosted.org/packages/39/1e/c0bcba1f8694116485fe28fd1be698c278fcda4141c5b0e53a2aed8b12a8/numpy-2.4.6-cp313-cp313-win_arm64.whl", hash = "sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778", size = 10222272, upload-time = "2026-05-18T23:35:02.167Z" },+ { url = "https://files.pythonhosted.org/packages/63/6d/cc5619247c8f4204e507f5883528372e4ac4bb189e579fb859a12e480b1f/numpy-2.4.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1", size = 14821197, upload-time = "2026-05-18T23:35:05.468Z" },+ { url = "https://files.pythonhosted.org/packages/00/58/f1c39161c87d9e9bed660f1ed4bafc0e403d5ec9650b6dd77aead07d489b/numpy-2.4.6-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe", size = 5326287, upload-time = "2026-05-18T23:35:08.693Z" },+ { url = "https://files.pythonhosted.org/packages/af/57/3917ab0fd97f271a8694513581b8a36c655f111c446852c302f04ccdb6fc/numpy-2.4.6-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997", size = 6646763, upload-time = "2026-05-18T23:35:11.459Z" },+ { url = "https://files.pythonhosted.org/packages/eb/0f/037e64c494b67581ae18193d770adef354c41f3f2c8ebf865602d949bf8f/numpy-2.4.6-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20", size = 15728070, upload-time = "2026-05-18T23:35:14.79Z" },+ { url = "https://files.pythonhosted.org/packages/21/a6/5d2bae9c9542eb4df16dc9c46dc79c186e9bad53805dfa5399a6023c6db0/numpy-2.4.6-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d", size = 16681752, upload-time = "2026-05-18T23:35:18.836Z" },+ { url = "https://files.pythonhosted.org/packages/92/14/23d1dfb410ae362cd59ce53e936b1513d545eb40db3949ced632e19a459e/numpy-2.4.6-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67", size = 17086024, upload-time = "2026-05-18T23:35:22.52Z" },+ { url = "https://files.pythonhosted.org/packages/4b/6e/23595a2c642cdf3bc567877064bdd7f91c8b0038a4453cf2daf7248eafe9/numpy-2.4.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd", size = 18403398, upload-time = "2026-05-18T23:35:26.398Z" },+ { url = "https://files.pythonhosted.org/packages/8a/90/0ac3bc947217e66dec77e7cbc6a1979d1af70b6461b82f620d3bccd5e4c8/numpy-2.4.6-cp313-cp313t-win32.whl", hash = "sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab", size = 6084971, upload-time = "2026-05-18T23:35:29.387Z" },+ { url = "https://files.pythonhosted.org/packages/77/71/5673e351671a1d2bd6063b91b44f70c0affea7d1516fa7a6572941ba4aa1/numpy-2.4.6-cp313-cp313t-win_amd64.whl", hash = "sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75", size = 12458532, upload-time = "2026-05-18T23:35:32.175Z" },+ { url = "https://files.pythonhosted.org/packages/3f/88/19d3503c5046e688f049274b27a3ef3d771152fa80d3ba3d01a3dff61abe/numpy-2.4.6-cp313-cp313t-win_arm64.whl", hash = "sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd", size = 10291881, upload-time = "2026-05-18T23:35:35.465Z" },+ { url = "https://files.pythonhosted.org/packages/f8/91/3ab2044d05fd16d343c5ac2e69b127f1b2854040dd20b193257c78028bd3/numpy-2.4.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079", size = 16683458, upload-time = "2026-05-18T23:35:38.353Z" },+ { url = "https://files.pythonhosted.org/packages/8e/62/764ce66fa4147ae6d73071a3abf804ffe606f174618697c571acdf26a7c9/numpy-2.4.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7", size = 14704559, upload-time = "2026-05-18T23:35:42.14Z" },+ { url = "https://files.pythonhosted.org/packages/60/61/23f27c172f022e04025b7dc2367f4d63c1a398120607ec896228649a6f48/numpy-2.4.6-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5", size = 5209716, upload-time = "2026-05-18T23:35:45.377Z" },+ { url = "https://files.pythonhosted.org/packages/03/71/21cf70dc6ea3e3acb95fc53a265b2fc248b981f0194ceb5b475271b8809d/numpy-2.4.6-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096", size = 6543947, upload-time = "2026-05-18T23:35:47.926Z" },+ { url = "https://files.pythonhosted.org/packages/d5/91/64288395ee1799bd2e0b04a305dce9666da90c961e1f3fe982a05ee1c036/numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b", size = 15685197, upload-time = "2026-05-18T23:35:50.863Z" },+ { url = "https://files.pythonhosted.org/packages/f3/eb/ebffaa97dc55502df69584a8f0dcf07f69a3e0b3e2323670a2722db9aa39/numpy-2.4.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8", size = 16638245, upload-time = "2026-05-18T23:35:54.752Z" },+ { url = "https://files.pythonhosted.org/packages/b8/0b/54f9da33128d7e350fab89c7455902eeae70349ee52bddb448dc4a576f45/numpy-2.4.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402", size = 17036587, upload-time = "2026-05-18T23:35:58.355Z" },+ { url = "https://files.pythonhosted.org/packages/b6/f0/fdebc1052db1cc37c64beb22072d67cd6d1c71adca1299f53dec2b5e20d3/numpy-2.4.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb", size = 18363226, upload-time = "2026-05-18T23:36:02.845Z" },+ { url = "https://files.pythonhosted.org/packages/aa/b4/298628d98c72b57e57f7165ae6a481a1deaf6f3c28262a6e4c739c275930/numpy-2.4.6-cp314-cp314-win32.whl", hash = "sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1", size = 6010196, upload-time = "2026-05-18T23:36:05.92Z" },+ { url = "https://files.pythonhosted.org/packages/df/ac/46de6dda46478f7942f839e094970be2d4a861e005c4b3bf07c92e291a09/numpy-2.4.6-cp314-cp314-win_amd64.whl", hash = "sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261", size = 12450334, upload-time = "2026-05-18T23:36:09.107Z" },+ { url = "https://files.pythonhosted.org/packages/78/92/b8b798ac784102c0da830d2257d59358e3d3d90d1e2b3f2575dad976c5cf/numpy-2.4.6-cp314-cp314-win_arm64.whl", hash = "sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6", size = 10495678, upload-time = "2026-05-18T23:36:12.766Z" },+ { url = "https://files.pythonhosted.org/packages/30/34/ec28d1aa8115971537c01469ab2011ee96827930f0a124de1000cc2a7ed7/numpy-2.4.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a", size = 14823672, upload-time = "2026-05-18T23:36:16.473Z" },+ { url = "https://files.pythonhosted.org/packages/16/bd/f6d1fede4e54e8042a7ff97bb495510f3c220f94bcd9e8b228e87c92cc0d/numpy-2.4.6-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e", size = 5328731, upload-time = "2026-05-18T23:36:19.767Z" },+ { url = "https://files.pythonhosted.org/packages/f4/f0/e105b9e2fd728a9910103884decd6951d9dd73896b914a98d9a231de02ee/numpy-2.4.6-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e", size = 6649805, upload-time = "2026-05-18T23:36:22.266Z" },+ { url = "https://files.pythonhosted.org/packages/82/dd/1206a7ca6ab15e3f02069707ca96222e202af681bb73756da7527f3cb837/numpy-2.4.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43", size = 15730496, upload-time = "2026-05-18T23:36:25.713Z" },+ { url = "https://files.pythonhosted.org/packages/51/e7/38d3ea825dcab85a591734decb2f6c67caa7c8367d374df1a1c3842f9b07/numpy-2.4.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e", size = 16679616, upload-time = "2026-05-18T23:36:29.652Z" },+ { url = "https://files.pythonhosted.org/packages/93/b7/caabfdf53edf663e0b4eb74d7d405d83baef09eb5e83bcd32d601d72b93e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895", size = 17085145, upload-time = "2026-05-18T23:36:33.449Z" },+ { url = "https://files.pythonhosted.org/packages/f9/45/68d7c33a6bcf3e5aa3bdbd57a367e6f615286dfd6482f97e8ffeb734306e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4", size = 18403813, upload-time = "2026-05-18T23:36:37.369Z" },+ { url = "https://files.pythonhosted.org/packages/9c/50/0753655aa844c99cd9e018aacf76f130f1bd81d881bb74bc0aef5d73a8ba/numpy-2.4.6-cp314-cp314t-win32.whl", hash = "sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063", size = 6156982, upload-time = "2026-05-18T23:36:40.817Z" },+ { url = "https://files.pythonhosted.org/packages/b2/d4/7c67becf668f973cb490cec3e98dfd799d866f9c989a54d355672cfa0db6/numpy-2.4.6-cp314-cp314t-win_amd64.whl", hash = "sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627", size = 12638908, upload-time = "2026-05-18T23:36:43.996Z" },+ { url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" },+ { url = "https://files.pythonhosted.org/packages/de/12/b422cc84439adc0d00de605bf4a308890ae5c26f2c71fbd73e5d08fbb0dd/numpy-2.4.6-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:55cced7c52e981362f708ad635198e97a752dfba412cc03c23bbf3bd8d5cd662", size = 16847511, upload-time = "2026-05-18T23:36:50.673Z" },+ { url = "https://files.pythonhosted.org/packages/44/53/f481bef68011740f8849418d82db07230e825013f31f4eef5ba5b805316a/numpy-2.4.6-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:d6da64deb6b8ed903e7560180a92f2d804ee1ba5eeb849ac2748b8c1aba1f6d7", size = 14889064, upload-time = "2026-05-18T23:36:53.879Z" },+ { url = "https://files.pythonhosted.org/packages/7f/57/42ed575c10ced8af951d426bc4e1f8aff16fd851db33f067036215a7f860/numpy-2.4.6-pp311-pypy311_pp73-macosx_14_0_arm64.whl", hash = "sha256:68a5124b13fa6cc2086764a20005d30bc0548146f7f5322f02fce212ca14317f", size = 5394157, upload-time = "2026-05-18T23:36:57.194Z" },+ { url = "https://files.pythonhosted.org/packages/6a/ef/f66cc724fcc36c1e364c67f51ae9146090b8b584f27d58b97fdae3edd737/numpy-2.4.6-pp311-pypy311_pp73-macosx_14_0_x86_64.whl", hash = "sha256:948424b06129ce883307e8cff868c31396d8dc7630a59c61d70d98dbe70f222c", size = 6708728, upload-time = "2026-05-18T23:36:59.575Z" },+ { url = "https://files.pythonhosted.org/packages/1a/9c/c531f2293b91265d8b48e9b329f54fdd7ffae73cb4134ea10cca4237e9cc/numpy-2.4.6-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5dbbdb29840ca3d91ee0fece42fc29278886d908280bfec0a5846c6f901a3eb0", size = 15798374, upload-time = "2026-05-18T23:37:02.674Z" },+ { url = "https://files.pythonhosted.org/packages/1a/b0/413077f6b1153ed3cba361401c6783bbad6114804a000cc22eb71c13e190/numpy-2.4.6-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8ad03c0965fb3c692200e74d458ca28c1dbb4ce96f9a479a8aa041ad5fabca02", size = 16747286, upload-time = "2026-05-18T23:37:06.327Z" },+ { url = "https://files.pythonhosted.org/packages/15/ce/e5ec180bc41812edcd8daeb8639d205622c0e8c02259d8ab25a0201b3c2a/numpy-2.4.6-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:2803abfebfc990042cd494d8ce2d5f82e9d847af6d35ec486923aa19dbad5e73", size = 12504263, upload-time = "2026-05-18T23:37:09.715Z" },+]++[[package]]+name = "numpy"+version = "2.5.4"+source = { registry = "https://pypi.org/simple" }+resolution-markers = [+ "python_full_version >= '3.12'",+]+sdist = { url = "https://files.pythonhosted.org/packages/95/b0/c7453d0b6e2073c3264468b106ee1563750cecc910965e67357e3698c83e/numpy-2.5.4.tar.gz", hash = "sha256:9a94cf751c9ad8ebaa835bcd3d40dacf8534ad086b88c38029b65123c7999d2a", size = 20866315, upload-time = "2026-10-10T20:05:31.422Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d0/97/ba2074e92b7befea137e77ea8471e768bbd87c339b7e8c9f5a931949f977/numpy-2.5.4-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c6342f54c67093cae5c0227eb0eb772fdb79f2a2c37a6eb278b9909ee06aa356", size = 17001609, upload-time = "2026-10-10T20:02:40.843Z" },+ { url = "https://files.pythonhosted.org/packages/ff/a9/bac826765e971d8e16e2064e9ac7525fd69b40ac17c905033a7f5442023f/numpy-2.5.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b11e8fda06a7d69f15ebf542660b74466c2e51094800c1fb794f47ad4faeef17", size = 12015718, upload-time = "2026-10-10T20:02:43.45Z" },+ { url = "https://files.pythonhosted.org/packages/31/2f/5ea3570fcb8ccd0882bea99436a513b2c85dad8f774a2057849130a8fb99/numpy-2.5.4-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:9cb18a327b49c5c337f972b03682f6a49855525faaf3c0d3e9c96cd0fd8880a8", size = 5451717, upload-time = "2026-10-10T20:02:46.169Z" },+ { url = "https://files.pythonhosted.org/packages/34/f2/b4fc1bafca03868220b5eaf729d2f21ebd7d7b151c0f9e144fe212bbca35/numpy-2.5.4-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:aec3fc4b32ff82421274f5d205c559c51c840c8df66a78efd7f3612dd005a26a", size = 6789926, upload-time = "2026-10-10T20:02:48.139Z" },+ { url = "https://files.pythonhosted.org/packages/dc/96/8319e2457ae4333c62c815c7006b869a4f60985c1e01024c2f8c6c040fe5/numpy-2.5.4-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fe4d21ab149f15e4e6043dfb0de87e6e5f34ac176cde83060e9802981fca2ac2", size = 15695312, upload-time = "2026-10-10T20:02:50.115Z" },+ { url = "https://files.pythonhosted.org/packages/43/a3/c799c62e19c337e6d3770b08e475887fb30ce8477d3c09efca6b2f0228a6/numpy-2.5.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fbde6962867ee75b48b0ee29b2b9372ec5d617799dbaf38e82dc0596f2f7738a", size = 16727283, upload-time = "2026-10-10T20:02:53.186Z" },+ { url = "https://files.pythonhosted.org/packages/39/6b/3604e53fb00314d0dc1b94ec9125a1484f649c0a17480b1f0f0c7a9d6250/numpy-2.5.4-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:381a7a3d2e65e64c0ec302795ab9dc12bb1e73f150904699c153716177eebdaf", size = 17047890, upload-time = "2026-10-10T20:02:56.038Z" },+ { url = "https://files.pythonhosted.org/packages/4a/7a/e8b58a5289a0d464c52885de47c35a935cdd70c03a4c3ab94a5126416dd0/numpy-2.5.4-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:b89d0aaae2fe498c648f4c4795c084db535af5bd98ef942b2a3681fb74ce8645", size = 18485839, upload-time = "2026-10-10T20:02:59.018Z" },+ { url = "https://files.pythonhosted.org/packages/6f/c9/47094f597015009f310b8c900def59065ef1ff5a6fe7b51fc65ec58ec2c6/numpy-2.5.4-cp312-cp312-win32.whl", hash = "sha256:9968ab7e49b93ac6e1c3b2239732183152c9150f16308d30b66a372cffe3483c", size = 6138936, upload-time = "2026-10-10T20:03:01.626Z" },+ { url = "https://files.pythonhosted.org/packages/12/33/fefe62073dc8acfd0f2b9ed7c003af2f50aa61555e113e6db02b8f79f145/numpy-2.5.4-cp312-cp312-win_amd64.whl", hash = "sha256:a7b1b6353e36a7e50de2973a38d705c88ee93adcf120673cee7f45a4a3fa223a", size = 12573091, upload-time = "2026-10-10T20:03:04.349Z" },+ { url = "https://files.pythonhosted.org/packages/1a/07/161270b0c2eec56e4c905f6d6d22e1b836887b2cb189d3f5820aa588e9dd/numpy-2.5.4-cp312-cp312-win_arm64.whl", hash = "sha256:aa1cce2ff3f8d953de38b76bf44602caeb69f101430208f64a10067f7cb4b1d3", size = 10521630, upload-time = "2026-10-10T20:03:06.767Z" },+ { url = "https://files.pythonhosted.org/packages/67/14/1c3ee0118a8fce08565a5d8482631608426a33af10a01077fada5dc7c119/numpy-2.5.4-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:2377da2dd3ba2c1200956acbab2a358c83b8e1f8531191672d1cd6ad83250d53", size = 16997729, upload-time = "2026-10-10T20:03:09.291Z" },+ { url = "https://files.pythonhosted.org/packages/83/8c/b0ea9477fb1f0d4484bbc5cba21678cc9969704d8d7f3f158d1db35f8e14/numpy-2.5.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:7415db95818b39ec475a5eea54d9e3b6bc83e3912158e46da3438cdce399804d", size = 12009826, upload-time = "2026-10-10T20:03:11.946Z" },+ { url = "https://files.pythonhosted.org/packages/e2/84/6a3d75b3ba3dfe84ac0053450753d1e6d250a8bf80f66474cc46d1fb643f/numpy-2.5.4-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:6d6a71b9d9a97c03633aa12565ef2825ffa036cc1d99cfd50dacf0f128af4fe2", size = 5445803, upload-time = "2026-10-10T20:03:14.329Z" },+ { url = "https://files.pythonhosted.org/packages/61/18/bb993f267ca20b376e07092a16793a5b31ed3138751e9ba480011a14d742/numpy-2.5.4-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:d8200f16437b289a5bb927c6e184eccc3e8389bc0070fea4cd5b9e13c1757959", size = 6786220, upload-time = "2026-10-10T20:03:16.602Z" },+ { url = "https://files.pythonhosted.org/packages/db/b6/135bb0953b61dc21c6cafa14b424ae666944e4899cf140e00c2b322a1a45/numpy-2.5.4-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1c2e71b04c6cad90026e544501bbe0ab9290fa8a4d845e7e8c0d124fb429c988", size = 15689178, upload-time = "2026-10-10T20:03:18.721Z" },+ { url = "https://files.pythonhosted.org/packages/da/24/3bd070f3269dc609d8f26b2643f62ef91bb415841c0b294805aaf7fe06da/numpy-2.5.4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6ffa07666f8da0eef81d149934a626d0d95fbd6838432a33e66245423a9062c0", size = 16718044, upload-time = "2026-10-10T20:03:21.386Z" },+ { url = "https://files.pythonhosted.org/packages/c7/8e/9d15bd356b0a019c965312b1a3c6a727cac4cae5bc40045fbc12ce4cff9c/numpy-2.5.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:2fa3328f784fc8277fc48026f6cad516f5c561c5d8e2e39b3c9e0c8f23223b34", size = 17048364, upload-time = "2026-10-10T20:03:24.468Z" },+ { url = "https://files.pythonhosted.org/packages/dc/fe/9d5b560db964f15871885f2250795d15945f8699e17ef90c0c2ff4c875b2/numpy-2.5.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b86966fbe4ad7de710422175572bcdc75fdedadfb54bc6fab7deabccddd7780b", size = 18474904, upload-time = "2026-10-10T20:03:27.895Z" },+ { url = "https://files.pythonhosted.org/packages/e9/98/d27552990f1bd611ef3e7466adadc78312ea2df63b83aad47fdc3d3ca8df/numpy-2.5.4-cp313-cp313-win32.whl", hash = "sha256:5258bc06526964be5face2fc6f756857a3f24f21ec3e72ca131337a75b165d6c", size = 6134537, upload-time = "2026-10-10T20:03:30.511Z" },+ { url = "https://files.pythonhosted.org/packages/90/8c/140a40398a66b4471211be1affdb6ed24c486d581bd28d07b7f2fcb69540/numpy-2.5.4-cp313-cp313-win_amd64.whl", hash = "sha256:8b4d2fd2d34e5f8c9235ee787de5631a37a28402b15cb80814df973d2be54129", size = 12566113, upload-time = "2026-10-10T20:03:32.612Z" },+ { url = "https://files.pythonhosted.org/packages/34/52/01d205e5e8ccb27b2b0b141e801f22b830198c979111b0fa44771438d9a9/numpy-2.5.4-cp313-cp313-win_arm64.whl", hash = "sha256:bc39ac66a7a9a3fbd6134fda43136b60ffde99c8f4501e64e0d2b24da137babf", size = 10519523, upload-time = "2026-10-10T20:03:35.163Z" },+ { url = "https://files.pythonhosted.org/packages/99/ba/005cb5edd580d2f84d7ca3206b92dc17d4388e56e6f87ffe8f2762f83139/numpy-2.5.4-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:c668b2f0d651605b58892644b0e302c7157f7159544227758c896982ef384b18", size = 17005499, upload-time = "2026-10-10T20:03:37.961Z" },+ { url = "https://files.pythonhosted.org/packages/f3/49/fee7587c33ee35f7977f9051d7f2023d4e7246d62710c80f20c2361ea232/numpy-2.5.4-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:ffa6ce09a1c6a08e9667dd9c97aa0b14184e8d18f2a14b78b2a2328c9147f076", size = 12019666, upload-time = "2026-10-10T20:03:40.606Z" },+ { url = "https://files.pythonhosted.org/packages/d5/b2/c6ce165acffceb15a82c07b9cc77d391f86b3f379ba62911908ae5d34b91/numpy-2.5.4-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:956555e0603a4d38019ae6925711cb9dc43195c076a928accf7ea5d50bddfe53", size = 5455617, upload-time = "2026-10-10T20:03:43.138Z" },+ { url = "https://files.pythonhosted.org/packages/77/7f/dd85ce260a669a89be06842cf355d7353a33e6cfbc590fb8ebb947d88dc9/numpy-2.5.4-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:2c2c4afffdeb7920e445028dd71eb932cac3e704792e964bc2a232426d4f1255", size = 6791932, upload-time = "2026-10-10T20:03:44.874Z" },+ { url = "https://files.pythonhosted.org/packages/63/d6/34b0a2b0741386a63025a65a2c09caaaaaad6d0ca95b66cd65c30dd7fcb5/numpy-2.5.4-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4054173604cd8658796053f1f3bc0befb68ec1c0762c57fdad61e199256a8617", size = 15710899, upload-time = "2026-10-10T20:03:46.839Z" },+ { url = "https://files.pythonhosted.org/packages/16/d5/928078d2b28f26829b138b4a6c3980045022fb409f570657a224ae60ef4e/numpy-2.5.4-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d549420b8858885cea8838a727842249218b9c1da24dd517e25c9c7a948310a3", size = 16721710, upload-time = "2026-10-10T20:03:49.489Z" },+ { url = "https://files.pythonhosted.org/packages/f9/cf/673fd1b8f4cd78eb6320e87ec4c90ac19c095644259e3749853a405c70f4/numpy-2.5.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:823874a507a84af050493b622affde94b6f7c3a0dc22cb2801381bc03b871c00", size = 17066182, upload-time = "2026-10-10T20:03:52.25Z" },+ { url = "https://files.pythonhosted.org/packages/f3/92/a77b5061b1b3e2643928c37976d79ee173e1b171ed158b7a3c61056b41bc/numpy-2.5.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:4e263278bfb5ee6409db8aedbc4cc32973b1b82bc1e8d3c668551d04d83a7e37", size = 18480315, upload-time = "2026-10-10T20:03:55.39Z" },+ { url = "https://files.pythonhosted.org/packages/bb/1d/1486ef3d3fb2279fd93c4c43c1bbbf1ca389a19816696684409f71babaab/numpy-2.5.4-cp314-cp314-win32.whl", hash = "sha256:cfd73180400042a7c532d30c5e287bdd03c59ff9ee1b4c0316af0539e29dfe23", size = 6185739, upload-time = "2026-10-10T20:03:58.186Z" },+ { url = "https://files.pythonhosted.org/packages/52/9a/e1e512ebc948d5b9dd33b08736760f0ebbed2848fd4eda1f553088a6dcee/numpy-2.5.4-cp314-cp314-win_amd64.whl", hash = "sha256:2ca144f15135b6212a5c47b1e2aeca6e412f102f95a2d5d88d8aec77eb255de3", size = 12703552, upload-time = "2026-10-10T20:04:00.28Z" },+ { url = "https://files.pythonhosted.org/packages/2c/05/de709a982d7bbcd688a3fad71f002e9ff80c2db39e03ee726609b610f1d1/numpy-2.5.4-cp314-cp314-win_arm64.whl", hash = "sha256:468397ba3c64427474706e5c9123fe266395496714dc684294eac75cd4930d1e", size = 10803901, upload-time = "2026-10-10T20:04:02.659Z" },+ { url = "https://files.pythonhosted.org/packages/13/34/083570ada3bb2a30fbe5d77c8c6fef9141144a15d33e6f793a67e9749ab8/numpy-2.5.4-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1ef3aa6d7e29bb13677323114280b05acc57607fa2300e66432d665d5418a162", size = 12138695, upload-time = "2026-10-10T20:04:05.012Z" },+ { url = "https://files.pythonhosted.org/packages/94/06/1f9c24db48eef0c2d1207e3b11fffb0478e39dfd8c1e1be7476936885eed/numpy-2.5.4-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:98b053943e5a0474ec0da309d2cb9d3f18ea57f8a2067c2ab7b5f763d1068380", size = 5574615, upload-time = "2026-10-10T20:04:07.316Z" },+ { url = "https://files.pythonhosted.org/packages/da/0f/593fba2e1560e949123bc7d2fc48b5893d56e58cd4bd5a273d2fbf60b220/numpy-2.5.4-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:b64a85f40e154983960a4167d4c1d57a50c7f109b3d3264a3a984154e90a8454", size = 6889383, upload-time = "2026-10-10T20:04:09.918Z" },+ { url = "https://files.pythonhosted.org/packages/eb/9f/b799dfdce4e05e80ed4bc815c71ff343a11533b2c0ffc221cae8538cda63/numpy-2.5.4-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a813ed7719bf45463c51779e6a98d0385fe905e48447526938a4b8337333d551", size = 15753763, upload-time = "2026-10-10T20:04:12.278Z" },+ { url = "https://files.pythonhosted.org/packages/34/88/16c5f12f86f5ad2817c4d103205131fc6c8acb3d1878af05a1a4f23ec859/numpy-2.5.4-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c9b80cdf5cedba0e90d93fa5f9a333c4d65bd545cd669b71bb97ce2b703c9d73", size = 16757212, upload-time = "2026-10-10T20:04:14.799Z" },+ { url = "https://files.pythonhosted.org/packages/ff/4f/a1fe40e18a898e6a5089f4f0d891f0a493eb0574d5b34458f0fbe5aa3e5c/numpy-2.5.4-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:2199ed071f460487c8db2c0e5c0b564494190edb4772fe80f9aad88b2604def5", size = 17116471, upload-time = "2026-10-10T20:04:17.58Z" },+ { url = "https://files.pythonhosted.org/packages/aa/46/e923a11c78e65c1722e7aaad817c06bd591324174b9d28ce5d31eee4d432/numpy-2.5.4-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:64f9c9878c1938476365e11ccfb6b770f3b9e5f045ccddc514235041e6959365", size = 18524063, upload-time = "2026-10-10T20:04:20.365Z" },+ { url = "https://files.pythonhosted.org/packages/5a/fa/84ab064514440c1f64a1b21088f2c82756defdd05e07c75ab233899565b2/numpy-2.5.4-cp314-cp314t-win32.whl", hash = "sha256:64d1c8ac28a4077cf987e0a71a7a0ef7e2df70722f07f0baa42dbb7eb6938647", size = 6340926, upload-time = "2026-10-10T20:04:22.865Z" },+ { url = "https://files.pythonhosted.org/packages/7e/7e/6cd886876f435b10685db9b9f7eeb70356f99e052116f4e5f11c5792c714/numpy-2.5.4-cp314-cp314t-win_amd64.whl", hash = "sha256:067374eb538c34c745436365cf7b0112595c1d326f21ce4ff340f61230239fbb", size = 12901584, upload-time = "2026-10-10T20:04:24.99Z" },+ { url = "https://files.pythonhosted.org/packages/38/1b/3c1684f6a06f7307f2335fca6e486cb162847fb97e91d65f8eb5cabad213/numpy-2.5.4-cp314-cp314t-win_arm64.whl", hash = "sha256:e94aef2c639da4a960ad0db8e06471208d8589974953d78b61d345b4eb99e394", size = 10891152, upload-time = "2026-10-10T20:04:27.52Z" },+ { url = "https://files.pythonhosted.org/packages/08/f4/3224deff3af2bef6bc0b175369698d8cb348f3d91d9bb0286cd5c9eae9e0/numpy-2.5.4-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:8dddfbee2e68d26d0d7d7d9cb247b1fd4409241cce32d815a11d97ec2cfde179", size = 17003231, upload-time = "2026-10-10T20:04:30.021Z" },+ { url = "https://files.pythonhosted.org/packages/be/75/fee0b8c6d94b44b2fdfae74f6a4ad5a138739589a8aebaec28ce4e713ed5/numpy-2.5.4-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:81e3420b27048b65eb14c3acf0c174a8cb0e023277716110347d2dcb26026dad", size = 12018300, upload-time = "2026-10-10T20:04:32.519Z" },+ { url = "https://files.pythonhosted.org/packages/47/c0/d0b335a499a04b65f532c3f034346ef390f81299060f928492dabc1e0272/numpy-2.5.4-cp315-cp315-macosx_14_0_arm64.whl", hash = "sha256:0b4724a19de67bea8cfc4970798efa78bcbbe2ac2613cfac16721a42d44de2a5", size = 5454250, upload-time = "2026-10-10T20:04:34.943Z" },+ { url = "https://files.pythonhosted.org/packages/5a/0e/461b3783c03d668052e6a21b01b673db6ffcb7831fd32d9aa5368c1cd426/numpy-2.5.4-cp315-cp315-macosx_14_0_x86_64.whl", hash = "sha256:2132418bf8dd124a427ca9e6a1daf9ee1a87185344c95119ceae868b99466da1", size = 6789644, upload-time = "2026-10-10T20:04:37.258Z" },+ { url = "https://files.pythonhosted.org/packages/b3/02/5dad269b02166965a7b4ca14adaddd75dbee0de42435bfecf561b84ba5a6/numpy-2.5.4-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:325518d4245b9e331387702aa58c2ce1dc4cdcbb41dfb4ccd5dcbc7e08db1266", size = 15704353, upload-time = "2026-10-10T20:04:39.616Z" },+ { url = "https://files.pythonhosted.org/packages/93/3a/01360c8036822ed9f7aa32189a77d1476567ec1e8e1383522389e4faac45/numpy-2.5.4-cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:56733449d2544178beaa4545cee357370440cf056c197f9c7bfb19dbfdd0e86d", size = 16718648, upload-time = "2026-10-10T20:04:42.383Z" },+ { url = "https://files.pythonhosted.org/packages/7d/5c/b863a2c093c4d6f21a597fcaf24ead0835c09ab16a8312d5a5a8868af683/numpy-2.5.4-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:5ec3753760c1a6d8bb91200666e545c3a9728e6269dfb5d6ce02340996698aa3", size = 17059053, upload-time = "2026-10-10T20:04:44.976Z" },+ { url = "https://files.pythonhosted.org/packages/0a/60/ced4f57f9a1258a0af74f17cb0b0c2700b5c67cd6678823c803b263e4df3/numpy-2.5.4-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:b1185012870173de7ae33d370bd45b1cf5baee747ea4b97036b65f4e93016877", size = 18477406, upload-time = "2026-10-10T20:04:47.863Z" },+ { url = "https://files.pythonhosted.org/packages/f9/bd/0ef22dafaafcc7d4bb3ca26b8d2afbd55dedad8eaba99a8c864e1997456f/numpy-2.5.4-cp315-cp315-win32.whl", hash = "sha256:298eca75243f2cbbfdb460560b9fb2a1792a33cf2ab4286efd43d92e8d3df508", size = 6185133, upload-time = "2026-10-10T20:04:50.467Z" },+ { url = "https://files.pythonhosted.org/packages/50/bc/d2651b155ecc608a77e6f4d15495c11f14f19bb98f8bf0c5b0d38f86dda1/numpy-2.5.4-cp315-cp315-win_amd64.whl", hash = "sha256:332f3378fe077dd850e677ec01bdcc4f22368fb5d50ef10b2c79230b1bf5a592", size = 12703085, upload-time = "2026-10-10T20:04:52.63Z" },+ { url = "https://files.pythonhosted.org/packages/dc/d2/45e404f8abb26fb9eda12b94012936873e827b1be76f2ee7890be128312e/numpy-2.5.4-cp315-cp315-win_arm64.whl", hash = "sha256:d4cccbbc78717966f764cd3af4fb70276fa01fc7a2688af11c78901fa5c04f05", size = 10801451, upload-time = "2026-10-10T20:04:55.677Z" },+ { url = "https://files.pythonhosted.org/packages/c6/c3/2ae14e09cfdb67dc187a342e15308a21c15bf4d2071f8079e6aee5fe56dc/numpy-2.5.4-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:950ea81d57ef070665581b6e1b5f6a029306423cd1739c5b95fe78aa30db6b9d", size = 17097121, upload-time = "2026-10-10T20:04:58.403Z" },+ { url = "https://files.pythonhosted.org/packages/f5/cf/305ae624ef8a039414317224abe9ec9c2fe7ea3c2e1cf204d43ff6b2ffb9/numpy-2.5.4-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:c05ede731b03fb1b7591faca9389ade3267d2bddf1ad8882bb3f2cc5e101694f", size = 12135439, upload-time = "2026-10-10T20:05:01.65Z" },+ { url = "https://files.pythonhosted.org/packages/a9/a8/f75c63813aef95827bb2c0d13b12803016853056e8792c280058cdbfe783/numpy-2.5.4-cp315-cp315t-macosx_14_0_arm64.whl", hash = "sha256:5fbf7141bbfd63aea22f435c9062a032b9ea0082fe9845dad7f021d3f1234e71", size = 5571451, upload-time = "2026-10-10T20:05:04.135Z" },+ { url = "https://files.pythonhosted.org/packages/6f/0f/f17763f983868b5c49b4101ebd7e00760bd1769478a6bb6a8de6e085bbac/numpy-2.5.4-cp315-cp315t-macosx_14_0_x86_64.whl", hash = "sha256:3573cd22564692a5b899ec344e5d5b9cc4576f2985b96f22af3564ed54f2710f", size = 6883356, upload-time = "2026-10-10T20:05:06.249Z" },+ { url = "https://files.pythonhosted.org/packages/67/a7/8af04c5a79e047996cfa38854dcfbececdd0343a7c933a46fdd03ef6f5da/numpy-2.5.4-cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6c109eac9cd439193678f69d70733c1108487546ca8eafc107b510ae10c1aecd", size = 15750991, upload-time = "2026-10-10T20:05:08.376Z" },+ { url = "https://files.pythonhosted.org/packages/57/7a/648254290d0c504faa8f2d07aa206660c728802c781a6f3fc68ab7cb5d71/numpy-2.5.4-cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:80d6ef6e8620eb2c2b4c4caad50b5935d6db3cde2d51581b55dcc79e14016d1d", size = 16757675, upload-time = "2026-10-10T20:05:11.393Z" },+ { url = "https://files.pythonhosted.org/packages/b8/fe/4a8c3cdb0c70400cfe4c5bec42d3099a5673802a95064614b33e07b82aa1/numpy-2.5.4-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:77045a4b175bbf5316ec08003880804336c78f92281a1b72222b274ea85ec5ac", size = 17113846, upload-time = "2026-10-10T20:05:14.49Z" },+ { url = "https://files.pythonhosted.org/packages/1b/7e/619692bb67778702c0e9eb2d468568a7573f4e269386ea61aed01ee4e557/numpy-2.5.4-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:0f02a46e49cfb6c73bdb7aea1c0d3461dbae9aba613542b65f657cd3d17b9fab", size = 18522915, upload-time = "2026-10-10T20:05:17.33Z" },+ { url = "https://files.pythonhosted.org/packages/b7/b5/4da41c328788f575838f97a098fe8ca691ebc6f6fd73ad4a262ee40b184d/numpy-2.5.4-cp315-cp315t-win32.whl", hash = "sha256:ad62a416ddcf863bf44bba76fbf6b53366ab0692e294f51cae4b5fbe0d246788", size = 6335804, upload-time = "2026-10-10T20:05:19.921Z" },+ { url = "https://files.pythonhosted.org/packages/98/94/6482ddfa3d312490cb9358f375bf2ad56427dbea8769187158e94d653753/numpy-2.5.4-cp315-cp315t-win_amd64.whl", hash = "sha256:38f47be9f74ab870d2633b5456ae519c43758a8d1fd05342f0ce4ecc034396ee", size = 12890095, upload-time = "2026-10-10T20:05:21.875Z" },+ { url = "https://files.pythonhosted.org/packages/48/7f/c2d1b436b6e7cfebac140c2579a298344b85f2991a2ce5c3615cefb29400/numpy-2.5.4-cp315-cp315t-win_arm64.whl", hash = "sha256:7a14a461d9340f1b46b8648578aed9cdb8b3b018a8fac6c1dde2c9192a01a87f", size = 10883718, upload-time = "2026-10-10T20:05:28.547Z" },+]++[[package]]+name = "nvidia-cublas"+version = "13.1.1.3"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-cuda-nvrtc" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/a7/a1/0bd24ee8c8d03adac032fd2909426a00c88f8c57961b1277ded97f91119f/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:b7a210458267ac818974c53038fbec2e969d5c99f305ab15c72522fa9f001dd5", size = 542848918, upload-time = "2026-04-08T18:46:22.985Z" },+ { url = "https://files.pythonhosted.org/packages/3b/cd/154ca20c38269e05eff77c1464e6c1da89f50a6390b565e9d82e06bc11e1/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:37936a16db8fe4ac1f065c2139360608a543a09275cb1a1af612e08cfa065436", size = 423138758, upload-time = "2026-04-08T18:46:58.655Z" },+]++[[package]]+name = "nvidia-cuda-cupti"+version = "13.0.85"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/2a/2a/80353b103fc20ce05ef51e928daed4b6015db4aaa9162ed0997090fe2250/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_aarch64.whl", hash = "sha256:796bd679890ee55fb14a94629b698b6db54bcfd833d391d5e94017dd9d7d3151", size = 10310827, upload-time = "2025-09-04T08:26:42.012Z" },+ { url = "https://files.pythonhosted.org/packages/33/6d/737d164b4837a9bbd202f5ae3078975f0525a55730fe871d8ed4e3b952b0/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl", hash = "sha256:4eb01c08e859bf924d222250d2e8f8b8ff6d3db4721288cf35d14252a4d933c8", size = 10715597, upload-time = "2025-09-04T08:26:51.312Z" },+]++[[package]]+name = "nvidia-cuda-nvrtc"+version = "13.0.88"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/c3/68/483a78f5e8f31b08fb1bb671559968c0ca3a065ac7acabfc7cee55214fd6/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:ad9b6d2ead2435f11cbb6868809d2adeeee302e9bb94bcf0539c7a40d80e8575", size = 90215200, upload-time = "2025-09-04T08:28:44.204Z" },+ { url = "https://files.pythonhosted.org/packages/b7/dc/6bb80850e0b7edd6588d560758f17e0550893a1feaf436807d64d2da040f/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d27f20a0ca67a4bb34268a5e951033496c5b74870b868bacd046b1b8e0c3267b", size = 43015449, upload-time = "2025-09-04T08:28:20.239Z" },+]++[[package]]+name = "nvidia-cuda-runtime"+version = "13.0.96"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/87/4f/17d7b9b8e285199c58ce28e31b5c5bbaa4d8271af06a89b6405258245de2/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:ef9bcbe90493a2b9d810e43d249adb3d02e98dd30200d86607d8d02687c43f55", size = 2261060, upload-time = "2025-10-09T08:55:15.78Z" },+ { url = "https://files.pythonhosted.org/packages/2e/24/d1558f3b68b1d26e706813b1d10aa1d785e4698c425af8db8edc3dced472/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7f82250d7782aa23b6cfe765ecc7db554bd3c2870c43f3d1821f1d18aebf0548", size = 2243632, upload-time = "2025-10-09T08:55:36.117Z" },+]++[[package]]+name = "nvidia-cudnn-cu13"+version = "9.24.0.43"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-cublas" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/ca/30/7c257e3d5cb4fecb147b93895c66e29c93f8e76d74b45bb418ff0587c4ec/nvidia_cudnn_cu13-9.24.0.43-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:a6812a554a1ff0413e9c52b84c26c050380649ab9615f9c16bded368ce9f421f", size = 650976863, upload-time = "2026-07-02T16:23:39.248Z" },+ { url = "https://files.pythonhosted.org/packages/5c/ba/791cffd048fe5b044e620df55267e3e95c0e6e07d50b41e377c03dfc910f/nvidia_cudnn_cu13-9.24.0.43-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:71f181cd810e90f9b6023b01186fe82d13d65f0ec098581ee201d39fad769e4b", size = 553099438, upload-time = "2026-07-02T16:27:42.58Z" },+]++[[package]]+name = "nvidia-cufft"+version = "12.0.0.61"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-nvjitlink" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/8b/ae/f417a75c0259e85c1d2f83ca4e960289a5f814ed0cea74d18c353d3e989d/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2708c852ef8cd89d1d2068bdbece0aa188813a0c934db3779b9b1faa8442e5f5", size = 214053554, upload-time = "2025-09-04T08:31:38.196Z" },+ { url = "https://files.pythonhosted.org/packages/a8/2f/7b57e29836ea8714f81e9898409196f47d772d5ddedddf1592eadb8ab743/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:6c44f692dce8fd5ffd3e3df134b6cdb9c2f72d99cf40b62c32dde45eea9ddad3", size = 214085489, upload-time = "2025-09-04T08:31:56.044Z" },+]++[[package]]+name = "nvidia-cufile"+version = "1.15.1.6"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/3f/70/4f193de89a48b71714e74602ee14d04e4019ad36a5a9f20c425776e72cd6/nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:08a3ecefae5a01c7f5117351c64f17c7c62efa5fffdbe24fc7d298da19cd0b44", size = 1223672, upload-time = "2025-09-04T08:32:22.779Z" },+ { url = "https://files.pythonhosted.org/packages/ab/73/cc4a14c9813a8a0d509417cf5f4bdaba76e924d58beb9864f5a7baceefbf/nvidia_cufile-1.15.1.6-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:bdc0deedc61f548bddf7733bdc216456c2fdb101d020e1ab4b88d232d5e2f6d1", size = 1136992, upload-time = "2025-09-04T08:32:14.119Z" },+]++[[package]]+name = "nvidia-curand"+version = "10.4.0.35"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/1e/72/7c2ae24fb6b63a32e6ae5d241cc65263ea18d08802aaae087d9f013335a2/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:133df5a7509c3e292aaa2b477afd0194f06ce4ea24d714d616ff36439cee349a", size = 61962106, upload-time = "2025-08-04T10:21:41.128Z" },+ { url = "https://files.pythonhosted.org/packages/a5/9f/be0a41ca4a4917abf5cb9ae0daff1a6060cc5de950aec0396de9f3b52bc5/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:1aee33a5da6e1db083fe2b90082def8915f30f3248d5896bcec36a579d941bfc", size = 59544258, upload-time = "2025-08-04T10:22:03.992Z" },+]++[[package]]+name = "nvidia-cusolver"+version = "12.0.4.66"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-cublas" },+ { name = "nvidia-cusparse" },+ { name = "nvidia-nvjitlink" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/c8/c3/b30c9e935fc01e3da443ec0116ed1b2a009bb867f5324d3f2d7e533e776b/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:02c2457eaa9e39de20f880f4bd8820e6a1cfb9f9a34f820eb12a155aa5bc92d2", size = 223467760, upload-time = "2025-09-04T08:33:04.222Z" },+ { url = "https://files.pythonhosted.org/packages/5f/67/cba3777620cdacb99102da4042883709c41c709f4b6323c10781a9c3aa34/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:0a759da5dea5c0ea10fd307de75cdeb59e7ea4fcb8add0924859b944babf1112", size = 200941980, upload-time = "2025-09-04T08:33:22.767Z" },+]++[[package]]+name = "nvidia-cusparse"+version = "12.6.3.3"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "nvidia-nvjitlink" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/f8/94/5c26f33738ae35276672f12615a64bd008ed5be6d1ebcb23579285d960a9/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:80bcc4662f23f1054ee334a15c72b8940402975e0eab63178fc7e670aa59472c", size = 162155568, upload-time = "2025-09-04T08:33:42.864Z" },+ { url = "https://files.pythonhosted.org/packages/fa/18/623c77619c31d62efd55302939756966f3ecc8d724a14dab2b75f1508850/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:2b3c89c88d01ee0e477cb7f82ef60a11a4bcd57b6b87c33f789350b59759360b", size = 145942937, upload-time = "2025-09-04T08:33:58.029Z" },+]++[[package]]+name = "nvidia-cusparselt-cu13"+version = "0.8.1"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/46/e1/cdc1797eadf82d3a9a575a19b33fdc871a97edbec42c00b5b5e914f4aff4/nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_aarch64.whl", hash = "sha256:4dca476c50bf4780d46cd0bfbd82e2bc10a08e4fef7950917ce8d7578d22a23f", size = 221051344, upload-time = "2025-09-05T18:49:51.289Z" },+ { url = "https://files.pythonhosted.org/packages/34/7d/2661f2fb3ac4302f3a246f5fc030213ac60c1fe0bce84f9783dbd831dbb7/nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_x86_64.whl", hash = "sha256:786ce87568c303fadb5afcc7102d454cd3040d75f6f8626f5db460d1871f4dd0", size = 170148586, upload-time = "2025-09-05T18:50:50.248Z" },+]++[[package]]+name = "nvidia-nccl-cu13"+version = "2.30.7"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/d1/21/a73174c6157101bdf1ffc22b517f76ff0082613989dd9bc8f43e8034caac/nvidia_nccl_cu13-2.30.7-py3-none-manylinux_2_18_aarch64.whl", hash = "sha256:ca786ffa5a647c75d4d1f5cc72a6c4f537947e2ba8823d7c8aaf768e7a7b9f77", size = 215983881, upload-time = "2026-06-09T03:23:15.633Z" },+ { url = "https://files.pythonhosted.org/packages/3f/34/c500f90c7ae641b8e0f98965b36b8a7ac79cc8b296e8d251fe3eb592ee54/nvidia_nccl_cu13-2.30.7-py3-none-manylinux_2_18_x86_64.whl", hash = "sha256:cefa7fdb9710efd0f39c5f1be1d61ff6fc9a996c451265bd7fbdcf9455ed4b50", size = 215965170, upload-time = "2026-06-09T03:23:39.73Z" },+]++[[package]]+name = "nvidia-nvjitlink"+version = "13.4.92"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/1d/6b/eef7a9e32872b8f41e145bf10cddc9af26e153c338852811fe9a9baddf9e/nvidia_nvjitlink-13.4.92-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:e0391f24ed94ec879b84e3da4d4ec320c879aff681f2c7a638462f7199284323", size = 42452378, upload-time = "2026-09-16T20:45:29.042Z" },+ { url = "https://files.pythonhosted.org/packages/1f/a8/1cbd4014898af8b419e69b0d7dbc63da2121ee92d92b47d59f4fe9075349/nvidia_nvjitlink-13.4.92-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:25f74fad0d654271c921ac4dca614bd6258bc21791242fc7b2289dad7ae9c099", size = 40420120, upload-time = "2026-09-16T20:45:19.163Z" },+]++[[package]]+name = "nvidia-nvshmem-cu13"+version = "3.4.5"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/dc/0f/05cc9c720236dcd2db9c1ab97fff629e96821be2e63103569da0c9b72f19/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:6dc2a197f38e5d0376ad52cd1a2a3617d3cdc150fd5966f4aee9bcebb1d68fe9", size = 60215947, upload-time = "2025-09-06T00:32:20.022Z" },+ { url = "https://files.pythonhosted.org/packages/3c/35/a9bf80a609e74e3b000fef598933235c908fcefcef9026042b8e6dfde2a9/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:290f0a2ee94c9f3687a02502f3b9299a9f9fe826e6d0287ee18482e78d495b80", size = 60412546, upload-time = "2025-09-06T00:32:41.564Z" },+]++[[package]]+name = "nvidia-nvtx"+version = "13.0.85"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/c2/f3/d86c845465a2723ad7e1e5c36dcd75ddb82898b3f53be47ebd429fb2fa5d/nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:4936d1d6780fbe68db454f5e72a42ff64d1fd6397df9f363ae786930fd5c1cd4", size = 148047, upload-time = "2025-09-04T08:29:01.761Z" },+ { url = "https://files.pythonhosted.org/packages/a8/64/3708a90d1ebe202ffdeb7185f878a3c84d15c2b2c31858da2ce0583e2def/nvidia_nvtx-13.0.85-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:cb7780edb6b14107373c835bf8b72e7a178bac7367e23da7acb108f973f157a6", size = 148878, upload-time = "2025-09-04T08:28:53.627Z" },+]++[[package]]+name = "packaging"+version = "26.3"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/7d/fa/3944b40b07da9ce895c0e6303a5ab7d53da063554f534556b134a54d6093/packaging-26.3.tar.gz", hash = "sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79", size = 313412, upload-time = "2026-08-04T18:15:28.737Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", size = 129956, upload-time = "2026-08-04T18:15:27.159Z" },+]++[[package]]+name = "pillow"+version = "12.3.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/1c/3d/bb7fca845737cf9d7dbde16ed1843984665ff2e0a518f5db43e77ec540b9/pillow-12.3.0.tar.gz", hash = "sha256:3b8182a766685eaa002637e28b4ec8d6b18819a0c71f579bf0dbaa5830297cce", size = 47025035, upload-time = "2026-07-01T11:56:38.965Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/fb/c8/0a78b0e02d7ac54bc03e5321c9220da52f0c2ea83b21f7c40e7f3169c502/pillow-12.3.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756", size = 5392415, upload-time = "2026-07-01T11:53:47.162Z" },+ { url = "https://files.pythonhosted.org/packages/b2/5b/a02d30018abd97ced9f5a6c63d28597694a00d066516b9c1c6de45859fc9/pillow-12.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6", size = 4785266, upload-time = "2026-07-01T11:53:49.079Z" },+ { url = "https://files.pythonhosted.org/packages/c8/98/766667a4be768150a202836acd9fad19c06824ca86c4286d3cf6b274964e/pillow-12.3.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd", size = 6263814, upload-time = "2026-07-01T11:53:51.32Z" },+ { url = "https://files.pythonhosted.org/packages/3b/2d/ede717bc1144f63886c21fd349bb95860b0d1a21149ff16f2bb362b612b6/pillow-12.3.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd", size = 6934408, upload-time = "2026-07-01T11:53:53.487Z" },+ { url = "https://files.pythonhosted.org/packages/a3/48/9c58b685e69d49c31af6c8eb9012055fab7e665785165c84796e2c73ce72/pillow-12.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c", size = 6337160, upload-time = "2026-07-01T11:53:55.457Z" },+ { url = "https://files.pythonhosted.org/packages/ff/fa/dc2a5c0ba6df93f67c31d34b808b7ce440b40cdbf96f0b81cde1d1e6fa93/pillow-12.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5", size = 7045172, upload-time = "2026-07-01T11:53:57.736Z" },+ { url = "https://files.pythonhosted.org/packages/86/a5/444817a4d4c4c2417df00513086ca196f388d8f9ef40c2e4ccd1ad1af54b/pillow-12.3.0-cp311-cp311-win32.whl", hash = "sha256:10e41f0fbf1eec8cfd234b8fe17a4caac7c9d0db4c204d3c173a8f9f6ef3232b", size = 6472232, upload-time = "2026-07-01T11:53:59.767Z" },+ { url = "https://files.pythonhosted.org/packages/63/c6/4bad1b18d132a50b27e1365e1ab163616f7a5bb56d330f66f9d1d9d4f9d4/pillow-12.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:8e95e1385e4998ae9694eeaa4730ba5457ff61185b3a55e2e7bea0880aef452a", size = 7233653, upload-time = "2026-07-01T11:54:02.066Z" },+ { url = "https://files.pythonhosted.org/packages/fd/16/00f91ab7760dc842f5aad55217e80fc4a7067a0604535249bc8a2d6d9870/pillow-12.3.0-cp311-cp311-win_arm64.whl", hash = "sha256:ebaea975e03d3141d9d3a507df75c9b3ec90fa9d2ffd07567b3a978d9d790b26", size = 2568195, upload-time = "2026-07-01T11:54:04.622Z" },+ { url = "https://files.pythonhosted.org/packages/37/bf/fb3ebff8ddcb76aac5a01389251bbbb9519922a9b520d8247c1ca864a25d/pillow-12.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965", size = 5345969, upload-time = "2026-07-01T11:54:06.397Z" },+ { url = "https://files.pythonhosted.org/packages/d8/66/9a386a92561f402389a4fc70c18838bf6d35eb5eb5c6850b4b2dc64f5048/pillow-12.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7", size = 4780323, upload-time = "2026-07-01T11:54:09.351Z" },+ { url = "https://files.pythonhosted.org/packages/25/27/ac8f99618ffd3dde21db0f4d4b1d2ab00c0880595bfd17df103f7f39fd0c/pillow-12.3.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9", size = 6266838, upload-time = "2026-07-01T11:54:11.71Z" },+ { url = "https://files.pythonhosted.org/packages/84/21/a35af28dcc61f37ed850a2d64c65c701321dfbf25085e469d5559360cbbf/pillow-12.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91", size = 6940830, upload-time = "2026-07-01T11:54:13.732Z" },+ { url = "https://files.pythonhosted.org/packages/eb/51/8b08617af3ad95e33ce6d7dd2c99ed6c8298f7fb131636303956be022e25/pillow-12.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c", size = 6344383, upload-time = "2026-07-01T11:54:15.756Z" },+ { url = "https://files.pythonhosted.org/packages/1d/72/cf78ac9780bb93c28328f408973845a309d4d145041665f734572ced1b52/pillow-12.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df", size = 7052934, upload-time = "2026-07-01T11:54:17.721Z" },+ { url = "https://files.pythonhosted.org/packages/20/20/25e0f4dc178a6bc0696793720055519a0de89e7661dae886992decbd2f81/pillow-12.3.0-cp312-cp312-win32.whl", hash = "sha256:dbce0b29841537a2fa4a214c2bbf14de3587c9680caa9b4e217568472490b28f", size = 6472684, upload-time = "2026-07-01T11:54:19.839Z" },+ { url = "https://files.pythonhosted.org/packages/45/89/da2f7971a317f83d807fdd4065c0af40208e59e692cc43d315a71a0e96d1/pillow-12.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:a2b55dd6b2a4c4b7d87ffa56bdb33fdc5fdb9a462173861a7bc097f17d91cb09", size = 7227137, upload-time = "2026-07-01T11:54:22.025Z" },+ { url = "https://files.pythonhosted.org/packages/de/47/4845a0a6c0dbf1db8456bd9fc791f13c5ced7ced20606d08a0aacfd25b49/pillow-12.3.0-cp312-cp312-win_arm64.whl", hash = "sha256:331b624368d4f1d069149002f25f44bc61c8919ce8ddb3c45bdad8f6e2d89510", size = 2568267, upload-time = "2026-07-01T11:54:24.051Z" },+ { url = "https://files.pythonhosted.org/packages/9d/ac/31fb64e1e7efb5a4b50cd3d92049ba89ac6e4d8d3bb6a74e15048ca3353e/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89", size = 4161684, upload-time = "2026-07-01T11:54:25.934Z" },+ { url = "https://files.pythonhosted.org/packages/87/b4/9805e23d2b4d77842b468513841fda254ee42f0289d25088340e4ff46e2d/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace", size = 4255487, upload-time = "2026-07-01T11:54:27.935Z" },+ { url = "https://files.pythonhosted.org/packages/df/39/ecf519435a200c693fe053a6ee4d835b41cf963a4dfc2551c4e637cb2a71/pillow-12.3.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec", size = 3696433, upload-time = "2026-07-01T11:54:29.813Z" },+ { url = "https://files.pythonhosted.org/packages/42/92/2fc3ffad878ae8dd5469ec1bc8eb83b71f48e13efdf68f02709003982a32/pillow-12.3.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66", size = 5345889, upload-time = "2026-07-01T11:54:31.97Z" },+ { url = "https://files.pythonhosted.org/packages/10/76/8803c13605b763d33d156c4678fc77f8443389c0c51c8aef707bb02015f4/pillow-12.3.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35", size = 4780109, upload-time = "2026-07-01T11:54:34.026Z" },+ { url = "https://files.pythonhosted.org/packages/1f/01/e18aff37cb0b4aac47ac90f016d347a49aca667ef97f190b06ac2aabc928/pillow-12.3.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65", size = 6263736, upload-time = "2026-07-01T11:54:36.131Z" },+ { url = "https://files.pythonhosted.org/packages/f7/62/de5bdd77d935331f4f802edc11e4d82950f642caad6cb2f949837b8560e2/pillow-12.3.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3", size = 6937129, upload-time = "2026-07-01T11:54:38.216Z" },+ { url = "https://files.pythonhosted.org/packages/70/4d/105627a13300c5e0df1d174230b32fd1273062c96f7745fd552b945d1e1d/pillow-12.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a", size = 6339562, upload-time = "2026-07-01T11:54:40.354Z" },+ { url = "https://files.pythonhosted.org/packages/6b/1d/f13de01a553988ab895ba1c722e06cf3144d4f57656fd5b81b6d881f1179/pillow-12.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e", size = 7049439, upload-time = "2026-07-01T11:54:42.489Z" },+ { url = "https://files.pythonhosted.org/packages/c9/f9/066794cca041b969964f779ee5fa66a9498bbf34248ac39c5d7954e4198f/pillow-12.3.0-cp313-cp313-win32.whl", hash = "sha256:a876864214e136f0eb367788dbd7df045f4806801518e2cfe9e13229cfe06d8f", size = 6473287, upload-time = "2026-07-01T11:54:44.9Z" },+ { url = "https://files.pythonhosted.org/packages/a6/9b/7a58e61d62be561da3a356fe2384d4059a6345fc130e23ef1c36a5b81d24/pillow-12.3.0-cp313-cp313-win_amd64.whl", hash = "sha256:1cca606cd25738df4ed873d5ad46bbdb3d83b5cbca291f6b4ff13a4df6b0bbe8", size = 7239691, upload-time = "2026-07-01T11:54:47.141Z" },+ { url = "https://files.pythonhosted.org/packages/aa/b0/c4ed4f0ef8f8fa5ee8351537db6650bb8189f7e118842978dd6589065692/pillow-12.3.0-cp313-cp313-win_arm64.whl", hash = "sha256:b629de27fda84b42cde7edef0d85f13b958b47f6e9bbcbba9b673c562a89bd8b", size = 2568185, upload-time = "2026-07-01T11:54:49.137Z" },+ { url = "https://files.pythonhosted.org/packages/dc/01/001f65b68192f0228cc1dbbc8d2530ab5d58b61037ba0587f946fea607cd/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330", size = 4161736, upload-time = "2026-07-01T11:54:51.156Z" },+ { url = "https://files.pythonhosted.org/packages/1a/d2/0219746d0fd16fc8a84498e79452375be3797d3ce4044596ce565164b84f/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217", size = 4255435, upload-time = "2026-07-01T11:54:53.414Z" },+ { url = "https://files.pythonhosted.org/packages/c8/02/8d0bc62ef0302318c46ff2a512822d2610e81c7aa46c9b3abe6cbaca5ad0/pillow-12.3.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930", size = 3696262, upload-time = "2026-07-01T11:54:55.739Z" },+ { url = "https://files.pythonhosted.org/packages/85/e2/73c77d218410b14f5f2d565e8a998d5317b7b9c75368d29985139f7a46f0/pillow-12.3.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8", size = 5350344, upload-time = "2026-07-01T11:54:57.657Z" },+ { url = "https://files.pythonhosted.org/packages/c7/da/32c752228ae345f489e3a42499d817b6c3996da7e8a3bc7a04fc806b243b/pillow-12.3.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0", size = 4780131, upload-time = "2026-07-01T11:54:59.713Z" },+ { url = "https://files.pythonhosted.org/packages/b1/9d/8b2c807dbef61a5197c047afe99823787eb66f63daf9fb2432f91d6f0462/pillow-12.3.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321", size = 6263757, upload-time = "2026-07-01T11:55:01.778Z" },+ { url = "https://files.pythonhosted.org/packages/5c/44/c85361f65dbe00eea8576ee467c768d25129989efb76e94f205e9ca9bb46/pillow-12.3.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b", size = 6936962, upload-time = "2026-07-01T11:55:03.93Z" },+ { url = "https://files.pythonhosted.org/packages/18/7e/e483414b35800b86b6f08dbbc7803fb5cd52c4d6f897f47d53ea2c7e6f65/pillow-12.3.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198", size = 6339171, upload-time = "2026-07-01T11:55:05.989Z" },+ { url = "https://files.pythonhosted.org/packages/f0/f4/68c491844841ede6bed70189546b3ee9731cf9f2cbad396faff5e1ccba45/pillow-12.3.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130", size = 7048116, upload-time = "2026-07-01T11:55:08.131Z" },+ { url = "https://files.pythonhosted.org/packages/a3/34/77f3f793fed8efc7d243f21b33c5a3f0d1c97ee70346d3db855587e155ff/pillow-12.3.0-cp314-cp314-win32.whl", hash = "sha256:af8d94b0db561cf68b88a267c5c44b49e134f525d0dc2cb7ed413a66bc23559a", size = 6467209, upload-time = "2026-07-01T11:55:10.408Z" },+ { url = "https://files.pythonhosted.org/packages/f1/e0/492879f69d94f91f60fc8cd05ba03650e9520afebb2fb7aa12777d7c7f38/pillow-12.3.0-cp314-cp314-win_amd64.whl", hash = "sha256:fdafc9cce40277e0f7a0feabce0ee50dd2fa1800f3b38015e51296b5e814048d", size = 7237707, upload-time = "2026-07-01T11:55:12.745Z" },+ { url = "https://files.pythonhosted.org/packages/c9/ac/6b11f2875f1c2ac040d84e1bbf9cf22a88038f901ca1037898b280b38365/pillow-12.3.0-cp314-cp314-win_arm64.whl", hash = "sha256:e91206ee562682b51b98ef4b26a6ef48fd84e15fd4c4bc5ec768eb641d206838", size = 2565995, upload-time = "2026-07-01T11:55:14.736Z" },+ { url = "https://files.pythonhosted.org/packages/52/69/c2208e56af9bfc1913afb24020297a691eb1d4ef688474c8a04913f65e04/pillow-12.3.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e", size = 5352503, upload-time = "2026-07-01T11:55:17.076Z" },+ { url = "https://files.pythonhosted.org/packages/07/70/e5686d753e898a45d778ff1718dba8516ead6ab6b95d85fc8c4b70650cf2/pillow-12.3.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17", size = 4782956, upload-time = "2026-07-01T11:55:19.448Z" },+ { url = "https://files.pythonhosted.org/packages/d5/37/25c6692f06927ee973ff18c8d9ee98ad0b4d84ee67a09610c2dd1447958e/pillow-12.3.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385", size = 6322855, upload-time = "2026-07-01T11:55:21.613Z" },+ { url = "https://files.pythonhosted.org/packages/cc/91/420637fcb8f1bc11029e403b4538e6694744428d8246118e45719f944556/pillow-12.3.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c", size = 6989642, upload-time = "2026-07-01T11:55:24.006Z" },+ { url = "https://files.pythonhosted.org/packages/10/08/b94d7811281ccf0d143a1cf768d1c49e1e54af63e7b708ab2ee3eb87face/pillow-12.3.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d", size = 6391281, upload-time = "2026-07-01T11:55:26.252Z" },+ { url = "https://files.pythonhosted.org/packages/d2/87/24233f785f55474dc02ce3e739c5528a77e3a862e9333d1dd7a25cc31f70/pillow-12.3.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931", size = 7096716, upload-time = "2026-07-01T11:55:28.318Z" },+ { url = "https://files.pythonhosted.org/packages/23/26/fcb2f6e37175b04f53570b59937867e2b80ee1685e744023153028fc14f9/pillow-12.3.0-cp314-cp314t-win32.whl", hash = "sha256:4b0a7fe987b14c31ebda6083f74f22b561fd3739bc0ac51e019622e3d72668c7", size = 6474125, upload-time = "2026-07-01T11:55:30.956Z" },+ { url = "https://files.pythonhosted.org/packages/90/de/3634abee5f1c9e13c56787b7d5517b0ba8d6de51700b95578cf338349c9f/pillow-12.3.0-cp314-cp314t-win_amd64.whl", hash = "sha256:962864dc93511324d51ddbb5b9f8731bf71675b93ca612a07441896f4688fb8c", size = 7242939, upload-time = "2026-07-01T11:55:34.044Z" },+ { url = "https://files.pythonhosted.org/packages/ce/2a/fd13f8eb24de5714a6eb444a3d67e2842c6c576e159a43793adf23051351/pillow-12.3.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0740a512dc522224c77d9aa5a8d70d8b7d73fb91f2c21125d8d025d3b8990e45", size = 2567506, upload-time = "2026-07-01T11:55:35.988Z" },+ { url = "https://files.pythonhosted.org/packages/5d/dc/8fdce34ec725a33c81c6ba122b904d6b9024e50ea9ac7bede62fab54506c/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139", size = 4162063, upload-time = "2026-07-01T11:55:37.941Z" },+ { url = "https://files.pythonhosted.org/packages/76/66/2044b9a63d3b84ff048228dfcb7cd9bf0df983e8470971bf7d4c57b693de/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402", size = 4255549, upload-time = "2026-07-01T11:55:40.022Z" },+ { url = "https://files.pythonhosted.org/packages/52/7e/1f67e6f4ece6b582ee4b539decbcc9f848dc245a93ed8cd7338bafef72f1/pillow-12.3.0-cp315-cp315-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c", size = 3696331, upload-time = "2026-07-01T11:55:41.98Z" },+ { url = "https://files.pythonhosted.org/packages/12/40/d306fc2c8e4d45d7f175c77edca7063be7b86fe7fe6e68f4353bf71d808c/pillow-12.3.0-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:dc624f6bc473dacdf7ef7eb8678d0d08edf15cd94fad6ae5c7d6cc67a4e4902f", size = 5350370, upload-time = "2026-07-01T11:55:44.028Z" },+ { url = "https://files.pythonhosted.org/packages/dd/44/668fb1437e8ce420f62d6106eb66e44a5971602a4d794615bdf79315d82d/pillow-12.3.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:71d6097b330eea8fd15097780c8e89cb1a8ce7838669f48c5bacd6f663dd4701", size = 4780147, upload-time = "2026-07-01T11:55:46.073Z" },+ { url = "https://files.pythonhosted.org/packages/0c/08/93fa2e70e30a2d81547e481b6ee2bb9522117221fb1e0ce4b5df70967677/pillow-12.3.0-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:28ce87c5ab450a9dd970b52e5aca5fe63ed432d18a2eaddd1979a00a1ba24ace", size = 6273659, upload-time = "2026-07-01T11:55:48.264Z" },+ { url = "https://files.pythonhosted.org/packages/f8/6d/043e96ff814fc31a33077e4cba86082167db520c93632afdf2042febbb0c/pillow-12.3.0-cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6b02afb9b97f65fbca5f31db6a2a3ba21aa93030225f150fa3f249717e938fb4", size = 6947439, upload-time = "2026-07-01T11:55:50.503Z" },+ { url = "https://files.pythonhosted.org/packages/af/92/ba71d2ee2ac0edf3fa33bd9d5ee9ee080da70b1766f3ca3934f9938ddac9/pillow-12.3.0-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:1182d52bc2d5e5d7d0949503aa7e36d12f42205dc287e4883f407b1988820d39", size = 6353577, upload-time = "2026-07-01T11:55:52.697Z" },+ { url = "https://files.pythonhosted.org/packages/0f/ce/e63064e2122923ff687c8ad792d0d736a7b3920a56a46982e81a7fdd25d6/pillow-12.3.0-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:e795b7eb908249c4e43c7c99fac7c2c75dab0c43566e37db472a355f63693d71", size = 7060394, upload-time = "2026-07-01T11:55:55.149Z" },+ { url = "https://files.pythonhosted.org/packages/54/76/a09cc3ccc8d773a7283d34c38bec1708f9e3cc932093cbc4c5e71ac4060b/pillow-12.3.0-cp315-cp315-win32.whl", hash = "sha256:57b3d78c95ba9059768b10e28b813002261d3f3dfc55cc48b0c988f625175827", size = 6467375, upload-time = "2026-07-01T11:55:57.769Z" },+ { url = "https://files.pythonhosted.org/packages/3e/03/1846c49ba3b1d5550392a4bbd06d6fb4578e1cd91a803198b5c90f5f7d53/pillow-12.3.0-cp315-cp315-win_amd64.whl", hash = "sha256:fa4ecea169a355be7a3ade2c783e2ed12f0e40d2c5621cda8b3297faf7fbb9f5", size = 7237048, upload-time = "2026-07-01T11:55:59.975Z" },+ { url = "https://files.pythonhosted.org/packages/fb/bb/89f35dcc79610423f9f195504d7def7f0d1416a711541b42867e25fe3412/pillow-12.3.0-cp315-cp315-win_arm64.whl", hash = "sha256:877c3f311ff35410f690861c4409e7ccbf0cd2f878e50628a28e5a0bb689e658", size = 2566006, upload-time = "2026-07-01T11:56:02.143Z" },+ { url = "https://files.pythonhosted.org/packages/30/88/707027ba09942dfa2c28759b5c222d769290a41c6d20ea60ec250801941f/pillow-12.3.0-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:e9871b1ffbfa9656b60aeee92ed5136a5742696006fa322b29ea3d8da0ecc9cf", size = 5352509, upload-time = "2026-07-01T11:56:04.2Z" },+ { url = "https://files.pythonhosted.org/packages/b0/6d/00352fa25332c2569cd387851f568cc5a4b75a9adbfb37ac4fbce4c02eec/pillow-12.3.0-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:53aa02d20d10c3d814d536aa4e5ac9b84ca0ff5a88377963b085ad6822f93e64", size = 4783167, upload-time = "2026-07-01T11:56:06.631Z" },+ { url = "https://files.pythonhosted.org/packages/13/4f/9e049dfa21af7c22427275720e2490267ba8138120add5c4c574deb69782/pillow-12.3.0-cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:446c34dcc4324b084a53b705127dc15717b22c5e140ae0a3c38349d4efec071e", size = 6329237, upload-time = "2026-07-01T11:56:08.868Z" },+ { url = "https://files.pythonhosted.org/packages/36/16/cf6eeaae8d0fce8dd390a33437cf68c5d5bd73834a2bc6e2f14efda0ab45/pillow-12.3.0-cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cf1845d02ad822a369a49f2bb9345b1614744267682e7a03527dc3bf6eea1777", size = 6997047, upload-time = "2026-07-01T11:56:11.379Z" },+ { url = "https://files.pythonhosted.org/packages/1e/69/dbf769bdd55f48bf5733cac28edc6364ffaa072ec9ba336266e4fe66be55/pillow-12.3.0-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:186941b6aef820ad110fb01fb06eb925374dc3a21b17e37ec9a53b250c6fe2d1", size = 6400440, upload-time = "2026-07-01T11:56:13.908Z" },+ { url = "https://files.pythonhosted.org/packages/a0/e1/ffc9cfc2eea0d178da8018e18e959301ad9d6bc9f3edb7181e748a474b97/pillow-12.3.0-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:f13c32a3abd6079a66d9526e18dad9b6d280384d49d7c54040cd57b6424041d9", size = 7105895, upload-time = "2026-07-01T11:56:16.575Z" },+ { url = "https://files.pythonhosted.org/packages/18/f0/a5595c1e8c3ae44b9828cb2f0fa8155e5095ef04d6327b8f61cf44a3df85/pillow-12.3.0-cp315-cp315t-win32.whl", hash = "sha256:1657923d2d45afb66526e5b933e5b3052e6bdea196c90d3abb2424e18c77dae8", size = 6474384, upload-time = "2026-07-01T11:56:18.855Z" },+ { url = "https://files.pythonhosted.org/packages/e4/04/62bcd9f844984c5938d3b05264a61d797a29d3e0812341a8204af70bbdee/pillow-12.3.0-cp315-cp315t-win_amd64.whl", hash = "sha256:8cd2f7bdda092d99c9fc2fb7391354f306d01443d22785d0cbfafa2e2c8bb418", size = 7243537, upload-time = "2026-07-01T11:56:21.214Z" },+ { url = "https://files.pythonhosted.org/packages/3d/68/1f3066acedf37673694a7141381d8f811ae97f30d34413d236abe7d489f1/pillow-12.3.0-cp315-cp315t-win_arm64.whl", hash = "sha256:06ff022112bc9cbf83b60f8e028d94ad87b60621706487e65f673de61610ab59", size = 2567491, upload-time = "2026-07-01T11:56:23.506Z" },+ { url = "https://files.pythonhosted.org/packages/75/18/2e8b40223153ccbc60df07f9e8928dc0c76202aa4e55ae9f53962b6510d6/pillow-12.3.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:b3c777e849237620b022f7f297dd67705f9f5cf1685f09f02e46f93e92725468", size = 5302510, upload-time = "2026-07-01T11:56:25.736Z" },+ { url = "https://files.pythonhosted.org/packages/46/3e/51fabf59d5ab801ceab709453d3ab6b180083496579549de4c45ced6528a/pillow-12.3.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:b343699e8308bdc51978310e1c959c584e7869cc8c40780058c87da7781a1e94", size = 4736058, upload-time = "2026-07-01T11:56:28.041Z" },+ { url = "https://files.pythonhosted.org/packages/bf/20/22fe9384b7949e25fb1293bcfc84fb82590ff4ea6b37c95b24d26d793d86/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbd139c8447d25dd750ab79ee274cc5e1fe80fc56340ab10b18a195e1b6eca3e", size = 5237776, upload-time = "2026-07-01T11:56:30.263Z" },+ { url = "https://files.pythonhosted.org/packages/08/14/f6ba68107680ffa74b39985f3f30884e41318fbc4250caa423c79b4788bb/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e7e480451b9fa137494bccd3a7d69adbe8ac65a87d97be61e11f1b1050a5bac3", size = 5860358, upload-time = "2026-07-01T11:56:32.68Z" },+ { url = "https://files.pythonhosted.org/packages/36/54/0169bc772ec491108b62f644f8ecf1fe5d8ae5ebafde2ee2142210166903/pillow-12.3.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:04f01d28a6aaff387bf842a13be313df23ba0597a44f1a976c9feb3c6ff4711a", size = 7231786, upload-time = "2026-07-01T11:56:35.046Z" },+]++[[package]]+name = "pluggy"+version = "1.6.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },+]++[[package]]+name = "pygame"+version = "2.6.1"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/49/cc/08bba60f00541f62aaa252ce0cfbd60aebd04616c0b9574f755b583e45ae/pygame-2.6.1.tar.gz", hash = "sha256:56fb02ead529cee00d415c3e007f75e0780c655909aaa8e8bf616ee09c9feb1f", size = 14808125, upload-time = "2024-09-29T13:41:34.698Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/c4/ca/8f367cb9fe734c4f6f6400e045593beea2635cd736158f9fabf58ee14e3c/pygame-2.6.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:20349195326a5e82a16e351ed93465a7845a7e2a9af55b7bc1b2110ea3e344e1", size = 13113753, upload-time = "2024-09-29T14:26:13.751Z" },+ { url = "https://files.pythonhosted.org/packages/83/47/6edf2f890139616b3219be9cfcc8f0cb8f42eb15efd59597927e390538cb/pygame-2.6.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:f3935459109da4bb0b3901da9904f0a3e52028a3332a355d298b1673a334cf21", size = 12378146, upload-time = "2024-09-29T14:26:22.456Z" },+ { url = "https://files.pythonhosted.org/packages/00/9e/0d8aa8cf93db2d2ee38ebaf1c7b61d0df36ded27eb726221719c150c673d/pygame-2.6.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c31dbdb5d0217f32764797d21c2752e258e5fb7e895326538d82b5f75a0cd856", size = 13611760, upload-time = "2024-09-29T11:10:47.317Z" },+ { url = "https://files.pythonhosted.org/packages/d7/9e/d06adaa5cc65876bcd7a24f59f67e07f7e4194e6298130024ed3fb22c456/pygame-2.6.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:173badf82fa198e6888017bea40f511cb28e69ecdd5a72b214e81e4dcd66c3b1", size = 14298054, upload-time = "2024-09-29T11:39:53.891Z" },+ { url = "https://files.pythonhosted.org/packages/7a/a1/9ae2852ebd3a7cc7d9ae7ff7919ab983e4a5c1b7a14e840732f23b2b48f6/pygame-2.6.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ce8cc108b92de9b149b344ad2e25eedbe773af0dc41dfb24d1f07f679b558c60", size = 13977107, upload-time = "2024-09-29T11:39:56.831Z" },+ { url = "https://files.pythonhosted.org/packages/31/df/6788fd2e9a864d0496a77670e44a7c012184b7a5382866ab0e60c55c0f28/pygame-2.6.1-cp311-cp311-win32.whl", hash = "sha256:811e7b925146d8149d79193652cbb83e0eca0aae66476b1cb310f0f4226b8b5c", size = 10250863, upload-time = "2024-09-29T11:44:48.199Z" },+ { url = "https://files.pythonhosted.org/packages/d2/55/ca3eb851aeef4f6f2e98a360c201f0d00bd1ba2eb98e2c7850d80aabc526/pygame-2.6.1-cp311-cp311-win_amd64.whl", hash = "sha256:91476902426facd4bb0dad4dc3b2573bc82c95c71b135e0daaea072ed528d299", size = 10622016, upload-time = "2024-09-29T12:17:01.545Z" },+ { url = "https://files.pythonhosted.org/packages/92/16/2c602c332f45ff9526d61f6bd764db5096ff9035433e2172e2d2cadae8db/pygame-2.6.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:4ee7f2771f588c966fa2fa8b829be26698c9b4836f82ede5e4edc1a68594942e", size = 13118279, upload-time = "2024-09-29T14:26:30.427Z" },+ { url = "https://files.pythonhosted.org/packages/cd/53/77ccbc384b251c6e34bfd2e734c638233922449a7844e3c7a11ef91cee39/pygame-2.6.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:c8040ea2ab18c6b255af706ec01355c8a6b08dc48d77fd4ee783f8fc46a843bf", size = 12384524, upload-time = "2024-09-29T14:26:49.996Z" },+ { url = "https://files.pythonhosted.org/packages/06/be/3ed337583f010696c3b3435e89a74fb29d0c74d0931e8f33c0a4246307a9/pygame-2.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c47a6938de93fa610accd4969e638c2aebcb29b2fca518a84c3a39d91ab47116", size = 13587123, upload-time = "2024-09-29T11:10:50.072Z" },+ { url = "https://files.pythonhosted.org/packages/fd/ca/b015586a450db59313535662991b34d24c1f0c0dc149cc5f496573900f4e/pygame-2.6.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:33006f784e1c7d7e466fcb61d5489da59cc5f7eb098712f792a225df1d4e229d", size = 14275532, upload-time = "2024-09-29T11:39:59.356Z" },+ { url = "https://files.pythonhosted.org/packages/b9/f2/d31e6ad42d657af07be2ffd779190353f759a07b51232b9e1d724f2cda46/pygame-2.6.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1206125f14cae22c44565c9d333607f1d9f59487b1f1432945dfc809aeaa3e88", size = 13952653, upload-time = "2024-09-29T11:40:01.781Z" },+ { url = "https://files.pythonhosted.org/packages/f3/42/8ea2a6979e6fa971702fece1747e862e2256d4a8558fe0da6364dd946c53/pygame-2.6.1-cp312-cp312-win32.whl", hash = "sha256:84fc4054e25262140d09d39e094f6880d730199710829902f0d8ceae0213379e", size = 10252421, upload-time = "2024-09-29T11:14:26.877Z" },+ { url = "https://files.pythonhosted.org/packages/5f/90/7d766d54bb95939725e9a9361f9c06b0cfbe3fe100aa35400f0a461a278a/pygame-2.6.1-cp312-cp312-win_amd64.whl", hash = "sha256:3a9e7396be0d9633831c3f8d5d82dd63ba373ad65599628294b7a4f8a5a01a65", size = 10624591, upload-time = "2024-09-29T11:52:54.489Z" },+ { url = "https://files.pythonhosted.org/packages/e1/91/718acf3e2a9d08a6ddcc96bd02a6f63c99ee7ba14afeaff2a51c987df0b9/pygame-2.6.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ae6039f3a55d800db80e8010f387557b528d34d534435e0871326804df2a62f2", size = 13090765, upload-time = "2024-09-29T14:27:02.377Z" },+ { url = "https://files.pythonhosted.org/packages/0e/c6/9cb315de851a7682d9c7568a41ea042ee98d668cb8deadc1dafcab6116f0/pygame-2.6.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2a3a1288e2e9b1e5834e425bedd5ba01a3cd4902b5c2bff8ed4a740ccfe98171", size = 12381704, upload-time = "2024-09-29T14:27:10.228Z" },+ { url = "https://files.pythonhosted.org/packages/9f/8f/617a1196e31ae3b46be6949fbaa95b8c93ce15e0544266198c2266cc1b4d/pygame-2.6.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:27eb17e3dc9640e4b4683074f1890e2e879827447770470c2aba9f125f74510b", size = 13581091, upload-time = "2024-09-29T11:30:27.653Z" },+ { url = "https://files.pythonhosted.org/packages/3b/87/2851a564e40a2dad353f1c6e143465d445dab18a95281f9ea458b94f3608/pygame-2.6.1-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4c1623180e70a03c4a734deb9bac50fc9c82942ae84a3a220779062128e75f3b", size = 14273844, upload-time = "2024-09-29T11:40:04.138Z" },+ { url = "https://files.pythonhosted.org/packages/85/b5/aa23aa2e70bcba42c989c02e7228273c30f3b44b9b264abb93eaeff43ad7/pygame-2.6.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ef07c0103d79492c21fced9ad68c11c32efa6801ca1920ebfd0f15fb46c78b1c", size = 13951197, upload-time = "2024-09-29T11:40:06.785Z" },+ { url = "https://files.pythonhosted.org/packages/a6/06/29e939b34d3f1354738c7d201c51c250ad7abefefaf6f8332d962ff67c4b/pygame-2.6.1-cp313-cp313-win32.whl", hash = "sha256:3acd8c009317190c2bfd81db681ecef47d5eb108c2151d09596d9c7ea9df5c0e", size = 10249309, upload-time = "2024-09-29T11:10:23.329Z" },+ { url = "https://files.pythonhosted.org/packages/7e/11/17f7f319ca91824b86557e9303e3b7a71991ef17fd45286bf47d7f0a38e6/pygame-2.6.1-cp313-cp313-win_amd64.whl", hash = "sha256:813af4fba5d0b2cb8e58f5d95f7910295c34067dcc290d34f1be59c48bd1ea6a", size = 10620084, upload-time = "2024-09-29T11:48:51.587Z" },+]++[[package]]+name = "pygments"+version = "2.21.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/49/2e/ced460408999b33da6b31b0021b0f37d329e202d4169aeb164493778f25b/pygments-2.21.0.tar.gz", hash = "sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c", size = 5005329, upload-time = "2026-08-17T08:02:48.824Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/71/46/17f022dd3e953bf20a04a028a21ec746d942f8d2af30fa0f124fa0e6a684/pygments-2.21.0-py3-none-any.whl", hash = "sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9", size = 1250147, upload-time = "2026-08-17T08:02:44.912Z" },+]++[[package]]+name = "pytest"+version = "9.1.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "colorama", marker = "sys_platform == 'win32'" },+ { name = "iniconfig" },+ { name = "packaging" },+ { name = "pluggy" },+ { name = "pygments" },+]+sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },+]++[[package]]+name = "pyyaml"+version = "6.0.3"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/6d/16/a95b6757765b7b031c9374925bb718d55e0a9ba8a1b6a12d25962ea44347/pyyaml-6.0.3-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e", size = 185826, upload-time = "2025-09-25T21:31:58.655Z" },+ { url = "https://files.pythonhosted.org/packages/16/19/13de8e4377ed53079ee996e1ab0a9c33ec2faf808a4647b7b4c0d46dd239/pyyaml-6.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824", size = 175577, upload-time = "2025-09-25T21:32:00.088Z" },+ { url = "https://files.pythonhosted.org/packages/0c/62/d2eb46264d4b157dae1275b573017abec435397aa59cbcdab6fc978a8af4/pyyaml-6.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c", size = 775556, upload-time = "2025-09-25T21:32:01.31Z" },+ { url = "https://files.pythonhosted.org/packages/10/cb/16c3f2cf3266edd25aaa00d6c4350381c8b012ed6f5276675b9eba8d9ff4/pyyaml-6.0.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00", size = 882114, upload-time = "2025-09-25T21:32:03.376Z" },+ { url = "https://files.pythonhosted.org/packages/71/60/917329f640924b18ff085ab889a11c763e0b573da888e8404ff486657602/pyyaml-6.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d", size = 806638, upload-time = "2025-09-25T21:32:04.553Z" },+ { url = "https://files.pythonhosted.org/packages/dd/6f/529b0f316a9fd167281a6c3826b5583e6192dba792dd55e3203d3f8e655a/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a", size = 767463, upload-time = "2025-09-25T21:32:06.152Z" },+ { url = "https://files.pythonhosted.org/packages/f2/6a/b627b4e0c1dd03718543519ffb2f1deea4a1e6d42fbab8021936a4d22589/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4", size = 794986, upload-time = "2025-09-25T21:32:07.367Z" },+ { url = "https://files.pythonhosted.org/packages/45/91/47a6e1c42d9ee337c4839208f30d9f09caa9f720ec7582917b264defc875/pyyaml-6.0.3-cp311-cp311-win32.whl", hash = "sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b", size = 142543, upload-time = "2025-09-25T21:32:08.95Z" },+ { url = "https://files.pythonhosted.org/packages/da/e3/ea007450a105ae919a72393cb06f122f288ef60bba2dc64b26e2646fa315/pyyaml-6.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf", size = 158763, upload-time = "2025-09-25T21:32:09.96Z" },+ { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" },+ { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" },+ { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" },+ { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" },+ { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" },+ { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" },+ { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" },+ { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" },+ { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" },+ { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" },+ { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" },+ { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" },+ { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" },+ { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" },+ { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" },+ { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" },+ { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" },+ { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" },+ { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" },+ { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" },+ { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" },+ { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" },+ { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" },+ { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" },+ { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" },+ { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" },+ { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" },+ { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" },+ { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" },+ { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" },+ { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" },+ { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" },+ { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" },+ { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" },+ { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" },+ { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" },+ { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" },+ { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" },+]++[[package]]+name = "regex"+version = "2026.9.29"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/fc/f2/af1da9d3ceed77bfcdce40427d49ba0be94e4fe84245e3bfef68c10e75b6/regex-2026.9.29.tar.gz", hash = "sha256:8b5fcc4771732191b2b7d1dd68d8f0353f47f8d90b6150f6dce58bf1112442cb", size = 419199, upload-time = "2026-09-29T00:49:58.298Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/e8/6b/6dea87689c3a06a6e79d254bf824e6f3e3d724b5ba027c6112559aa6cd2c/regex-2026.9.29-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:6abb75ab16bc3281714a5b99548a2225db70dba1f995f6d7f7419b76eb5a8fbe", size = 495413, upload-time = "2026-09-29T00:46:14.51Z" },+ { url = "https://files.pythonhosted.org/packages/3a/a5/0c791a0e83ad1013d262c13247c4c77e0f4a8d05bdc167df96aba6681c0d/regex-2026.9.29-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:b7b893976e7fe42053da64f2aa27239c24252fd2ec6df471e1be197c0addc3b1", size = 295145, upload-time = "2026-09-29T00:46:16.292Z" },+ { url = "https://files.pythonhosted.org/packages/b1/07/9bf3607d8d13a12e436ab9d63f9791e10706827d535695b23964ad79fd79/regex-2026.9.29-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:066d0e3dbfdd739bce2bf8c2a41dd16f73e3d8adc2eb06dd803a36a307f56075", size = 292123, upload-time = "2026-09-29T00:46:17.646Z" },+ { url = "https://files.pythonhosted.org/packages/64/6b/32c2e6fc617e1d3f247e250fea31a9a35b1265bd32f585968aa13b9999b9/regex-2026.9.29-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7020ed44df30b3aa492c00ee3b52d0548c1f30c2c6c5bb13ae897680900d3413", size = 799537, upload-time = "2026-09-29T00:46:18.976Z" },+ { url = "https://files.pythonhosted.org/packages/bf/72/f041177f3c7a4606f7c81a95fe7eea03e2a0c4e8bff9e439a01432cbc9f2/regex-2026.9.29-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ae4613d7d9dda60fcba95f846cc6f808017f1843f392cf9daad14a6534493d71", size = 873255, upload-time = "2026-09-29T00:46:20.684Z" },+ { url = "https://files.pythonhosted.org/packages/d0/4e/a78948e11dd715e0e46716c2e0f3404b3fe6a44e2a2e9abdc7d965cab2b3/regex-2026.9.29-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:bec37990e3d6121f29ecfb594bd8f1bf009e9f7926daba2e50e3b27d3892a783", size = 914570, upload-time = "2026-09-29T00:46:22.599Z" },+ { url = "https://files.pythonhosted.org/packages/8a/70/aa08d1d2b294894b365e5f8ba5380fe3f8546acdb81f10639dfd74209c37/regex-2026.9.29-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:612b709381c0355b70d89cdb51b7f670591ed5cbbc0e3b5337488019dc667b65", size = 804978, upload-time = "2026-09-29T00:46:23.981Z" },+ { url = "https://files.pythonhosted.org/packages/21/32/1b03534c4715aca3b564416d28d518083ed4dab3bc913267600d2256140d/regex-2026.9.29-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a760da040b47767b4b873adfb7c3b691e9ba2fc60f113f9d0b88f1a62f323e85", size = 779297, upload-time = "2026-09-29T00:46:25.318Z" },+ { url = "https://files.pythonhosted.org/packages/76/a7/378f6f558d9e4444af315a307c5953565a511d1e3666f1bb7bdc82012b6b/regex-2026.9.29-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:49ee178ca31c94621294bf9b8b676a92a2e6bba8af0529591753719e57edb621", size = 786961, upload-time = "2026-09-29T00:46:26.963Z" },+ { url = "https://files.pythonhosted.org/packages/59/13/79f0b1846f5f342f92ddbd4b27b18bcb86da96d902c1a0be26520bde98d7/regex-2026.9.29-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:5eeb8edc6110d9194a4d0d54610f64c37a31c605b5dbb7e407fc6ec7fa34a4a1", size = 863234, upload-time = "2026-09-29T00:46:28.58Z" },+ { url = "https://files.pythonhosted.org/packages/97/19/05af70dec9f2eed6ba34e08d2dcc6a48e7ae5e307659d5fe4201a5d7bbee/regex-2026.9.29-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:ccb64d887a9db1cd76dbc0f92051a1a478a2a67e7f56c62d915cb881d7734704", size = 766487, upload-time = "2026-09-29T00:46:29.941Z" },+ { url = "https://files.pythonhosted.org/packages/01/e1/9c7486d4afe8fdd1fe0ad60139f8aa91427381f409af6a29b609d8fdcb3a/regex-2026.9.29-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:9e4482589065c8ecd761cff522dcd85f2d39e62f551e37e025d1c7d54772def3", size = 854792, upload-time = "2026-09-29T00:46:31.358Z" },+ { url = "https://files.pythonhosted.org/packages/26/c7/49d008ff5f741d9a9799d7315556f3a12b983ff0fcd2cdfb62904bedafbf/regex-2026.9.29-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:d60030baaa7bfbb02d650c126cdcddcb6e33dbff14d819434c8fa2fdcaeeeba5", size = 793050, upload-time = "2026-09-29T00:46:32.775Z" },+ { url = "https://files.pythonhosted.org/packages/cb/a1/46ba549e65562ca04608b24179b8a7bb6f146ae0e7c6d7f5e70f3339c8ba/regex-2026.9.29-cp311-cp311-win32.whl", hash = "sha256:18ae8eed4526e35bdb754d61562b90bf5c00a67fdcf3cc1380dd59597486631b", size = 268940, upload-time = "2026-09-29T00:46:34.179Z" },+ { url = "https://files.pythonhosted.org/packages/4d/4a/aab232183c70fdcf77bcf0c51819da02ec522e393e6a0bf00bcf2142e21f/regex-2026.9.29-cp311-cp311-win_amd64.whl", hash = "sha256:1043aedf5917caa861bcb25a9c11460049656bdf0017a90a309fa8f255467725", size = 280641, upload-time = "2026-09-29T00:46:35.484Z" },+ { url = "https://files.pythonhosted.org/packages/33/b1/7c05954af0f51de376df2ba97f7f78a8b79334c7e5b3d2d9f2aead1f4d3d/regex-2026.9.29-cp311-cp311-win_arm64.whl", hash = "sha256:352cf115a810b357caa35193ab656ecf5ef41056855e82f292c99e8514f8d954", size = 279411, upload-time = "2026-09-29T00:46:37.193Z" },+ { url = "https://files.pythonhosted.org/packages/84/48/3fdcde9a0baa84d7d25571223265d6e434e114763b438601d54a8028bf3e/regex-2026.9.29-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:dc79d36d0618752265f0d575915bdc5c5130ecb9c9f6b3bcefeae32e4bdfafcf", size = 497903, upload-time = "2026-09-29T00:46:38.938Z" },+ { url = "https://files.pythonhosted.org/packages/2e/1c/4ee3e97c76f53940488dfe7a7e18705e78daac8cd7fb161d246b9e328449/regex-2026.9.29-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3a21a9509d0ee88e7a70e1ad228cd2f0e0fd1e187458db132e8a8d18c97daf9d", size = 296416, upload-time = "2026-09-29T00:46:40.406Z" },+ { url = "https://files.pythonhosted.org/packages/37/14/f3f0ba083d2094392d5eabf56db5ea6ba469fd6e927afd187042054ea68a/regex-2026.9.29-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f57dc6b8fef170f105d2cf5cdce254f47b137d7755086cf7050f47e16582abba", size = 293633, upload-time = "2026-09-29T00:46:41.959Z" },+ { url = "https://files.pythonhosted.org/packages/c9/72/67e7a8ce17f1aea49df215564048efb49cc8c2b31a0e0fc30f36838f8516/regex-2026.9.29-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f93bc1c3486ef3747e07c9d7c1d0a147b8fbaab975f80e348aed6f71309dfaca", size = 805885, upload-time = "2026-09-29T00:46:43.373Z" },+ { url = "https://files.pythonhosted.org/packages/f6/78/25436bcfd4d2260b4b4090094d55d7ab53ec8a1ab4865a0b8bcb33c7d5c0/regex-2026.9.29-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9e1d3a4cb7993b708f0ada8d0c84590efd853f169e7147d2202c9da503180242", size = 878344, upload-time = "2026-09-29T00:46:45.328Z" },+ { url = "https://files.pythonhosted.org/packages/97/e6/a09ec3a23ae41d6179880e67f0aace9284b2d95f2d7b326eff203f8eec5e/regex-2026.9.29-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:dabee8f4935e731fb46b2a3091bdda0d3d94b3bbfb907d2b4f12eefce4009619", size = 919181, upload-time = "2026-09-29T00:46:47.041Z" },+ { url = "https://files.pythonhosted.org/packages/26/83/d2fbd2e4e3afb1167daa825187d196f313cbaa1a4768f311fb041bb0e3d2/regex-2026.9.29-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:39ab5894d971f9ac68baa6eca5c50387db579cfcacf36ae8df3feceb1815e6d0", size = 807783, upload-time = "2026-09-29T00:46:48.894Z" },+ { url = "https://files.pythonhosted.org/packages/46/0b/eb429a7016610d44fc89a597163f8c9127505f0d7dc724dc9effbb6a3ac0/regex-2026.9.29-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c1a9a6651197fbed6f0212591418b9def774fc3f8324f78d1bf0e6a63e5f8aa1", size = 783465, upload-time = "2026-09-29T00:46:50.64Z" },+ { url = "https://files.pythonhosted.org/packages/1b/07/58a3c0153c7476898430f6a7cf3d9062a1d17fbea4f43399ecaf411c7b4c/regex-2026.9.29-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:87fb80cbe3557e27e7b28b995c2b2eedf689b8886f941ab93e0e288f0976518a", size = 793519, upload-time = "2026-09-29T00:46:52.396Z" },+ { url = "https://files.pythonhosted.org/packages/2a/e8/161b94d39164520e21a7befe0245569bf7fda4c7cf1fc4e2df2b5def49da/regex-2026.9.29-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:3c5c2ef13797466aa64170cbb66ad98a32351dd4127694cea7199f80f213750d", size = 869293, upload-time = "2026-09-29T00:46:54.128Z" },+ { url = "https://files.pythonhosted.org/packages/8f/07/3b02ed829aa2decdc1955d222bd1e2f99d1c8bb4873bbb9a66b2f0a36bff/regex-2026.9.29-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:59b49507f47479e299a9e1bc41b5cb83a7afda0540625f1dbae886615978acbf", size = 770239, upload-time = "2026-09-29T00:46:56.106Z" },+ { url = "https://files.pythonhosted.org/packages/42/5b/ba61f6fe062eb8562e742367d177bb75370434138ef6c9d2a27114f8d613/regex-2026.9.29-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:0dd8af32e9f7b56b7f95cc1fd79b23054c3bdc172392ae560acc24d57b7ffe71", size = 861973, upload-time = "2026-09-29T00:46:57.665Z" },+ { url = "https://files.pythonhosted.org/packages/cc/27/767259b20e8a842948990f5e99138d6c077248fd42f8b5468b1d9ca4b814/regex-2026.9.29-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:db5e82ba15c142425b8406690032df89e39cca4a2e8afbbb9a3d84edc2373ac3", size = 796470, upload-time = "2026-09-29T00:46:59.236Z" },+ { url = "https://files.pythonhosted.org/packages/a0/05/2566c4ba849b68a8ab81a6bf428fa79d20aae7ddee83979103c0381df254/regex-2026.9.29-cp312-cp312-win32.whl", hash = "sha256:d0c3082bf79bcd6a614d55916590ad4b8f93200e10b97f463ea5d9d07c9b5f23", size = 269327, upload-time = "2026-09-29T00:47:01.135Z" },+ { url = "https://files.pythonhosted.org/packages/93/19/489bc8db91196381c935752df01ba3f607140daece33b78d88573f028e64/regex-2026.9.29-cp312-cp312-win_amd64.whl", hash = "sha256:fdd88ed5e20b1bcdd234421e454962c971aa44b653bdb7f1ea9ef683e90fb649", size = 280334, upload-time = "2026-09-29T00:47:04.436Z" },+ { url = "https://files.pythonhosted.org/packages/0b/47/fb88ba779d0e5e7d4b0ec1aceeb13845948a2cb876bd572a2d1dfdba090b/regex-2026.9.29-cp312-cp312-win_arm64.whl", hash = "sha256:4fe97894d1b306c919b4e50def1e6f6c522f4d03a7283811f4d108f1ce5d3ac2", size = 279614, upload-time = "2026-09-29T00:47:06.541Z" },+ { url = "https://files.pythonhosted.org/packages/79/d5/6080f7d1a6e7e36aa720f806ac93c035ba39c209ae6cc510e8ef4c0279c6/regex-2026.9.29-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:f1a0d5117230dd46b399a30a38afa44f79c99f3168988fdc4f425c3f928b39df", size = 497622, upload-time = "2026-09-29T00:47:08.251Z" },+ { url = "https://files.pythonhosted.org/packages/00/71/c87fc7a2e21a42f9d57489db32951c37eef56d153840459a80d464f0321d/regex-2026.9.29-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:f0fe9834e5aeccaf19a0d8feb296d66a24be1a7c9922002f842a682cd5abb787", size = 296281, upload-time = "2026-09-29T00:47:09.764Z" },+ { url = "https://files.pythonhosted.org/packages/11/9e/aa0f4cde3bc4688c1d58b0cd8415edd708339bc0bc401a195b0b1e8c8f0c/regex-2026.9.29-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c90fcf7804ea0a54b896ce0f2b9565350220b8d4890fd0db461a476a4c687963", size = 293472, upload-time = "2026-09-29T00:47:11.723Z" },+ { url = "https://files.pythonhosted.org/packages/90/d4/e835c487850ed922a8d6074f953b888c8ea99775c76b9ed5f8a4d72eab92/regex-2026.9.29-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e11edba5bc344a32b029a7af9d4b3173982dd79eeafa0b9dbd787364414b0509", size = 805966, upload-time = "2026-09-29T00:47:13.235Z" },+ { url = "https://files.pythonhosted.org/packages/2c/57/ba8809847fbae8d2cbc71367c6ded510a7ec88bf52493c65efc1acf4effb/regex-2026.9.29-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bb90e7177944b6684738c1fc36aabd2dd00d1de3be7dbe09f91e196f1bc0dc81", size = 878346, upload-time = "2026-09-29T00:47:14.877Z" },+ { url = "https://files.pythonhosted.org/packages/1a/52/e3da19fc3cc15ef67ab67e121e87887c3bccfdb683a7a9ec557c460ca5b7/regex-2026.9.29-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d06fcdecc10fc7954d7c8f27a03c96055fe525274dc84a7b0dbdc3d6b9e03dab", size = 919250, upload-time = "2026-09-29T00:47:16.622Z" },+ { url = "https://files.pythonhosted.org/packages/9a/8e/c1ed81f55f992f6aa0b699a592a50c1ce9e6d44ff1aee2c14c0537dcef9c/regex-2026.9.29-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d49c18f1ea294cf4adde2e5ac256e98c82ea9d708462ce4bf799dffa7cfe8a2c", size = 807902, upload-time = "2026-09-29T00:47:18.268Z" },+ { url = "https://files.pythonhosted.org/packages/ad/bc/5a6886eb470e41040e21e05b75024a18b6ebfe7ea400b72094a60f949101/regex-2026.9.29-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3e778bfccd63075167709136afbc251c1f683758d5bf49c803c60ac3f894ce6b", size = 783514, upload-time = "2026-09-29T00:47:19.921Z" },+ { url = "https://files.pythonhosted.org/packages/cb/52/6d951d453b023c6edb880f1ba474291b53b8ce1cc438b96a9db6d791d991/regex-2026.9.29-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:686ac5350fceae63830bb98805fcb8039325bf4c06d9f6f048ff65229d5bffa5", size = 793466, upload-time = "2026-09-29T00:47:21.552Z" },+ { url = "https://files.pythonhosted.org/packages/99/b9/d5a41adc08360f5eee0dc4846c578f002366947211fc8af5a69a64ee7b9f/regex-2026.9.29-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:26ec4ccce55aa533fbd603d08911b01101a8fcfec987845ac3ae2c7087b2bde3", size = 869464, upload-time = "2026-09-29T00:47:23.276Z" },+ { url = "https://files.pythonhosted.org/packages/4b/32/d76c9d91f5d798e2e9e67f6f85ec4ae35445ac425f7454797311cecb80ca/regex-2026.9.29-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:a655d34b2a6943af32401f3d94f72e9d731f6ad16285815550bf2b4ee69d420a", size = 770278, upload-time = "2026-09-29T00:47:25.193Z" },+ { url = "https://files.pythonhosted.org/packages/24/00/aeebdb540c620a0f7317f6d6fad80a47729ecf0599a24b5c34ec155351f5/regex-2026.9.29-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:0c992c19cd45058a4b92f68f139c93db168b48fb1f322c9a7cd620806afb6b51", size = 861949, upload-time = "2026-09-29T00:47:27.005Z" },+ { url = "https://files.pythonhosted.org/packages/12/62/d0314bcedfd3586197e4596931fa220260eb2385bf53184e5b9ae67db24b/regex-2026.9.29-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ebb8912f565b8cdbbf27debfe00df04202c20e2f651b9e32767930c5eace3621", size = 796602, upload-time = "2026-09-29T00:47:29.233Z" },+ { url = "https://files.pythonhosted.org/packages/ae/c7/d5a8c13a613facb03e0fb55c1ebaaf7bb35d8e2c1abe8bef8dca809fc1d9/regex-2026.9.29-cp313-cp313-win32.whl", hash = "sha256:4d7d93613b01b0199961330e49cfc52d479b3d5776c56c691db31130c0a07d91", size = 269306, upload-time = "2026-09-29T00:47:31.14Z" },+ { url = "https://files.pythonhosted.org/packages/80/a7/bf93a3a6afa5f7bc16b7afb94ae581b01cae620b8ad56bd8f9572a985959/regex-2026.9.29-cp313-cp313-win_amd64.whl", hash = "sha256:61956f074ecd123f55adca68ee3eab46e6a07ad3f8e64e6db95dfacb444f55c4", size = 280307, upload-time = "2026-09-29T00:47:32.709Z" },+ { url = "https://files.pythonhosted.org/packages/b2/7d/388274e53605a86297f433a08102a7bbdcf9379d47683d307ccaefd88e2c/regex-2026.9.29-cp313-cp313-win_arm64.whl", hash = "sha256:bfc71e6d970419c1309b3640305298643e2a734cad3f7cfb6d2ddee4175ab53d", size = 279599, upload-time = "2026-09-29T00:47:34.674Z" },+ { url = "https://files.pythonhosted.org/packages/93/1f/d9dc6f02f569625faf67a4daec926cd5023472dcd69bb44286dccd5a5ab3/regex-2026.9.29-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:957bb708e8057ab1649ba566456429d691ec9b90d1c9ad1af1ba7ffbbeaf05f2", size = 497598, upload-time = "2026-09-29T00:47:36.541Z" },+ { url = "https://files.pythonhosted.org/packages/9c/83/9b693a3fd1451381e812031a8961ec5b3b8f0c8cc6871f14c5223642804d/regex-2026.9.29-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:c9b602fae1e00b7c035d661ce85575365719192a7b46784bd71cf64c68053aa0", size = 296250, upload-time = "2026-09-29T00:47:38.233Z" },+ { url = "https://files.pythonhosted.org/packages/dd/5f/52bc2abc3fef040cd9de76ab29c918d6a717a454ae2b9dd7938b0c95656d/regex-2026.9.29-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0166844493626c5015c6088ee15c9ca2fd060ca15b7641d1657da6a58432ae33", size = 293518, upload-time = "2026-09-29T00:47:39.957Z" },+ { url = "https://files.pythonhosted.org/packages/dc/fc/cf50671215ee0057046980b4571ef8646a005819bb67f0957e779ed107a5/regex-2026.9.29-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b97a38fb4c732b6832db6bf108963adbcd82ef1268ba2025dce390f45af75efa", size = 806037, upload-time = "2026-09-29T00:47:41.676Z" },+ { url = "https://files.pythonhosted.org/packages/14/4b/dddef8fc15c63e4347cc9efb138d0cd306f30e6c98acbcc81a8f780083b9/regex-2026.9.29-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a540abfab208e1b7ef2df231c40ef3b6cbb30a0aad6204e9b6a81c10a6794628", size = 878881, upload-time = "2026-09-29T00:47:43.755Z" },+ { url = "https://files.pythonhosted.org/packages/9f/cb/38daabed32d28f7e58a06e9344ce00dc67952e9996bc578ed6a29fe1240e/regex-2026.9.29-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ddfa987262763c3c22a8367d2a49c244b018a74c3a8e3ab1a864119ad45c5633", size = 918684, upload-time = "2026-09-29T00:47:45.594Z" },+ { url = "https://files.pythonhosted.org/packages/a9/4d/041d9458a645fee4fce4d642a89d27271a3cfcd91095104f6dde44da70bf/regex-2026.9.29-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2f7f7aa47b229f2b39a2ae2596d2ad5625d77b5eb9856fac2dab3eb506cdd0a0", size = 807176, upload-time = "2026-09-29T00:47:47.372Z" },+ { url = "https://files.pythonhosted.org/packages/bf/c4/4383eed7aa5aef67616cb1b3f3ad06b7c624c4e6cced48630cd5ce133d85/regex-2026.9.29-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d9b77b25b4f395f92de6099ab08e8ae2bc7e51dfe157f22900902243a5cc90c7", size = 784315, upload-time = "2026-09-29T00:47:49.518Z" },+ { url = "https://files.pythonhosted.org/packages/5c/a6/0086ad31cebb183c637d3198547075aa493afde308e1ff61fccccb29ba6e/regex-2026.9.29-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:34b6925af9853bf461950e6508910f179fd6e9b1a7ec8548e069606b7e51a26b", size = 793748, upload-time = "2026-09-29T00:47:51.279Z" },+ { url = "https://files.pythonhosted.org/packages/d5/a0/f9005cba3f629a859573fc5d1224ea4e1f97919ec8581d018e03a351a604/regex-2026.9.29-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:addd736a0547d553283adaf4e05d7104e7f2c7b0b092e9b4d28756825f14531f", size = 870302, upload-time = "2026-09-29T00:47:53.368Z" },+ { url = "https://files.pythonhosted.org/packages/01/4f/e1a3e46bb5315a4e18b01a990e7a28e2a16595609d50c442baf2815a3c65/regex-2026.9.29-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:fe3fa1dd453ed5c7f5ea23a26218329790ed7197a99b90e94330e313959a7f52", size = 770299, upload-time = "2026-09-29T00:47:55.606Z" },+ { url = "https://files.pythonhosted.org/packages/2c/fe/f303b4acfda44e1ff1379368748c1ef2dad04a6a8e9c0ecbc970b19d97ca/regex-2026.9.29-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:0cc63b5e47c12a48d90c7e9d7de6a035dd14f62868aaedbb4e0ff8ba2b8bfe7b", size = 861570, upload-time = "2026-09-29T00:47:57.617Z" },+ { url = "https://files.pythonhosted.org/packages/60/b6/b4f7e99249f596017c60ccad5faf9310fc8e3e59bb2244940a90a1b0bdff/regex-2026.9.29-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:724184b4aafed865e4f13ca313fdcb43024300c028ec67319cfa16847d84685e", size = 795967, upload-time = "2026-09-29T00:47:59.922Z" },+ { url = "https://files.pythonhosted.org/packages/fb/d3/fc865a4638d9f6762192b6bab5b7aa1f33a90e9e99578c2e111e2a63c8c3/regex-2026.9.29-cp314-cp314-win32.whl", hash = "sha256:c6c8fabf1dafc1f1ddcbb67896d3f93efb092e8c4b6322d7389b944e76a484e5", size = 274758, upload-time = "2026-09-29T00:48:01.8Z" },+ { url = "https://files.pythonhosted.org/packages/31/e2/c2b466924ccbeb874862968ca638051b15a8fd29d994a0e99004a5cbf78e/regex-2026.9.29-cp314-cp314-win_amd64.whl", hash = "sha256:1c2a0026062abcc321a53db4a185ceba0b59a66b5d37b0808917a88b55a5257f", size = 283817, upload-time = "2026-09-29T00:48:03.614Z" },+ { url = "https://files.pythonhosted.org/packages/c6/42/ea0f8dbaa924fa75c6338935eaee2f44dab369b27f02db1e03d74344b049/regex-2026.9.29-cp314-cp314-win_arm64.whl", hash = "sha256:121a76a0985db80ceae9e171c337f8c927868e37d01b54e3ce87bc87f9c6a208", size = 283663, upload-time = "2026-09-29T00:48:05.624Z" },+ { url = "https://files.pythonhosted.org/packages/44/48/d58e5081119f5c223bbb37d2340acde3d069e1df8e8cd166c37502eee4da/regex-2026.9.29-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:e31f72490b7c12f7790e1e25c3afffd20503ee1bfb43461d7838b871ff244b19", size = 501393, upload-time = "2026-09-29T00:48:07.833Z" },+ { url = "https://files.pythonhosted.org/packages/72/3c/c49945287d4f9efee7d41f98072f8ad880efb8f430595a612fbdea996a4e/regex-2026.9.29-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:80ea96f5c1a30bf09007d48466521d9c294bebe197c708c3359096e3e3691632", size = 298237, upload-time = "2026-09-29T00:48:09.684Z" },+ { url = "https://files.pythonhosted.org/packages/f9/1f/688cb61c3d4cf7bcc1ed444b5cc49399eba3e51c469ae285cf87fea3022e/regex-2026.9.29-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:554bffadcbcb6d5f4e5fb10a61cc52084b9a63d1dab5f10bcd2c4343972e8e2c", size = 295936, upload-time = "2026-09-29T00:48:11.454Z" },+ { url = "https://files.pythonhosted.org/packages/26/a3/de43ac6b877b7d09c19a3a426b1bd5acdd209eaaf68f406466f80439ccf6/regex-2026.9.29-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:864e9b87ac33c3fb9fb4ad48166d4fdb579c351d5c77deb0d34bccb36a775cd9", size = 816905, upload-time = "2026-09-29T00:48:13.321Z" },+ { url = "https://files.pythonhosted.org/packages/62/14/9940763201c51d537786304984c67d0fc3d2ed18837ffb6f09a869f6b6c9/regex-2026.9.29-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:044265d77d94f5e3cb2fd72c76723807c429cb8c533e9d4672d0334a6f14f588", size = 881527, upload-time = "2026-09-29T00:48:15.313Z" },+ { url = "https://files.pythonhosted.org/packages/d3/e1/c842d8df0b23245ebf202f8ab9c39fd48e2db39959454ec39a41c8c72082/regex-2026.9.29-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:2089fe39c406784d90101c726755ffa1497bb74638fd434300d2b88006186de8", size = 923115, upload-time = "2026-09-29T00:48:17.328Z" },+ { url = "https://files.pythonhosted.org/packages/d8/c1/98622479e3c354a446a75232e522d747d2b3df23092dcd8a5309380a2020/regex-2026.9.29-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0def9fb6abac55492d6d51cddb7225d07d6f279e774e0adc08569a54a5fc8d46", size = 820674, upload-time = "2026-09-29T00:48:19.32Z" },+ { url = "https://files.pythonhosted.org/packages/6c/d0/5808c95f9c79ed27b5eedaafc3df6239ec56a49f2e23ea8f831b18427c82/regex-2026.9.29-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:888d60953908dcf761aa320c3e390ab8556efbdb551ace63921de90f6ae0848d", size = 793306, upload-time = "2026-09-29T00:48:21.615Z" },+ { url = "https://files.pythonhosted.org/packages/bf/d3/021ca2638671ad20603bcd9b4d5bfa35d2610cd216a043ea7f0b44ea39f6/regex-2026.9.29-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:ed511a0708e2297e1d6431e7fb217e3402791e491e02da800658ace4973df1bb", size = 803103, upload-time = "2026-09-29T00:48:23.871Z" },+ { url = "https://files.pythonhosted.org/packages/6b/2d/755c6d13ef9c657378013676c391c7a402166b3f419a464a3e058dcbe533/regex-2026.9.29-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:e1172147d28d8fbcf8cb8d26c41506169f5ad8fe9ec969cb116835a19d4d8eca", size = 872175, upload-time = "2026-09-29T00:48:26.255Z" },+ { url = "https://files.pythonhosted.org/packages/6c/fc/e1cab183b9dafe8597f58c1c766da9bf96204d3b2f232bcf3eeb75ff7b6c/regex-2026.9.29-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:92f05c9c42bde5785dc48770bc2194d9f7442544156f951e19cd31b096cec562", size = 777362, upload-time = "2026-09-29T00:48:28.389Z" },+ { url = "https://files.pythonhosted.org/packages/06/7c/e10ea17fba31fb4a1f9d13ed53a2d2a9066a2aea58d7557e263f6d99e7b0/regex-2026.9.29-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:f37964e4a5e993d2fd45147741e9dff7f34a2d8c00ab94c4ea0514a4677f959e", size = 865606, upload-time = "2026-09-29T00:48:30.4Z" },+ { url = "https://files.pythonhosted.org/packages/8e/6e/69824d9aee1fd41c54ea7264654a47c8d9d84d8a228e11c2bcf4c201ed81/regex-2026.9.29-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:951733b1bbdb71e377cec567b409f1a7881b47cfcad84121aa74cb575fa425ea", size = 807945, upload-time = "2026-09-29T00:48:32.375Z" },+ { url = "https://files.pythonhosted.org/packages/89/22/857050a86e21ce60193e02a8ef662521f2e263a645c8b1b905fc136b61a7/regex-2026.9.29-cp314-cp314t-win32.whl", hash = "sha256:65b408d8fcb273e3499e7ef2ce796810da1becd208c7fb4373692a242d79d461", size = 276758, upload-time = "2026-09-29T00:48:34.72Z" },+ { url = "https://files.pythonhosted.org/packages/4d/96/56808fe029553d7d4c703414f2a527faad2ea2bfa9ca094a2e7f8762b530/regex-2026.9.29-cp314-cp314t-win_amd64.whl", hash = "sha256:bf48516e35cf848390ea68850aba53e7c333720d2945b4d2c25b69fc5171723f", size = 286527, upload-time = "2026-09-29T00:48:36.864Z" },+ { url = "https://files.pythonhosted.org/packages/01/aa/074e2cfb3d8101a6a764aba5f7c5d1e21de087483e35bdc0c4ce2eb60364/regex-2026.9.29-cp314-cp314t-win_arm64.whl", hash = "sha256:9173db3be74a35cb6731701094b98120f7ee4876a287882a59cdea1fa7da342f", size = 285954, upload-time = "2026-09-29T00:48:38.901Z" },+ { url = "https://files.pythonhosted.org/packages/a7/dc/d84990386c9dfdf8c377f00f371b241fdc9a2c8aea0e3d66941b2e51be0b/regex-2026.9.29-cp315-cp315-macosx_10_15_universal2.whl", hash = "sha256:c3589f40749acce747510bf5d589d54e376cb0930ea58b35effac97e5312b0c1", size = 497836, upload-time = "2026-09-29T00:48:40.858Z" },+ { url = "https://files.pythonhosted.org/packages/c2/ab/a569ebde875fa12ff8c6c9a30e07503620f195e4be4d54c3d3ee8eecc283/regex-2026.9.29-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:32ab11df9677ca80bcbb5fe4eb1da9109a5019239a054836efc6fa1c64e683cf", size = 296244, upload-time = "2026-09-29T00:48:42.952Z" },+ { url = "https://files.pythonhosted.org/packages/f3/3e/7d548e82a108e7c8b2d5246650e397a2f8db599f9b2e975466939c5b4e70/regex-2026.9.29-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:7c03031610e3e6ed1768a2b7a8fc84637c1257b50c5eacaf094c6e17a84fc563", size = 293748, upload-time = "2026-09-29T00:48:44.985Z" },+ { url = "https://files.pythonhosted.org/packages/40/34/a8e19a52f452bbb07b32a2bef70dcdf90c2737049749f74cc12d7486fb4f/regex-2026.9.29-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:42e82e578c904445d4c8a35b8f28052cf567593215fa5db06266fbc6f77aaa2e", size = 807840, upload-time = "2026-09-29T00:48:46.948Z" },+ { url = "https://files.pythonhosted.org/packages/88/7b/11fbd4640b3bb82b72822a63c20ade4013d562d291703a9debeedc24e682/regex-2026.9.29-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:0b65c72739f981377c9c22e0c5c3cd7f42da7bd8a3c9209330fac772c7d893ed", size = 879330, upload-time = "2026-09-29T00:48:49.168Z" },+ { url = "https://files.pythonhosted.org/packages/f3/55/de58c74f1f4e31586d83eb39c56872d686c4e0d0966d151884c833b94ced/regex-2026.9.29-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:4408b2b27a95ca8cc48b7411945753773353b5c93b307754781086c99d3a576f", size = 919251, upload-time = "2026-09-29T00:48:51.322Z" },+ { url = "https://files.pythonhosted.org/packages/81/42/a8c480f6dd5ac59fa28ddae79afd9d7ac7e596fdb61813adc65bb6e674b8/regex-2026.9.29-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a714befaacbd10092ffe4cea0d3c5f008fb9efe9bc322c715bcdfdee414b9a3d", size = 808808, upload-time = "2026-09-29T00:48:53.529Z" },+ { url = "https://files.pythonhosted.org/packages/68/60/0bc0d1ec8b37ad64be6fa30e035251f11de9667a0fac9e82ee74517d81be/regex-2026.9.29-cp315-cp315-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:33026515aebc0e70d1c89978e53e8d695d35d9e472f8d5b34465ba3c74028650", size = 789907, upload-time = "2026-09-29T00:48:56.036Z" },+ { url = "https://files.pythonhosted.org/packages/da/84/116a3ef19b3acfe81077f0bf2cbc7714a5e94bc8935b7243ab61cb0f1c3c/regex-2026.9.29-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:31b003f9a070335e2a8233ee9b14a3ca8e6d792012ae011f741bf0aaf11744c5", size = 795770, upload-time = "2026-09-29T00:48:58.284Z" },+ { url = "https://files.pythonhosted.org/packages/96/ba/e38c3f203e7e7e18c957d48e6cb6dbf96c11e95a44efa4a480522afc5d6d/regex-2026.9.29-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:c03c6eb6ece86dfdcbb34799efaa339b093132e1aceed491ba5e08fe06cdf699", size = 870671, upload-time = "2026-09-29T00:49:00.506Z" },+ { url = "https://files.pythonhosted.org/packages/2f/0f/9ee0b0cb76c55f63684bd7fff554978e8773b4fc86e2bcb2d50772dc1086/regex-2026.9.29-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:a5300757f8a68f5b6cc33f57338d72a0e3589c5cc9ad5f8504ea06f028be582a", size = 778631, upload-time = "2026-09-29T00:49:02.984Z" },+ { url = "https://files.pythonhosted.org/packages/b6/19/e6e3eeb226af5872c4958002f6edef4e4f40ea4cc5f5665023f2019eb045/regex-2026.9.29-cp315-cp315-musllinux_1_2_s390x.whl", hash = "sha256:80c7cadd3fd2bfde5df8aa0787e315812cad0c313a753095d02f4c2b6c01677b", size = 862187, upload-time = "2026-09-29T00:49:05.264Z" },+ { url = "https://files.pythonhosted.org/packages/5b/62/823c102e106bb2711d6b7dfe5981552fe4467b2969c46a20c5c383cf498c/regex-2026.9.29-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:3f1e6cb402a89457582cd696f982559217d13484a193202c394015297968c86d", size = 798423, upload-time = "2026-09-29T00:49:07.644Z" },+ { url = "https://files.pythonhosted.org/packages/37/e0/e927776258fa70b2f6feffc3be584ffc85ba4c1e20a320f0aee9a632fc7d/regex-2026.9.29-cp315-cp315-win32.whl", hash = "sha256:a64b85a4760337cfefdb27d42da6ed8b58e8cde3f2d57b6ef43e76ef6ea9ef47", size = 274757, upload-time = "2026-09-29T00:49:10.513Z" },+ { url = "https://files.pythonhosted.org/packages/77/04/358de85d1860238e1b4fa98fc2c80c990124a25d2e14739e28cc02c25562/regex-2026.9.29-cp315-cp315-win_amd64.whl", hash = "sha256:b3e445b66c80b4eb4234e855ce94d9adc183eedbd632816228d89930b91b2c5b", size = 283824, upload-time = "2026-09-29T00:49:12.849Z" },+ { url = "https://files.pythonhosted.org/packages/92/d3/d5c5b264784a5ab2b0f8cf620c1eeb4dbf3440d306761905e7d99345bef5/regex-2026.9.29-cp315-cp315-win_arm64.whl", hash = "sha256:8f39588af4731c8923c26810eb3b33f76f17633985e40f59c3cd45a33805a895", size = 283664, upload-time = "2026-09-29T00:49:15.331Z" },+ { url = "https://files.pythonhosted.org/packages/02/dc/f63ec2c201445ce1150fe780f5c56f16a10124d9a9da3a93161dbb0d8892/regex-2026.9.29-cp315-cp315t-macosx_10_15_universal2.whl", hash = "sha256:fb99cc9d45f48895d9d67f6a0b8a57f08d39c174d9f25ad97a313e0470267b1c", size = 501549, upload-time = "2026-09-29T00:49:17.705Z" },+ { url = "https://files.pythonhosted.org/packages/9a/5c/d2a698dc6bfc11fbce03f1cb0249c13284e93b79ed11f893edf6fac431c9/regex-2026.9.29-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:720537c7ea6f80dc61913184edb0ce2497a306b39ef19f28505b322553d52bdb", size = 298187, upload-time = "2026-09-29T00:49:20.171Z" },+ { url = "https://files.pythonhosted.org/packages/85/b7/88dcdb38cd3935d4ee9e9ce9b8e56cb3b3518d1f020acfa7dd62ad289bf8/regex-2026.9.29-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:0fd2c901cc307a745ad4bc87f20060d7a0825a3371d1e93488af22e7a387f78f", size = 296157, upload-time = "2026-09-29T00:49:22.342Z" },+ { url = "https://files.pythonhosted.org/packages/d3/8e/ba6c01dde33a69fc294b38b43f6677baaa5735a6248f39708031a738158a/regex-2026.9.29-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b11b589e00095ec69cf79841a76360f9b079e95b0368a25b5ebb951ab0c157ff", size = 818468, upload-time = "2026-09-29T00:49:24.612Z" },+ { url = "https://files.pythonhosted.org/packages/2a/f1/2586693e3a2d6b1247852593d37a6c17b42a92ee44f7cdcb9a0c1494e64a/regex-2026.9.29-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d7cab119d0df0b9413f106b4d7fc34f2872d3574ed3806fb48959c830b1537da", size = 882825, upload-time = "2026-09-29T00:49:26.996Z" },+ { url = "https://files.pythonhosted.org/packages/30/51/084f3e7bdcd0e9c33665c938cf5d134dc3548cbb4a75f0197ec7bfd754b1/regex-2026.9.29-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b89efc38431793d28b7cd91227e2f952ad7c48df19132b17f43a5fec3c14143b", size = 923314, upload-time = "2026-09-29T00:49:29.822Z" },+ { url = "https://files.pythonhosted.org/packages/5a/f1/066c6fc23b7dc229789c21c880b5ba5ad689fb95fed12e078266f55a1f9b/regex-2026.9.29-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:80a5ea3b4fd9d6a5b9a44f7976a9acaaab35aa3c1f6b29e5bd857dfabaded223", size = 822222, upload-time = "2026-09-29T00:49:32.404Z" },+ { url = "https://files.pythonhosted.org/packages/0a/56/592cd46fdb8f2f8682a1d7fd1310e4d0bcb93fbd0e6bbe4141ac28240227/regex-2026.9.29-cp315-cp315t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:19959129885356df0e97556856f77eb2888380dac18bed075a7c05c5128c618d", size = 794882, upload-time = "2026-09-29T00:49:35.076Z" },+ { url = "https://files.pythonhosted.org/packages/ee/4d/d65384bb071c864b01aa8314e3a6a687845ebd57588390976edc960c218b/regex-2026.9.29-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:6a1a824fbed817e0a891103886b68f063b1e83cc51bc97192a90a60195a9291f", size = 805887, upload-time = "2026-09-29T00:49:37.395Z" },+ { url = "https://files.pythonhosted.org/packages/65/b6/358de0d8f40d5178e4f7e7e121cfd5b961c812b77a055d11f5079e3f8fd7/regex-2026.9.29-cp315-cp315t-musllinux_1_2_ppc64le.whl", hash = "sha256:1ba8c6a416569ce0d37e83e28a254a61dc99a419084dfb6476cea02d997f74fa", size = 872901, upload-time = "2026-09-29T00:49:39.927Z" },+ { url = "https://files.pythonhosted.org/packages/00/06/6bfded72d043240c6b52bbb5e16f639d81affbf7484b4fe2ec45f3d4afc9/regex-2026.9.29-cp315-cp315t-musllinux_1_2_riscv64.whl", hash = "sha256:446654b29bfaa30500d80947eda42cef1449dc8a87f4e3cf061cc8485d3a1f0b", size = 782970, upload-time = "2026-09-29T00:49:42.581Z" },+ { url = "https://files.pythonhosted.org/packages/5a/20/9f418a50baa78b3ed8308fcb0cc49e472dd000b7ef935a7295af202ea744/regex-2026.9.29-cp315-cp315t-musllinux_1_2_s390x.whl", hash = "sha256:bf3c49863c23a1ad6da9c30351aed6cff8d5ddbeb63c5c8420ae54e98c7d0138", size = 865441, upload-time = "2026-09-29T00:49:45.238Z" },+ { url = "https://files.pythonhosted.org/packages/2c/29/817c7eacdeaf8463123e949bd394c39ad024eea1ec38ddf5ad141da2f3bd/regex-2026.9.29-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:01000ddf0e3ffef97f2413ceb514f6313040106b6d18a03ee00a4fe35c1eb1db", size = 809431, upload-time = "2026-09-29T00:49:47.878Z" },+ { url = "https://files.pythonhosted.org/packages/63/0b/83aab3b5b739947f744135a7a3a446e25433ebc92b05e01aae197ccbfdda/regex-2026.9.29-cp315-cp315t-win32.whl", hash = "sha256:c4e38dd8f39c43a91d2410ad2b85610701b0979342c3df1d69eaf8e838c757d8", size = 276964, upload-time = "2026-09-29T00:49:50.524Z" },+ { url = "https://files.pythonhosted.org/packages/72/f2/6314b5fc68789b5dcc38885bc6e3d6986b34fb3372b7231088ee5cecaa05/regex-2026.9.29-cp315-cp315t-win_amd64.whl", hash = "sha256:e2c89e9b762c57f59d5e99ee8b20202adb892e35f8d3485741340999ca55058e", size = 286487, upload-time = "2026-09-29T00:49:53.224Z" },+ { url = "https://files.pythonhosted.org/packages/56/bc/97b2245c8c7b2dd01f2db74f2bea003cd33c15009b4996a2447f46b5325c/regex-2026.9.29-cp315-cp315t-win_arm64.whl", hash = "sha256:e8c65ef3862a8ad6e86492b6ed9327805dd66904c012bd3649dc67d822ed6c34", size = 285955, upload-time = "2026-09-29T00:49:55.655Z" },+]++[[package]]+name = "rich"+version = "15.0.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "markdown-it-py" },+ { name = "pygments" },+]+sdist = { url = "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz", hash = "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36", size = 230680, upload-time = "2026-04-12T08:24:00.75Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" },+]++[[package]]+name = "ruff"+version = "0.17.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/e9/a7/70debb024dfacda67b8e560cc7511f52b34b9348a8cc8c1ec23036dcd51d/ruff-0.17.0.tar.gz", hash = "sha256:5cd03240d8208a557c2a9655a5cb07ebe36aa6bb35065f97d48c1f6adef5a322", size = 5320554, upload-time = "2026-10-09T19:47:29.248Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/bc/f8/ee5ab9da6089eae2a33e6008b01deb1eda19992c1c8e10661e98cee1640f/ruff-0.17.0-py3-none-linux_armv6l.whl", hash = "sha256:0e271826af9a20d18c6cfae8c51e82959167c24859686ddd3eb9a7f0842ce81e", size = 10148907, upload-time = "2026-10-09T19:46:38.695Z" },+ { url = "https://files.pythonhosted.org/packages/9f/d9/2f81fb5a9d580afbb11b1c8ff915233a11f2a1b27405d7991f183c5e1976/ruff-0.17.0-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:5f0ca4a40f81403689c04f12966e22f44e329ae362072d8f1587b7bda87f603b", size = 10224208, upload-time = "2026-10-09T19:46:41.711Z" },+ { url = "https://files.pythonhosted.org/packages/a7/20/643f3c8f75594f937b2bf74801241c56a2e2b8e139d24dff8b66b28cdd7f/ruff-0.17.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:cbf7149e0927dc3295d5d64679a4765576eef71b00782b2ae969ef82274d6bb9", size = 9273324, upload-time = "2026-10-09T19:46:44.323Z" },+ { url = "https://files.pythonhosted.org/packages/ec/91/627700b233d367736cb274f1bd0b47d1f2b12f68878192812bd875adadc3/ruff-0.17.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:13ee90156522998c3037059d8f66885c8adeeaf7643bdce2caceee196ecd23e0", size = 9953190, upload-time = "2026-10-09T19:46:47.155Z" },+ { url = "https://files.pythonhosted.org/packages/cd/92/91f7b5ed39490f89d6cbf56e1f543c383667a725efa8e2c2dee0f01f5591/ruff-0.17.0-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3d8e4a002a94cd9d0dc48b51dc69d807a172b5b9bf2b668e656424dc5b55ead1", size = 9961746, upload-time = "2026-10-09T19:46:50.098Z" },+ { url = "https://files.pythonhosted.org/packages/87/c5/7310f9fc63ce11ff6394edbd5e85433dfb0e14c9fbf6ccc97f1538491bc7/ruff-0.17.0-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c0b8a60c06a218c337e1161638d34757f83449243e2db161483ddf948e53ad14", size = 10625483, upload-time = "2026-10-09T19:46:53.379Z" },+ { url = "https://files.pythonhosted.org/packages/a5/8d/97443f0dca4a03a0bc7629fd396fd494a1cb6666121e38c5075acb217d8f/ruff-0.17.0-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a330178bdffc4205dbf3bda11d93e059e388fd6546f8cdd304501a9160363c0d", size = 11350129, upload-time = "2026-10-09T19:46:56.486Z" },+ { url = "https://files.pythonhosted.org/packages/9c/0a/c525efd9777be4b6b012e6969a3012648468e7e6c4b3e5b46af69f46e8eb/ruff-0.17.0-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7bb08489e234876fa2da67ae3ea938e9a2156da80293e0e4365abd6973d98329", size = 10906223, upload-time = "2026-10-09T19:47:00.263Z" },+ { url = "https://files.pythonhosted.org/packages/2c/3c/4a01195d93420cad1175bedad13a515dc8a56f95a6e39789b92e582682f5/ruff-0.17.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bc73e7c133e82d55b5f15897b2a442d72c0cb4a0c886c46801ce3c247150b60c", size = 10386314, upload-time = "2026-10-09T19:47:03.057Z" },+ { url = "https://files.pythonhosted.org/packages/c7/72/1a3951665485a921f6375f91e754a1854d5a645d41acc3642668064ff64d/ruff-0.17.0-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:db4f74c533403ab70fe4007873f6ae0c9f94a8b03158cf48d78788e47cdbe399", size = 10565464, upload-time = "2026-10-09T19:47:05.831Z" },+ { url = "https://files.pythonhosted.org/packages/90/8c/539b4d8c082f57e18db8ae2be85a460d77861c79dcd5798e32b536a6a06f/ruff-0.17.0-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:3d8cc360e666d1914e47b0777c6906d70cf18891a55532bd0a16844195d70859", size = 10079468, upload-time = "2026-10-09T19:47:08.617Z" },+ { url = "https://files.pythonhosted.org/packages/e0/b8/84286966db79434e8c26b585b0a0f6897cb3ab1c51a4aa4df10c28488b62/ruff-0.17.0-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:d66de796b726c4801e05fa99a2a8d7a780e107be222486c304ab61765561e866", size = 9958556, upload-time = "2026-10-09T19:47:11.324Z" },+ { url = "https://files.pythonhosted.org/packages/69/50/27b6eed27b83fcdd5bfa0d52b83231e29094754374698da404d094487ae3/ruff-0.17.0-py3-none-musllinux_1_2_i686.whl", hash = "sha256:c3f268baf004aea944f040623327119527ea231af15f7fb7890e82cea0679589", size = 10352173, upload-time = "2026-10-09T19:47:14.188Z" },+ { url = "https://files.pythonhosted.org/packages/2a/fa/955399fd13044cd827862044117d784a59e3196f6cce7424908ac9a7f914/ruff-0.17.0-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:864b6c1acb6b0bccf94b5a3938a1531fd09aaca5e5659a2e7bf0f3cf2a685540", size = 10791761, upload-time = "2026-10-09T19:47:16.931Z" },+ { url = "https://files.pythonhosted.org/packages/ae/bf/024e01e1f6aec87768696725e648ed5b438941341eea8f8100beb681961f/ruff-0.17.0-py3-none-win32.whl", hash = "sha256:5e50aa5b84decd9fe5b0bb0e6f71c3b592f1767ed09faa4b7207d933961e35cd", size = 9904112, upload-time = "2026-10-09T19:47:19.75Z" },+ { url = "https://files.pythonhosted.org/packages/cc/77/1ee73df41dcc8d1cdb686ee4bc46ea29704ea175feb6b95c78420f631ab8/ruff-0.17.0-py3-none-win_amd64.whl", hash = "sha256:8ab76bcda86dfd28e13776cb5de3c7bcdcf1ae3d37ed761113d1a5a415dc134c", size = 10115585, upload-time = "2026-10-09T19:47:22.698Z" },+ { url = "https://files.pythonhosted.org/packages/fd/71/eb4f0ccc844aece56963e8578df9c95d4c00f580547d52329d4035d3af18/ruff-0.17.0-py3-none-win_arm64.whl", hash = "sha256:c154c73ff43f9854395e24cac507af13078962e53d2b511605058d22af1fdb88", size = 9872035, upload-time = "2026-10-09T19:47:26.306Z" },+]++[[package]]+name = "safetensors"+version = "0.8.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/45/06/f955dbbb1859e3bd23c8ac6141af5106e7ad5fedec4a3a6e3d60f94b7001/safetensors-0.8.0.tar.gz", hash = "sha256:fabaf3e0f18a6618d9b36560682562157f77c2b71fcffc7b432be2baed9d753d", size = 325846, upload-time = "2026-06-09T07:52:25.563Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/39/a0/f718cda65b05407d228f97602cf60dca269c979867aa5beb25410de26cd3/safetensors-0.8.0-cp310-abi3-macosx_10_12_x86_64.whl", hash = "sha256:c554f85858e05226d3c2828e32395e677434685d6d94594a41643361c5e837f0", size = 473568, upload-time = "2026-06-09T07:52:18.829Z" },+ { url = "https://files.pythonhosted.org/packages/f5/b1/fa7c600e7dceae12e9606c7578cbc9ff1e1ed55844883ee5c92205e86226/safetensors-0.8.0-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:c80201d22cbf405b80647a60ada77bba06c8fba2da2743ba1e89cdcc39a81f25", size = 484562, upload-time = "2026-06-09T07:52:17.518Z" },+ { url = "https://files.pythonhosted.org/packages/09/7d/65a7de0af421317bb36a067241e4235fff194eed60b961ed6d3f59a3fc60/safetensors-0.8.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7a46e5ff292c356d6991e60942ba7f79817682d3a2cef0702136448cb9c4d235", size = 502844, upload-time = "2026-06-09T07:52:07.624Z" },+ { url = "https://files.pythonhosted.org/packages/91/4f/3175c9d75634e0e0dda0082794193521035edd7c70a6f212bf33ca06ddf4/safetensors-0.8.0-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:4124502b78f03534117c848f87a39b8f31e577b15eff423bf8bfb95f2a8c30d0", size = 511823, upload-time = "2026-06-09T07:52:09.565Z" },+ { url = "https://files.pythonhosted.org/packages/20/87/846c289e7aa2299eff406335717cf43ce8777194ece8aad75772e0411615/safetensors-0.8.0-cp310-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7bc0a787ba8a35be368ee3574edfa2b1ad389eebd0a72e482ae275490e3f6c98", size = 633461, upload-time = "2026-06-09T07:52:11.128Z" },+ { url = "https://files.pythonhosted.org/packages/76/22/8d64d9df2c45d5ded401df889d0ad90882804ca172d79ec4f0df8f727fe0/safetensors-0.8.0-cp310-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:040070828e36dc8e122178bbbd5830ff9e97920affb84cbe0f46442497bed358", size = 545148, upload-time = "2026-06-09T07:52:13.603Z" },+ { url = "https://files.pythonhosted.org/packages/28/50/f203ff3a3ddfe19308efc83c5a3a29ed02bf786732ec35e68bf9162f3365/safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fd6f3f93c9a0a7cc2788ee63fb763353d4bd2e89b0751bc78fcf7dda00bea774", size = 516040, upload-time = "2026-06-09T07:52:16.29Z" },+ { url = "https://files.pythonhosted.org/packages/46/fb/cdaed17ceb2948784fd9c36b6fd3e951b608547cea81a48e8ee6f8cfdfcb/safetensors-0.8.0-cp310-abi3-manylinux_2_31_riscv64.whl", hash = "sha256:fcdd41ec4628fee5799f807c73c353629130fbd942aa23d83c623dd6c9d52d78", size = 513832, upload-time = "2026-06-09T07:52:12.37Z" },+ { url = "https://files.pythonhosted.org/packages/0d/49/1e15de264dcc3b77943d2d0c56a95809956883b1c2d6d585c792523f180b/safetensors-0.8.0-cp310-abi3-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:8e9f537aa183a38ace122d27303dcd986b26bd2a7591f9181d7f0c396f4677ca", size = 559930, upload-time = "2026-06-09T07:52:14.743Z" },+ { url = "https://files.pythonhosted.org/packages/2a/43/bf38443278eab4b1be1fce2931e2b012ad9cb7df52ada751d0aab8f7659a/safetensors-0.8.0-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:87eec7ffed2b809f05a398a8becb7d013f19f7837cd15d9748580d6cf30dbaf4", size = 678670, upload-time = "2026-06-09T07:52:20.032Z" },+ { url = "https://files.pythonhosted.org/packages/72/e3/68cd3fa5b48488e84add63e04cb12f3bc28ae4638c06d4508c6e88823d0e/safetensors-0.8.0-cp310-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:4a95ae2b05d7726d751da4ebf626a2ca782b706e101bd894c95bc2450b1cffcc", size = 786679, upload-time = "2026-06-09T07:52:21.322Z" },+ { url = "https://files.pythonhosted.org/packages/29/4b/1c19c509d56e01f4fbb3d0a2e597450f6cc04d1d56cf52defb0a62dfd715/safetensors-0.8.0-cp310-abi3-musllinux_1_2_i686.whl", hash = "sha256:3ae091f16662658bdc019a4ff6cb4c085bb7d725eb5978b183ffd265863b6d2d", size = 765683, upload-time = "2026-06-09T07:52:22.594Z" },+ { url = "https://files.pythonhosted.org/packages/27/43/41c1621732edd934d868a00d1b891584c892a7b62a9aab82ea5a0a5623ee/safetensors-0.8.0-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:8e080062fcde23be189565e1c3305d16751a218ecf9412c8601e64204eb6f846", size = 722361, upload-time = "2026-06-09T07:52:23.924Z" },+ { url = "https://files.pythonhosted.org/packages/8e/3f/73ccf82579412b4a71c4ca673f10b5f1f888d7cf5af7fe24f27d30307be4/safetensors-0.8.0-cp310-abi3-win32.whl", hash = "sha256:2ddf52eac562eda224f99acfa7889d02968c1fd59a5b011ae7d8137c37e9c02d", size = 342401, upload-time = "2026-06-09T07:52:28.895Z" },+ { url = "https://files.pythonhosted.org/packages/1b/6d/3fba214c1e5e0f69991677ec3bc17023f0421776975e1de0c682dca475e2/safetensors-0.8.0-cp310-abi3-win_amd64.whl", hash = "sha256:096ec1a98435df7beb08853bb5aa9081a84f23d0adc67ed1a0a10550f608373f", size = 355540, upload-time = "2026-06-09T07:52:27.832Z" },+ { url = "https://files.pythonhosted.org/packages/8d/fc/7eedc3510d97878876e32774eebbeb61c43f148a96e915c84229a3e967aa/safetensors-0.8.0-cp310-abi3-win_arm64.whl", hash = "sha256:f7838e5135a406ad3e02efdcb8cf2e5397d368b0154537c4fec682dbc544d452", size = 340500, upload-time = "2026-06-09T07:52:26.745Z" },+]++[[package]]+name = "setuptools"+version = "84.0.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/6d/44/f5da03a8ef95d369145c5bb53050e7877c9f3d312e128605fd9504829143/setuptools-84.0.0.tar.gz", hash = "sha256:f4695c21257f0d9b537ec2692c941d02ee143b7cc1276941349a546573b2ef73", size = 1168449, upload-time = "2026-08-08T18:27:58.365Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/95/9c/c510029fc6ef33a6275cd2c5d3cecd6613dfd6aa401d57c54f1c18852ccf/setuptools-84.0.0-py3-none-any.whl", hash = "sha256:51a52592b3b99e102b609654876bd65f19f999935166d1352678931132b0c670", size = 818216, upload-time = "2026-08-08T18:27:56.719Z" },+]++[[package]]+name = "shellingham"+version = "1.5.4"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/58/15/8b3609fd3830ef7b27b655beb4b4e9c62313a4e8da8c676e142cc210d58e/shellingham-1.5.4.tar.gz", hash = "sha256:8dbca0739d487e5bd35ab3ca4b36e11c4078f3a234bfce294b0a0291363404de", size = 10310, upload-time = "2023-10-24T04:13:40.426Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl", hash = "sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", size = 9755, upload-time = "2023-10-24T04:13:38.866Z" },+]++[[package]]+name = "sympy"+version = "1.14.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "mpmath" },+]+sdist = { url = "https://files.pythonhosted.org/packages/83/d3/803453b36afefb7c2bb238361cd4ae6125a569b4db67cd9e79846ba2d68c/sympy-1.14.0.tar.gz", hash = "sha256:d3d3fe8df1e5a0b42f0e7bdf50541697dbe7d23746e894990c030e2b05e72517", size = 7793921, upload-time = "2025-04-27T18:05:01.611Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353, upload-time = "2025-04-27T18:04:59.103Z" },+]++[[package]]+name = "tokenizers"+version = "0.23.3"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "huggingface-hub" },+]+sdist = { url = "https://files.pythonhosted.org/packages/e0/7c/2cabb2174e772636683008f2c5621949b645da7d303c596589e84516a184/tokenizers-0.23.3.tar.gz", hash = "sha256:cded33237c77caeef62944d32aa9a7ef42bdce2b3497e18d137e072a8c4be438", size = 385286, upload-time = "2026-10-09T10:16:55.759Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/aa/2e/4ce5b9716f26e526eff6b0502ebed4ea8d7161f03b3c77617c9f25528e97/tokenizers-0.23.3-cp310-abi3-macosx_10_12_x86_64.whl", hash = "sha256:9d2b5c97daf61688c2ad1803ca851800feaba50fb68d5821779e9ea5880d968c", size = 3148800, upload-time = "2026-10-09T10:00:51.457Z" },+ { url = "https://files.pythonhosted.org/packages/b2/72/01e49f032bb346e5aaf06c10c74fe8aeec847173adbadd66eb7c53054bf2/tokenizers-0.23.3-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:68649e97d5b43c44c031d8d848874a6eecae8f8fe40ea989aa777a5a83aca716", size = 3101381, upload-time = "2026-10-09T10:00:54.063Z" },+ { url = "https://files.pythonhosted.org/packages/15/fc/ae987741829b1cd547668c4c94be732ae3eefd1d74344e64c3d2ca714acd/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ec82e80e65a862275b97c3d90b7a523df8d9519ee48aeb4e9625b2cc909274e0", size = 3519944, upload-time = "2026-10-09T10:00:55.885Z" },+ { url = "https://files.pythonhosted.org/packages/1c/da/cc8f6c030afaf05fbddc608158fbb761dca46913cbeba6b112e59fc82e2a/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:c64a0713180ff16829d4e7f39a658b77ea11443af4e1aa46523692943c9b1414", size = 3397695, upload-time = "2026-10-09T10:00:57.444Z" },+ { url = "https://files.pythonhosted.org/packages/ec/f1/256f78d1365fa2cd3ea6db716883d74667c8cbb6a21f15fa5b89a773cdc2/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ddedfd4b3b4be6be24ff6ca645c4a37fddfd305f6f3e354c54cf10b715c48215", size = 3753125, upload-time = "2026-10-09T10:01:00.165Z" },+ { url = "https://files.pythonhosted.org/packages/60/93/eee007ac2fcbf4ecfce7fbc354826cf3611f56bdb886f3e91b1f7dd06b8f/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:2a89614730d7b80940a5d2ed9320e1ec8add5a745c6151d8d05071b7215505b6", size = 4018598, upload-time = "2026-10-09T10:01:02.05Z" },+ { url = "https://files.pythonhosted.org/packages/bf/f9/0c96c4739461fce9d8d865b416728081bf6230022d7163bd6244f35f4b31/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e88646b8580c5ad7f4361477f1298e9cc01771a1ee9aecfe32c47b8ff614cc38", size = 3602442, upload-time = "2026-10-09T10:01:03.77Z" },+ { url = "https://files.pythonhosted.org/packages/3a/40/6706b82693715581457c6d5423eaa7faae576bb0526c5738a57085eb4449/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:376851d22bcf9d650a5c3090bb83e6cf9e895fbf0595369fa4cd43c1f69b5f87", size = 3396193, upload-time = "2026-10-09T10:01:05.48Z" },+ { url = "https://files.pythonhosted.org/packages/fe/0c/85946de40e25b7364b8f1bcf56def129069acd5bb364b7c86a32919e1a23/tokenizers-0.23.3-cp310-abi3-manylinux_2_31_riscv64.whl", hash = "sha256:bf501c40b72d2d5c8623620210430e9cac1ce47a46e45b34107b70a1557d46b0", size = 3553483, upload-time = "2026-10-09T10:01:07.387Z" },+ { url = "https://files.pythonhosted.org/packages/f1/6b/8d615d92cad1d511ca5ab188d1c7c167f0b3d295cc0d96207f9f82d486d8/tokenizers-0.23.3-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:114e2b55ed177179d59f4ab98200a4471e11e78f9e4b5a922d146740f96fcf52", size = 9972248, upload-time = "2026-10-09T10:01:09.437Z" },+ { url = "https://files.pythonhosted.org/packages/c9/7d/a922e37ddd58d1b463bbc2ad08120c8f59c60b814cd353519a116b24f8ba/tokenizers-0.23.3-cp310-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:d3407fb7b9c4d75dd68850ffd7180bc0a5d2dbaf0762d888e612f31fec3f9c6b", size = 9802957, upload-time = "2026-10-09T10:01:11.869Z" },+ { url = "https://files.pythonhosted.org/packages/4b/06/5d3f506a86ae0699a0e4ea05c05978f9aee169ef2c1d844e68c971cf8194/tokenizers-0.23.3-cp310-abi3-musllinux_1_2_i686.whl", hash = "sha256:84513ef0aeb8bf8f4ea11a2e8a7ac163ec5288aa115e649a59b470ac5c3107df", size = 10145487, upload-time = "2026-10-09T10:01:14.268Z" },+ { url = "https://files.pythonhosted.org/packages/26/e5/065625317690ea3548d834dad81f48ea1fd32e4964610e658e195d7fe28e/tokenizers-0.23.3-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:e05ab7baf7f47b406a95fea6f3b0a484b2ddcd9e1d14b68844c457eb755085a3", size = 10266026, upload-time = "2026-10-09T10:16:33.054Z" },+ { url = "https://files.pythonhosted.org/packages/77/4e/babede85d0d19f5e3deeef0063e01848141329934d3d77c31b5cab5ac2b4/tokenizers-0.23.3-cp310-abi3-win32.whl", hash = "sha256:1ebf28794e7e4954e20a7f70fbea410b2d1f0418f7dbbca97ca384fcfef38c25", size = 2588086, upload-time = "2026-10-09T10:16:35.686Z" },+ { url = "https://files.pythonhosted.org/packages/d1/6c/24f074c9a0efb98e61b20aafe6b2641922d5db24e447d5d6daffd9e17555/tokenizers-0.23.3-cp310-abi3-win_amd64.whl", hash = "sha256:1f0823bb00c5fdc98e487354d54dd55a03848d61a1a0bf29a68c77f24f3b26c3", size = 2872101, upload-time = "2026-10-09T10:16:37.533Z" },+ { url = "https://files.pythonhosted.org/packages/53/77/a476b6f73a661c11d113a342d2326b91506cf2285f0995d1212a6bb2022d/tokenizers-0.23.3-cp310-abi3-win_arm64.whl", hash = "sha256:7e48734d2de9260d86f03ab056d2cfeeff3869f61dbd49aaa15a2793b5f3458b", size = 2742580, upload-time = "2026-10-09T10:16:39.244Z" },+ { url = "https://files.pythonhosted.org/packages/65/46/f66baaedd42414a3f583c47379dc350e3e1f858a690d2574fd85ae70681b/tokenizers-0.23.3-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:efa3d7318406b4d115dce61ad5061953f1f44b128e79c020ce4615d763e23b6e", size = 3154274, upload-time = "2026-10-09T10:16:40.876Z" },+ { url = "https://files.pythonhosted.org/packages/c6/41/8de8c63b2d935eee5a0f42011fb7b786ffafeab0b8eb6d17acb8af2293b7/tokenizers-0.23.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:a4fbb3662f9f59d199d61338e54b4bcc11d07ebbb1aeb3540dacb2be9c521cb7", size = 3077805, upload-time = "2026-10-09T10:16:42.856Z" },+ { url = "https://files.pythonhosted.org/packages/e3/08/b1cbae8dc8fc7c91f992ac2d87a086e9b3f25a28814047ca16a82fe8c87b/tokenizers-0.23.3-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:de536665495cb4b409d25bade41963f801aff4225c19a6b804b048f7d14e34c7", size = 3491678, upload-time = "2026-10-09T10:16:45.093Z" },+ { url = "https://files.pythonhosted.org/packages/3e/0d/aac0cb2f3a1fdbef514145b4c5f2df4d05deeb1ee8f73ae641a1b4a62a85/tokenizers-0.23.3-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5cc24bb457dd4a8af89c8fcb40074d570129ec473df2a866c276ee55db4749d7", size = 3367420, upload-time = "2026-10-09T10:16:47.112Z" },+ { url = "https://files.pythonhosted.org/packages/1e/1d/41a697d0c193a320b243fbd68b2057b6eb2f01ecf80899e1a16e646ff699/tokenizers-0.23.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:acd5c57b4bd3e56e246e2731a3a3a6825a7a7d89b7e3b761ba80bc521710f04b", size = 9945973, upload-time = "2026-10-09T10:16:49.326Z" },+ { url = "https://files.pythonhosted.org/packages/37/e9/b56e619fcd583000a2b1254bb46af8dc6a174d3ba3329f454ad5a95a2be2/tokenizers-0.23.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:82eb480f6f1c21cea3349dec32cf1a6384c6c1e775f00f83b0d51197bc013687", size = 10237491, upload-time = "2026-10-09T10:16:51.943Z" },+ { url = "https://files.pythonhosted.org/packages/6f/68/f58b3beb95f3b62816e91e5e768e684cd63e58f9cbece22036dae3b1c971/tokenizers-0.23.3-cp314-cp314t-win_amd64.whl", hash = "sha256:1554a6eed34d9d6a78d23360f4e06df8dffab1ae08c7e8488e0b3e3b36cc266f", size = 2847654, upload-time = "2026-10-09T10:16:54.166Z" },+]++[[package]]+name = "torch"+version = "2.14.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "cuda-bindings", marker = "python_full_version < '3.15' and sys_platform == 'linux'" },+ { name = "cuda-toolkit", extra = ["cublas", "cudart", "cufft", "cufile", "cupti", "curand", "cusolver", "cusparse", "nvjitlink", "nvrtc", "nvtx"], marker = "sys_platform == 'linux'" },+ { name = "filelock" },+ { name = "fsspec" },+ { name = "jinja2" },+ { name = "networkx", version = "3.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },+ { name = "networkx", version = "3.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },+ { name = "nvidia-cudnn-cu13", marker = "sys_platform == 'linux'" },+ { name = "nvidia-cusparselt-cu13", marker = "sys_platform == 'linux'" },+ { name = "nvidia-nccl-cu13", marker = "sys_platform == 'linux'" },+ { name = "nvidia-nvshmem-cu13", marker = "sys_platform == 'linux'" },+ { name = "setuptools" },+ { name = "sympy" },+ { name = "triton", marker = "python_full_version < '3.15' and sys_platform == 'linux'" },+ { name = "typing-extensions" },+]+wheels = [+ { url = "https://files.pythonhosted.org/packages/5d/9b/fd1cd143a7bda5abc1fc352ed7b431f6f52aa3c454293211a33656db2cb5/torch-2.14.1-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:b6074b130fd26bc50f5d171f07dfed70db22dd0c354ab34767729249f9e0f042", size = 127281154, upload-time = "2026-09-30T17:43:58.328Z" },+ { url = "https://files.pythonhosted.org/packages/8d/0e/21281ff4015779255229f487f4481c5bfba53b8c83e01eaa4a8a8f1180c9/torch-2.14.1-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:ef1276fd07adb8d463ac402a24ea7482e45c9a42df8ebdac211aa757779200ba", size = 453982557, upload-time = "2026-09-30T17:44:52.656Z" },+ { url = "https://files.pythonhosted.org/packages/aa/4f/7af6fbada912b3023cc79b6ab30f89234b258a9a373880ea52ed914a2ae9/torch-2.14.1-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:305a61f61f35f128579f299c5bd33d475f6a01c6307336139632e30856c4854d", size = 554581699, upload-time = "2026-09-30T17:49:10.777Z" },+ { url = "https://files.pythonhosted.org/packages/fe/54/4e0a71376b2fb15aaf65ed9e6a4c8043d36f554338c6daabc63059a5e157/torch-2.14.1-cp311-cp311-win_amd64.whl", hash = "sha256:0bd2fb0c4098856bc5af59fe5988bf38bdbcdb6550b44273c12ae425cb8ec17e", size = 124096268, upload-time = "2026-09-30T17:44:04.362Z" },+ { url = "https://files.pythonhosted.org/packages/4b/3c/379349820e9a6e713826de3d0b88f07728541bdcabe2a3afd128871cdc4b/torch-2.14.1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:420dbf314c180ee4b86e9bc00aee5746a7d6e5bacdd7df925af671ed393f0b2e", size = 127307099, upload-time = "2026-09-30T17:44:10.421Z" },+ { url = "https://files.pythonhosted.org/packages/a4/3e/474930ab971e306957ec763b7df695d659179e15ccab9e7073d47ae14ce8/torch-2.14.1-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:0b13acde294b401509830d85a8906cbf2fc97425a12a66cce1d9fa13c5106a2a", size = 453998918, upload-time = "2026-09-30T17:52:45.134Z" },+ { url = "https://files.pythonhosted.org/packages/ba/fd/c470857d0dfc5d4c94430658b3eed60fd5292b0bf596a8aae7b9fd66d674/torch-2.14.1-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:23011fe29a99b591eabb3a31a25080c62e2c2f0690a3c798744e5489dae50451", size = 554618695, upload-time = "2026-09-30T17:53:06.8Z" },+ { url = "https://files.pythonhosted.org/packages/ad/1a/b9648b0d228e513f69f8453ec3f90a8d436d78ff6d85a2773e539f0c07cf/torch-2.14.1-cp312-cp312-win_amd64.whl", hash = "sha256:38bee9f2a2ccfc6898172a5075e4ba52522fbe143eeb099674fc67ab390e858d", size = 124113906, upload-time = "2026-09-30T17:52:21.377Z" },+ { url = "https://files.pythonhosted.org/packages/7d/11/faaca4f8541c45127b7e0d6bb141221fe944c8466d89986b8466c6c195f9/torch-2.14.1-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:dbe359d705f4d67236743794c296c6dee93a922fd8117eff8c3e880d7d0fb2b9", size = 127315286, upload-time = "2026-09-30T17:52:27.43Z" },+ { url = "https://files.pythonhosted.org/packages/60/1f/0330275c705b846882531c64e80d32b13fb572eeb9b995e87e84154b0712/torch-2.14.1-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:6d530bd11479fb574567af3a1f384af9a77bf5bb9550cde8080b356ca7220d5a", size = 453999200, upload-time = "2026-09-30T17:53:21.716Z" },+ { url = "https://files.pythonhosted.org/packages/7c/cc/bb579ac0c80e077c58204e43b37a254b74dea5873635100bea25789a55d4/torch-2.14.1-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:c8f71aabc67bcbfc9373dc131537a5968d04edce73e88add21354a7cd0a76985", size = 554618164, upload-time = "2026-09-30T17:53:42.73Z" },+ { url = "https://files.pythonhosted.org/packages/7b/4b/32c00cacbe682a4c08d9c6c3e7a7116c912184cc7155f06c64f545db97d1/torch-2.14.1-cp313-cp313-win_amd64.whl", hash = "sha256:711713391d26a1ce5e9fbc6c996d954a8c12e8825374cc77b809a6af29539b8c", size = 124113754, upload-time = "2026-09-30T17:52:33.474Z" },+ { url = "https://files.pythonhosted.org/packages/43/19/23a1aed488423a5055727256b25406e4b93bd2bcf1352bef582b9951c10c/torch-2.14.1-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:cee091caf2a6229e248daf41d18ceebf590ba02d9179205610062c64ee5fef03", size = 127325330, upload-time = "2026-09-30T17:52:54.752Z" },+ { url = "https://files.pythonhosted.org/packages/93/f4/94219ada13edd62fda1f976163292b7b8595f1fd2f4ef74af24b3baad375/torch-2.14.1-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:f68f5476e2bc0e8f60b74f7ca21acda885c4477af6392f8977e4f0d1ea1aa162", size = 454016297, upload-time = "2026-09-30T17:54:03.078Z" },+ { url = "https://files.pythonhosted.org/packages/fd/df/23c69e9b9fd19fe6563422f1bc59f89601bda47234a89a1cf432cfde3aee/torch-2.14.1-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:1d4df07be7338bbcc4d54085adee21363c91314702b9bd4d78ef72ffac9465ba", size = 554621744, upload-time = "2026-09-30T17:54:22.951Z" },+ { url = "https://files.pythonhosted.org/packages/d9/dc/a36a4431ab5e3ad168a75f341e24677a4bbbf0c5dda59c97a7079262bb94/torch-2.14.1-cp314-cp314-win_amd64.whl", hash = "sha256:d02a4c48a2ca5fb7654e36e71f710f74494d83f1f10aaff8e059dd554adca956", size = 124110781, upload-time = "2026-09-30T17:53:31.141Z" },+ { url = "https://files.pythonhosted.org/packages/2d/bc/1afbd1a22f6023eafcdb9295ac1d97137a7816f54091d693701072da2218/torch-2.14.1-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:06c3ec25f3b497f9a73dc0c36f293f295cfe8d5584fd446238d9a501b30a66d5", size = 127657364, upload-time = "2026-09-30T17:53:53.837Z" },+ { url = "https://files.pythonhosted.org/packages/e4/94/b97e863c9ceef2bc6e082e967cbfb9065b5e232f2a60c5af0f5115a1bc31/torch-2.14.1-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:9113f94f70429f9f302bf55b090b5411269083e8a72a5d015ffcc5a7f83f2c69", size = 453998592, upload-time = "2026-09-30T17:54:38.268Z" },+ { url = "https://files.pythonhosted.org/packages/03/d8/8272157c438cc26a199a8fa0caf87985477b8a1ed8a09084eb2ae40609d8/torch-2.14.1-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:e65d5fe136e533b23c2d134377f7f126721c7c78dd75b3dbb735af082a8aeb85", size = 554578730, upload-time = "2026-09-30T17:54:54.838Z" },+ { url = "https://files.pythonhosted.org/packages/45/05/451a69a4287033d8f106f5c65f92c2c0c37229d81ea101d4b047caf758cc/torch-2.14.1-cp314-cp314t-win_amd64.whl", hash = "sha256:e07306caa1de2a4ac1467e11ecfc92fc44f523dd6a521145039aef46d913963c", size = 124400129, upload-time = "2026-09-30T17:54:12.306Z" },+]++[[package]]+name = "tqdm"+version = "4.70.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "colorama", marker = "sys_platform == 'win32'" },+]+sdist = { url = "https://files.pythonhosted.org/packages/0d/ea/b2a5bd54b28a324dae8211928b2d730b6547500342c7e6c6dea08bd0a485/tqdm-4.70.1.tar.gz", hash = "sha256:cefd0eca11b2a37a3aee776544d4f4ae913f02688135b5556b8788dfa474afc4", size = 171846, upload-time = "2026-09-11T07:25:16.601Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/a7/03/921a3d3c75785aca9ebfbfcabfbc3a1be12e2ab5265deb026d55a5a3f83e/tqdm-4.70.1-py3-none-any.whl", hash = "sha256:c293e525e6fef9c20e8728fd4612df02a0aa31bb5fe91ecd93e123b1b7bffa73", size = 80199, upload-time = "2026-09-11T07:25:14.599Z" },+]++[[package]]+name = "transformers"+version = "5.19.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "huggingface-hub" },+ { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },+ { name = "numpy", version = "2.5.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },+ { name = "packaging" },+ { name = "pyyaml" },+ { name = "regex" },+ { name = "safetensors" },+ { name = "tokenizers" },+ { name = "tqdm" },+ { name = "typer" },+]+sdist = { url = "https://files.pythonhosted.org/packages/65/4c/70012ed0795235fb6bd7806fafd4a1842f03e49ebbfa3bb9580e451948ce/transformers-5.19.0.tar.gz", hash = "sha256:87f38dd25e4521151b97e94520ac457f44a0ae8a8358a5b112daff6c64a822d6", size = 10032022, upload-time = "2026-10-06T16:39:00.229Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/b9/59/1e7f1212c215c73690ae6cab4678efe1555ff985953e145d88cc212859b0/transformers-5.19.0-py3-none-any.whl", hash = "sha256:afcd2dd5f603ed28c1e1fcb00a338ccbb4ef5f878ed289635df8b58187afb518", size = 12599123, upload-time = "2026-10-06T16:38:56.794Z" },+]++[[package]]+name = "triton"+version = "3.8.0"+source = { registry = "https://pypi.org/simple" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/a3/cf/d21c9b1a4d1df9ba3aed218069f24f762a4b9565c5a362bef6f110c08e72/triton-3.8.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:372285307d4c44ee74cee32de0b4f04bd157e071427e38c6f4ee3e3beb2194f4", size = 226467015, upload-time = "2026-08-28T16:08:05.871Z" },+ { url = "https://files.pythonhosted.org/packages/6f/9a/2c3d3823726d5ad5359f7065b02b984925158b74611c1e3987f6a37461fb/triton-3.8.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:68988ac85d5e7086baeda0ddc175af9667db7529b3c5e11a5c0601b8bef2200a", size = 247945226, upload-time = "2026-08-28T15:55:43.795Z" },+ { url = "https://files.pythonhosted.org/packages/87/07/0f8cd8e8db0472334253efdaaab3d0819fea27aa99bf0e7f1aeea4ceb5ae/triton-3.8.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a9c404c69ed4a39e8ec632eaf6b9fe058a060bf98979c177f6ef666f06bb8d50", size = 226474486, upload-time = "2026-08-28T16:08:18.29Z" },+ { url = "https://files.pythonhosted.org/packages/c1/09/b7012e5bfae67640f268aa584caa80fe1674f6b0da949046b679972c33e3/triton-3.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e91ffa46d095b252248297292dd22bcbacd53a125a0c2eefbbbf74925a320bc3", size = 247972921, upload-time = "2026-08-28T15:55:53.157Z" },+ { url = "https://files.pythonhosted.org/packages/87/4d/4c564374bcdadb166fccbf3e45aee0d4a473f88d341761bd2fefe3b8e8c1/triton-3.8.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b7004666652f500ed854a86988e4b3d69d247188b5d2092b5df1e44f4a954099", size = 226476793, upload-time = "2026-08-28T16:08:30.956Z" },+ { url = "https://files.pythonhosted.org/packages/b0/b6/3394d5548404c1cabd1dadadd28d0b3f9478db1dff8180da53bb3f0a1e19/triton-3.8.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1f0497218e26b7d79773ad9c2a3fa3b539ee69f587a13fac2e552b1d322a8015", size = 247975122, upload-time = "2026-08-28T15:56:04.112Z" },+ { url = "https://files.pythonhosted.org/packages/b8/59/bf0e9493118bb353ab59a5d6a65db3618d9b314417cc1459f0121e0ec5c9/triton-3.8.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1f6b48d0591929a3867973acac3dccd4e058585f91bfb41022de496c9ffab304", size = 226488654, upload-time = "2026-08-28T16:08:47.141Z" },+ { url = "https://files.pythonhosted.org/packages/93/d9/08c75f3459f19ad00425b564058e40efa4bcd79b816064cf27499303ea42/triton-3.8.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:387dae4cb0089a7b6ba1a428ae0782b65c4c58f57d94617cb22ca8593d8ccbca", size = 247972313, upload-time = "2026-08-28T15:56:14.007Z" },+ { url = "https://files.pythonhosted.org/packages/7c/34/429c5592181cfb7361a0a8e0bff218e7224b726709d75da2472b3e819f70/triton-3.8.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b84e7d512490ba529111260fa6f7cad8b254a6bb5fbdf41d5ef9a5e57f52d0a", size = 226591133, upload-time = "2026-08-28T16:09:02.271Z" },+ { url = "https://files.pythonhosted.org/packages/fe/d1/aa8a3e935c37efee7945984fdb64d7e0851bf6d920afd97b2d21f9d23360/triton-3.8.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:74217bb56ed8692759227758e4c4b3bd2d608a209c1a7a081bf361fb4c2c1bf9", size = 248077577, upload-time = "2026-08-28T15:56:24.94Z" },+]++[[package]]+name = "truststore"+version = "0.10.5"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/ee/9f/c5201d42a484c061e528825fc8e2d565f5abd50a4ced6fb7d29c4ec99b2b/truststore-0.10.5.tar.gz", hash = "sha256:30d36967ccaded5cbb38d602c433f53600036c79d502f4533a49b60a03bbefcd", size = 28091, upload-time = "2026-10-12T22:27:31.808Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/51/e9/3a7820be2bb0fe53b6bc9c3be26d3d1158004e4c3ab953aa6840b955b1e9/truststore-0.10.5-py3-none-any.whl", hash = "sha256:9aaaedaefaf06d8b206278cf8b5012bc897f485a874503501e12d776df78951c", size = 19017, upload-time = "2026-10-12T22:27:30.377Z" },+]++[[package]]+name = "typer"+version = "0.27.3"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "annotated-doc" },+ { name = "colorama", marker = "sys_platform == 'win32'" },+ { name = "rich" },+ { name = "shellingham" },+]+sdist = { url = "https://files.pythonhosted.org/packages/03/51/d33db42cc72ffd8c30777547b42d01f0cbf9d95a770457698d0174b3ed71/typer-0.27.3.tar.gz", hash = "sha256:d0396f770a560ab1b0a8504e13b5f254b728cedb05c61cf0359e944e50ce8901", size = 205303, upload-time = "2026-10-06T17:24:16.61Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/07/ea/2e31b67051e91a133189e9c000c222502ddc6969856416de0d095de4c0b0/typer-0.27.3-py3-none-any.whl", hash = "sha256:e50022f28b82a86313e54501317a1db64bf8f8d036ff8cfe5ca7e47675454aff", size = 123312, upload-time = "2026-10-06T17:24:15.054Z" },+]++[[package]]+name = "typing-extensions"+version = "4.16.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz", hash = "sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5", size = 113555, upload-time = "2026-07-02T08:40:05.92Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", hash = "sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", size = 45571, upload-time = "2026-07-02T08:40:04.659Z" },+]