Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
Probe — does a bigger batch ride free under the cold disk-tier read? (R2, the batch control)
Status: rule committed 2026-10-01, BEFORE the run. Attempt 1 (2026-10-01 23:35) ended in NO
VERDICT, see §5; no round arm was judged, so the rule's rows 3-6 have not been applied to any
number. Attempt 2 (2026-10-07 10:15Z, --run-prefix batch2) completed; rows 1-3
did not fire, and the verdict is in §6. Before the run the only executions were two --dry-runs of the driver into a scratch
folder (stamps, the cache check, a 4-s sampler self-test, the plan; no arm), a synthetic check
of the rule code on fake arm files, and a mocked run of the driver's arm flow (fake arm
processes that only write JSON). None of them wrote anything under results/.
0. The question
The cold 70B-shaped NF4 step at seq 512, batch 1 is bound by the read: 17.0-17.6 s on one
drive, 9.93-10.05 s on two striped drives (the gate's sequence 4), against a read-free compute
floor of 6.43 s. The record that measured the floor drew the consequence and said it was not a
measurement (probe-rtx5070-what-bounds-streaming.md §21, Finding 21):
Because the step is read-bound and the read is per-step not per-token, tokens per step are free until the compute floor meets the read: from the two B points, B(S) = 1.24 s + 0.0101 s x S, which reaches 17.0 s at about 1,560 tokens per step — batch 3 x 512 would roughly balance the two on this fixture, if it fits, and tok/s scales with the batch until then. That is arithmetic from two shapes, not a measurement at batch 3.
So: does tok/s scale with --batch 1 -> 2 -> 3 at seq 512 on the cold disk tier as that
model predicts, and what peak VRAM does each batch need on this 8 GB card?
Two drive layouts, because the second drive moves the crossover. The model's step is
max(R, B(512 b)), with R the layout's own batch-1 step (arithmetic, not a prediction of R):
| layout | R (gate sequence 4 median) | B(1024) | B(1536) | crossover (R - 1.242) / 0.010124 | where it falls |
|---|---|---|---|---|---|
| SINGLE | 17.438 s | 11.609 s | 16.792 s | 1,600 tokens | just past batch 3 |
| STRIPED | 9.971 s | 11.609 s | 16.792 s | 862 tokens | between batch 1 and batch 2 |
On one drive the model says batch 2 and batch 3 are both near-free (tok/s x2.00 and x3.00). On two drives the step is already compute-bound at batch 2 (tok/s x1.72, then x1.78 at batch 3). At batch 3 the two layouts' predicted steps are 17.44 s and 16.79 s: the second drive, worth 1.75x at batch 1, would be worth about 4% at batch 3. Whether that holds decides whether striping and a bigger batch add up or substitute for each other.
This is also the control arm of the planned layer-major (L2L) experiment: on the 70B shape the question for L2L is what it adds over the largest batch that fits, not over batch 1. §1a lists what that experiment needs from here, and §1 says which arms it may cite.
1. The fixture, the arms and the order
The gate's fixture, unchanged: weights D:/synth/llama-70b-shape-v32k-f4 (synthetic
Llama-3.1-70B shape, bf16 source, 32k head, random values, untied); NF4, disk tier, seq 512;
LoRA r 8 on q/k/v/o with dropout 0.0, adapter seed 3, input seed 17, PagedAdamW8bit at lr
1e-4, 2 layer buffers, read_ranges 4; both layouts pinned.
- SINGLE — the L2L control. No
--shards: the harness resolves the default one-root cache,C:\Users\user\.soup\layer-stream\D___synth__llama-70b-shape-v32k-f4. Not theD___synth__llama-70b-shape-v32k-f4.refitcopy that sits beside it, and not a cache underSOUP_LAYER_STREAM_CACHE_DIR: row 2 checks every SINGLE arm'smeta.shard_diragainst that exact folder. The whole store is on C:, read through direct I/O,--read-ahead 2. - STRIPED — an extra, not the L2L control.
--shards C:/Users/user/.soup/layer-stream-striped/D___synth__llama-70b-shape-v32k-f4 --stripe-dir D:/soup-stripe --read-ahead 3. Under the code at 2c6ed087 root 1's folder isD:\soup-stripe\D___synth__llama-70b-shape-v32k-f4-cba3360a9c80(the driver prints it fromstripe_folder_name). It exists for the crossover question of §0; an L2L run onorigin/mainhas no striping to compare with it.
The code, and what the SINGLE arms are evidence for. Every arm runs 2c6ed087, the head of
PR #1536. Its disk reader is the per-drive dispatcher that PR introduced, so the SINGLE arms run
that dispatcher with one drive: not byte-for-byte the reader on origin/main, where an L2L run
would start. The evidence that this one-drive path reproduces the known step is the gate's
sequence 4, whose SINGLE arms read 17.552, 17.431 and 17.438 s (gate §5.18) inside the band
gate-974 set for the pre-PR reader. That was measured on f798c46e. Between it and 2c6ed087,
src/ gained the PR's review fixes and a merge of origin/main (a5dac3bc) that brought, among
others, #1143 (#842: a checkpoint-visible fused NF4 forward where bitsandbytes takes its fused
GEMM, and #841's cached per-buffer views). At this fixture's M (512 tokens and up) bitsandbytes
0.50.2 does not take that GEMM: for a weight of 4 M elements or more its heuristic reads "custom
wins up to M=512; dequant+F.linear wins above that", and its sm120 branch caps the custom kernel
at M <= 256 for weights of three waves or more (bitsandbytes/backends/cuda/ops.py,
_gemm_4bit_use_custom_cuda). So the forward stays dequantise plus F.linear, as in §21; what
#841 removes is at most §21's 0.65 ms per visit (Finding 22), about 0.1 s per step. That is
reading code, not a measurement: row 1 re-establishes the known step on 2c6ed087 in this session,
and the B blocks below measure the compute floor on it.
Each layout runs at --batch 1, 2 and 3. Every arm is --steps 3 --warmup 1 --step --skip-events: the gate's judged block exactly (three timed steps after one warm-up, the
uninstrumented step_plain point), so the SINGLE batch-1 arm is the quantity the gate's band
was written for. A SINGLE arm then runs the read-free compute floor at its own batch
(--ablate --arms B_nocopy --rounds 1: arm B of §21, no source read and no host-to-device copy,
three timed steps after one warm-up), after its step block, in the same process. B is not
judged; it measures the compute half of the model on this code at 512, 1024 and 1536 tokens.
It runs in SINGLE arms only because without a read the layout does not matter.
The command per arm, as the driver builds it (<b> the batch, <arm flags> from above,
<floor> = --ablate --arms B_nocopy --rounds 1 for SINGLE arms and nothing for STRIPED):
python benchmarks/harness/stream_probe.py --weights D:/synth/llama-70b-shape-v32k-f4 --quant nf4 --tier disk --seq 512 --batch <b> --steps 3 --warmup 1 --step --skip-events <floor> <arm flags> --label "R2 batch control round <r> arm <LAYOUT> batch <b>" --out benchmarks/results/probe-rtx5070/batch-control/batch_r<r>_<layout>_b<b>.json
with PYTHONPATH=C:/Users/user/projects/Soup-r2/src set by the driver.
The order: two rounds, the second the exact reverse of the first.
| position | 0 | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|---|
| round 0 | SINGLE b1 | STRIPED b1 | SINGLE b2 | STRIPED b2 | SINGLE b3 | STRIPED b3 |
| round 1 | STRIPED b3 | SINGLE b3 | STRIPED b2 | SINGLE b2 | STRIPED b1 | SINGLE b1 |
The first arm alternates (SINGLE, then STRIPED), the layouts interleave, and every one of the six arms sits at mean position 2.5 over the two rounds, so a drift that is linear over a round (the drives or the card warming) shifts round 0's batch-b ratio one way and round 1's the other, and cancels in their mean. Three rounds (the gate's number) would not balance (forward, reverse, forward), and a fourth would take the sequence past the time this probe shares an AC window with the sustained probe (§4).
Before the rounds: the build arm(s), not timed, never judged. The driver first runs the
sharder's own cache check on the CPU (index and stat reads only: no torch, no shard opened)
with the arguments the harness passes (bfloat16, nf4, double quant, quantised on cuda, no
external tensors). A STRIPED build arm always runs (--batch 1 --steps 1 --warmup 0 --step --skip-events); a SINGLE one runs only when the check does not report a hit. On 2026-10-01 the
check read: SINGLE hit at 20:12:56 and 20:16:35; STRIPED miss at 20:12:56 ("stripe
marker D:\soup-stripe\D___synth__llama-70b-shape-v32k-f4-cba3360a9c80\stripe.json is missing
or unreadable") and hit at 20:16:35, after the sustained probe's own build ran in between.
So a STRIPED build arm on a hit is one cold step (about 40 s); on a miss it re-shards (the
gate's took 275.9 s, §5.1 there). Either way the driver then waits 300 s before the first round
arm.
1a. What the L2L experiment needs from this record
The planned L2L step 0 runs k micro-batches of 1 x 512 through each loaded layer, on one drive. It may cite the SINGLE rows of this record as its control and not the STRIPED ones, with the code difference of §1 stated beside them. What it gets:
- tok/s at batch 1, 2 and 3, and the largest batch that fits (row 4). L2L's own contribution on this shape starts beyond that batch.
- The read-free floor B at 512, 1024 and 1536 tokens, in session. A rule that compares an L2L arm with "the read-removed arm at the same shape" has it here.
- The loss convention. The harness calls
model(input_ids=ids, labels=ids)with nonum_items_in_batch, so the step's loss is the mean over all b x 511 label tokens of the batch. An L2L arm over k micro-batches matches it only when each micro-batch's summed loss is divided by that same total, HF'snum_items_in_batchover all k micro-batches; otherwise the gradients differ by a scale. - No RNG in the forward on this fixture. LoRA dropout is 0.0 (
make_lora) and the fixture'sconfig.jsonsets noattention_dropout(LlamaConfig's default is 0.0), so the forward draws no random numbers, and an L2L arm needs no per-micro-batch RNG state here. A real model trained with the defaultlora.dropout0.05 still does. - The token ids, row by row. Every step point carries a SHA-256 per row of its
(b, 512)ids, and row 2 requires every arm of one batch to carry the same list, so the losses of two arms of one batch are losses on the same tokens. An L2L arm can show it trains on the same rows. Whether row 0 of a(b, 512)draw equals the(1, 512)draw of the batch-1 arm is recorded, not assumed. - What a loss difference means. Losses are not bit-identical across processes even within one arm (the gate's §5.20: same-arm differences reach 0.017391 on a first timed step), so an L2L-against-batch comparison of losses is read against the same-arm spread, as the gate did.
- Peak VRAM per batch. L2L moves the checkpointed boundary activations (2 x 80 layers x 8192 x 2 B = 1.31 MB per token on this shape) to host RAM; the growth of the peak with the batch says how much of it is that term.
2. The decision rule, written before the run
Nothing in this section is edited once a number exists.
Inputs. For layout l, batch b and round r:
step(l, b, r): the arm'sstep_s_mean(the JSON names are below).e(l, b, r) = step(l, 1, r) / step(l, b, r), andē(l, b), its mean over the two rounds.r̄(l, b) = b x ē(l, b): the tok/s ratio over batch 1, since tok/s is 512 b / step.R(l): the mean ofstep(l, 1, r)over the two rounds.- The model:
B(S) = 1.241971 s + 0.0101240 s x S, ande_pred(l, b) = R(l) / max(R(l), B(512 b)). in_use(l, b, r): the largestnvidia-smimemory-in-use sample (MiB) the driver took inside the arm's three timed steps (2-s sampling; each step's window runs from itsstarted_unixforstep_s), or over the whole arm if no sample fell inside them.- The class of an e at batch b: NEAR-FREE when e >= 0.95; else NONE when b x e <= 1.05; else PARTIAL.
| row | measured | verdict |
|---|---|---|
| 1 | any SINGLE batch-1 arm whose step is outside 15.0-21.5 s, or that has none |
NO VERDICT for both layouts: the session does not reproduce the known step; record and stop |
| 2 | any arm off the path under test: direct_io or pinned not true; a source other than AsyncDiskSource; read-ahead not 2 (SINGLE) or 3 (STRIPED); shard_dir not the layout's cache of §1; a STRIPED arm whose stripe roots are not D:\soup-stripe or whose 80 layers are not placed i mod 2, or a SINGLE arm with any stripe root; batch or seq not the arm's; tokens_per_step not 512 b; not 3 timed steps; a timed step that did not load 157 + 2; not b row hashes; shard_seconds above 30 s (it re-sharded). Two arms of one batch whose row hashes differ. Or the protocol stopped (a position void twice, a pre-arm wait expired), so a position that row 4 did not skip has no arm |
NO VERDICT for both layouts: the arms did not measure what the rule reads |
| 3 | any STRIPED batch-1 arm whose step is outside 7.69-12.0 s, or that has none |
NO VERDICT for STRIPED; rows 4-6 are applied to SINGLE alone |
| 4 | per layout, for b = 2 then 3, in either round: (i) an out-of-memory point or line; (ii) no step_plain point within 600 s; (iii) peak_alloc_gb above the arm's own vram_total_gb; (iv) a spill: in_use >= 7,620 MiB AND step(l, b, r) > step(l, 1, r) + B(512 b) |
DOES NOT FIT at b: a result, not a void. Quote which test fired, the peak, the error or the cap, and for (iv) the power, SM clock and utilisation of the timed steps and the shared GPU memory. Larger batches of that layout are not run and not judged |
| 5 | per layout, each b that fits: the class of ē(l, b) |
NEAR-FREE, PARTIAL or NONE at b. Quote ē, both rounds' e, r̄, tok/s and the peak VRAM. Two labels, each quoted beside the class wherever it appears: AT THE CARD'S EDGE when in_use >= 7,620 MiB in either round; SLOWER THAN READ + COMPUTE IN SERIES when step(l, b, r) > step(l, 1, r) + B(512 b) in either round |
| 6 | per layout, each b classed in row 5: its class against the class of e_pred(l, b) |
AGREES or DISAGREES AT b, with any label of row 5. Quote both classes, the measured crossover bracket and the model's crossover (R(l) - 1.241971) / 0.0101240 tokens |
The rows are applied in order. Rows 1 and 2 are validity conditions for the whole probe; row 3 is one for STRIPED. Rows 4-6 are the verdict, per layout and per batch: there is no single pass or fail. The measured crossover bracket of a layout runs from 512 x the largest b for which every batch up to it is NEAR-FREE (batch 1 counts as NEAR-FREE by definition) to 512 x the first b judged PARTIAL or NONE, and is open-ended ("not reached") when no such b exists among the batches that fit.
The model's prediction, computed now from R at the gate's sequence-4 medians (the rule uses this session's R, which moves these numbers a little):
| layout | b | B(512 b) | predicted step | e_pred |
class | predicted tok/s | r̄ |
|---|---|---|---|---|---|---|---|
| SINGLE | 1 | 6.425 s | 17.438 s | — | — | 29.36 | 1 |
| SINGLE | 2 | 11.609 s | 17.438 s | 1.000 | NEAR-FREE | 58.72 | 2.000 |
| SINGLE | 3 | 16.792 s | 17.438 s | 1.000 | NEAR-FREE | 88.08 | 3.000 |
| STRIPED | 1 | 6.425 s | 9.971 s | — | — | 51.35 | 1 |
| STRIPED | 2 | 11.609 s | 11.609 s | 0.859 | PARTIAL | 88.21 | 1.718 |
| STRIPED | 3 | 16.792 s | 16.792 s | 0.594 | PARTIAL | 91.47 | 1.781 |
Crossovers: SINGLE 1,600 tokens (the bracket the model predicts is "beyond 1536, not reached"); STRIPED 862 tokens (bracket (512, 1024]). Row 4 (iv)'s bound, read plus compute in series, is 29.05 s and 34.23 s for SINGLE at batch 2 and 3, 21.58 s and 26.76 s for STRIPED.
Peak VRAM, as arithmetic, judged by nothing. §21's round-0 baselines peaked at 3.894 GB
allocated at 256 tokens and 4.378 GB at 512 (i974_ablate_cold70b_seq{256,512}.json), i.e.
1.889 MB per token on top of 3.411 GB. Extended to the batch: 5.345 GB at batch 2 and 6.312 GB
at batch 3, against the 8.518 GB card (vram_total_gb, gate sequence 4). The boundary
activations alone (1.31 MB per token) would give 5.72 GB at batch 3. Neither counts the CUDA
context or reserved-but-free blocks (reserved ran 4.798 GB at 4.378 GB allocated in every gate
arm), so batch 3 should fit with about 2 GB to spare in allocated terms and much less in
what nvidia-smi sees: it may well read near 7,620 MiB without spilling, which is why row 4
(iv) asks for the slow step as well and the memory alone is only a label.
Where every threshold comes from. Each is read from a committed record; the source line is quoted.
| threshold | value | source |
|---|---|---|
| SINGLE band (row 1) | 15.0-21.5 s | gate-two-drive-striping.md §2, row 1: "any SINGLE arm outside 15.0-21.5 s — gate-974 §8's own cold rows are 16.175 s (15.77-17.06) and 17.311 s (16.28-20.42), widened ~5% each side". The gate's sequence 4 read 17.552, 17.431 and 17.438 s inside it |
| re-shard (row 2) | 30 s | gate §5.17: "Every arm's shard_seconds is 0.003-0.158 s, a cache hit."; gate §5.1: "Sharding: 231.812 s". 30 s is far from both |
| STRIPED band, upper (row 3) | 12.0 s | gate §2: "STRIPED <= 12.0 s in every round" (its SHIP line), met in every round of sequence 4 with "The median of the three STRIPED means is 9.971 s. The largest is 10.050 s." (§5.18) |
| STRIPED band, lower (row 3) | 7.69 s | gate §5.18: "Every arm moved 70,378,258,732 bytes per step"; gate §0: "Two drives read 7.65-9.15 GB/s together". 70,378,258,732 B / 9.15 GB/s = 7.692 s: a STRIPED step faster than that would read faster than any two-drive read on record, and is a measuring fault |
| out of memory (row 4 i) | exit 3 and a point of kind oom, or an out-of-memory line in the log |
stream_probe.py since this probe's harness commit |
| wall cap (row 4 ii) | 600 s | a protocol bound, not a measurement: about 4x the longest arm the plan expects (SINGLE batch 3 with its B block, about 152 s; §4). A spill that slows the step 3x still finishes under it and meets (iii) or (iv) |
| allocated above the card (row 4 iii) | peak_alloc_gb > vram_total_gb |
gate-836-bench-train-contract.md §8: "On this platform, a memory figure above the card's capacity is the signal that it did not." Allocated, not reserved: gate-v0.73.1-measured-vram-fit.md §3: "Reserved overshoots what must fit because it holds freed blocks that are returned under pressure. This is the reason the shipped gate reads max_memory_allocated." |
| the card as full as a spill (row 4 iv, row 5's first label) | 7,620 MiB in use | probe-rtx5070-what-bounds-streaming.md §18: "nvidia-smi reported 7,620 of 8,151 MiB in use, 100% utilisation, 1.8 GHz, 40 W — torch's own peak was 4.38 GB allocated / 4.83 GB reserved", the reading that record took as the WDDM spill signature on this card. A spill need not show in torch's own counters, which is why (iii) alone is not enough |
| slower than read plus compute in series (row 4 iv, row 5's second label) | step(l, 1, r) + B(512 b) |
The model's own worst case: the step is never longer than reading and computing one after the other with no overlap at all, since both are measured or modelled whole (R in this round; B from §21). A step past it has a cost the model has no term for. §18's spilled step is the scale of such a cost: "| B no read, no copy | 39.8 s |" at 1 x 512, where §21 measured "6.450 / 6.401 s" for the same arm and shape without a spill, 6.2x. The bound is 1.59-1.96x the predicted step here (STRIPED batch 3 to SINGLE batch 3), so a spill of that kind clears it and an imperfect overlap does not |
| NEAR-FREE (row 5) | e >= 0.95 | Two allowances, each from a record. What Finding 21 itself calls free: the baseline step grew from 17.005 to 17.139 s and from 17.001 to 17.208 s when the tokens doubled under the read (§21 table: "| A baseline | 17.139 / 17.208 s | 29.9 / 29.8 | 17.005 / 17.001 s | 15.1 / 15.1 |"), +0.79% and +1.21% per doubling, so +1.92% at batch 3 (log2 3 = 1.585 doublings). And two arm means of one session: gate §5.18's STRIPED means span 9.929-10.050 s (1.22%) and its SINGLE means "span 17.431-17.552 s" (0.70%); two arms anywhere in the wider span add up to 2 x 1.22%. Together 4.36%, i.e. e >= 0.958. 0.95 (a step at most 5.3% longer) is the round number just below, and is therefore slightly lenient |
| NONE (row 5) | b x e <= 1.05 | The model's own floor for a fully compute-bound step: with R far below B, r̄ = b x B(512) / B(512 b) = 1.107 at batch 2 and 1.148 at batch 3, above 1 because B carries a 1.242 s per-step cost (§21: "B(S) = 1.24 s + 0.0101 s x S"). A tok/s gain of 5% or less is below that floor, so no regime of the model produces it; 5% mirrors the NEAR-FREE band |
| model constants (rows 4 iv, 5, 6) | 1.241971 s, 0.0101240 s/token | §21 table: "| B no read, no copy | 6.450 / 6.401 s | 79.4 / 80.0 | 3.822 / 3.846 s | 67.0 / 66.6 |", unrounded from the JSON: 6.425469 s at 512 tokens and 3.833720 s at 256 (round means); slope (6.425469 - 3.833720) / 256, intercept 3.833720 - 256 x slope. Finding 21 rounds them to "1.24 s + 0.0101 s x S" |
| foreign reader (a void, §4) | 1 GB per arm | gate §5.21: "the largest foreign reader in any arm was MoUsoCoreWorker.exe (Windows Update's orchestrator, started during the arm), which read 58.2 MB during gate4_r0_single." 1 GB is 17x that; this probe's arms (55-152 s expected) are no longer than the gate's (94-154 s). The same condition and number as the sustained probe's §4 |
Why power and the SM clock are recorded and not judged. A spill is often described as "full
memory, low power, falling clocks", and §18's matched it ("1.8 GHz, 40 W"; in its arm D "the
reported clock falling 1965 -> 1462 MHz"). The other spill on record on this card did not:
gate-836-bench-train-contract.md §8, "Shape B completed while allocating 1.51x the card's
VRAM", ran at "100% utilisation whenever busy, 71–78% of samples busy, 2782 MHz median clock
while busy (2797 peak), 32.7–33.3 W median and 47.7 W peak against the 50 W cap". And no
committed record gives this step's healthy power or clock on mains to set a "low" against; the
gate's own SM samples ranged 907-2040 MHz across its valid arms (§5.18). A condition on either
would miss a spill like gate-836's, or fire on a healthy arm. So the driver samples power,
clock and utilisation every 2 s and the rule quotes them (median and minimum over the timed
steps) beside every row-4 spill and every batch-3 arm, without setting anything on them.
What the rule deliberately does, stated now so none of it is read in afterwards:
- The mean of the two rounds' ratios, not "in every round". The reversed second round is
what cancels a linear drift, and it cancels only in the mean; requiring each round alone
to clear a band would let that drift decide the class. Both rounds'
eare quoted beside every class, so a split is visible. - The model is applied to this session's own R. Row 6 does not test whether today's read matches §21's (rows 1 and 3 bound that); it tests the model's other two parts: that compute grows as B(S) says, and that the step is the larger of the read and the compute, with the other hidden.
- The model's B(S) was measured on older code. §21 ran PR #991's tree (3d2be4d0,
2026-09-15); this probe runs 2c6ed087, which carries #1143. At M >= 512 the forward is the
same dequantise plus
F.linear(§1), so B(S) is expected to hold within about the 0.1 s #841 can remove. If it does not, the in-session B blocks show it, and a DISAGREES that the B line explains is the code moving, not the model's structure failing. - SINGLE batch 3 sits on the model's edge. B(1536) is 16.79 s against a 17.44 s read, 3.7% under the crossover. The max() model has no term for imperfect overlap as the two meet, so a PARTIAL there is a DISAGREES that the record reports as such. It is not a fault in the protocol, and nothing in the rule moves to absorb it.
- A spill is a fit result, never a slow step. Row 4 (iv) takes it out of rows 5 and 6
before they can read it as compute catching up with the read. What (iv) cannot see is a
spill too small to push the step past read plus compute in series; the edge label marks
every batch whose card was that full, and §5 will quote, for every batch-2 and batch-3 arm,
the arm process's shared GPU memory (the Windows counter that rises when allocations land in
host memory), its dedicated memory,
peak_reserved_gbagainst the card andnum_alloc_retries. - An out-of-memory error at a larger batch is a result. The gate voided an arm with no
step_plainpoint and re-ran it; here an arm that stopped on out of memory (the harness'soompoint, exit 3, or an out-of-memory line in its log) or ran past the 600-s cap is not re-run. It is row 4's input, and the sequence goes on without that layout's larger batches. - The read-free floor is reported, not judged. The crossover from the in-session B line (the least-squares line through B at 512, 1024 and 1536 tokens, where R meets it) is quoted beside row 6's model crossover, so a reader can see whether a DISAGREES came from the compute growing differently from B(S) or from the overlap.
- Losses are recorded, not judged. Row 2 only makes sure the arms of one batch saw the same tokens.
The inputs, as the JSON names them (fixed with the rule):
step:step_s_meanof thekind: "step",label: "step_plain"record ofbatch_r<r>_<layout>_b<b>.json; itstok_per_s,peak_alloc_gb,peak_reserved_gb,tokens_per_step,ids.row_sha256andmemory(num_alloc_retries,num_ooms,free_gb) from the same record, and the per-stepstep_s,started_unix,loss,layer_loadsandlarge_loadsfrom itsrecords.vram_total_gb,shard_dir,direct_io,pinned,source_class,read_ahead,stripe_roots,layer_roots,shard_seconds,batch,seq: the arm JSON'smeta.- The out-of-memory point: the
kind: "oom"record (itslabel,error_type,error,peak_alloc_gb,peak_reserved_gb), or an out-of-memory line in the arm's.log. - The read-free floor:
step_s_meanof thekind: "ablate",label: "B_nocopy_r0"record of a SINGLE arm. - In
batch_driver.json, per arm:runs[*].timed_out; the 2-snvidia-smirowsruns[*].during_samples.gpuas[unix time, used MiB, total MiB, SM MHz, temperature C, power W, utilisation %, throttle reasons], from whichin_useand the timed-step power and clock are computed; the 1-s GPU memory rowsruns[*].during_samples.gpu_memory; the foreign readers fromruns[*].after.top_readers. python benchmarks/harness/batch_control_probe.py --summarizerecomputes every input and applies the rows from those committed files alone (the rule's code isbatch_control_rule.py). Comparisons are made on unrounded values against the thresholds as written; numbers are quoted at three decimals, and none is rounded in a way that moves a row.
Also recorded, not part of the verdict: the read-free floor per batch and its line; A / B
per batch (how far above the floor the step sits); implied_h2d_gb_per_s; each drive's read
rate per step, from the per-step started_unix and the driver's 1-s PDH counters; the SM
clock at the start and end of every block, and every 2 s with temperature, power, utilisation
and throttle reasons; dedicated and shared GPU memory every second, per adapter and for the
arm's process; the allocator's counters and the device's free memory after each block;
per-step losses; the per-row SHA-256 of the ids.
3. What this does NOT measure
- A real 70B checkpoint. Shape only, random values; no quality or correctness claim. The losses at batch 2 and 3 are not compared with anything.
- Batch above 3, any seq other than 512, packing, or gradient accumulation. Accumulation re-reads the model for every micro-batch; nothing here measures it.
- L2L itself. This is its control arm (§1a), not a test of it.
origin/main's reader. The SINGLE arms run the PR #1536 per-drive dispatcher with one drive (§1).- The setup path.
soup trainreaches a batch through the stream setup and its VRAM pre-flight (the fitted formula, orstream_vram_probe); the harness builds the model directly and checks nothing before allocating. "Fits" here means it ran on the card, not thatsoup trainwould accept the config. - A spill too small to slow the step past read plus compute in series, as a judged quantity (§2, note 5).
- Sustained load. Each arm reads for at most about a minute; the sustained probe on
probe/two-drive-sustainedasks that question. - Warm (page-cached) reads, the RAM tier, other drives or boxes. Both layouts read through direct I/O.
- The read-ahead difference between the layouts (2 against 3), unchanged from the gate (its §7). It does not enter a within-layout ratio, which is all the rule reads.
- Drive temperature (unreadable without elevation on this box, as the sustained probe's §3 records) and ambient conditions. The GPU's temperature and clocks are sampled.
4. Box state, per arm, and the commands
The gate's §4 protocol, run by batch_control_probe.py, which imports the gate driver's
functions rather than copying them. The same stamp is taken right before and right after every
arm: AC and battery (and whether it is charging), available RAM, commit used and limit, the
Python process count, GPU memory in use and compute apps, the AC ASPM index (read only),
soup_cli.__file__, and every process's cumulative read and write bytes, with the eight
processes whose reads grew most named in the after stamp. While an arm runs, the gate's sampler
takes AC and commit every 2 s and the same PDH counters every 1 s (per-volume read and write
bytes for C: and D:, paging, commit, CPU).
What differs from the gate's §4:
- Files. Everything goes to
results/probe-rtx5070/batch-control/: the stamps tobatch_box_state.log, every sample tobatch_driver.json, each arm's JSON and log tobatch_r<round>_<layout>_b<batch>.json/.log, a build arm's tobatch_build_<layout>.json/.log. The gate's committed logs are never appended to. - Two more samplers (
batch_control_sampler.py).nvidia-smievery 2 s (memory in use and total, SM clock, temperature, power, utilisation, throttle reasons), and the Windows GPU memory counters every 1 s (\GPU Adapter Memory(*)and\GPU Process Memory(*), Dedicated and Shared Usage; the per-process rows are kept for the arm's own pid, and the query is reopened every 5 s until that pid appears, so they do not depend on PDH picking up a process that started after it). - The GPU must be empty before an arm: 0 MiB in use, where the gate allowed 1024. Every stamp of the gate's four sequences read 0 MiB (its §5.6, §5.12, §5.21), and at this probe's first dry run a peer's CUDA process held 752 MiB, which the gate's bar would have let through. A fit measurement cannot share the card.
- A fourth void condition, as in the sustained probe: a process other than the driver that read at least 1 GB during the arm (§2's threshold table). If the per-process capture fails in either stamp the check has no input for that arm; that is recorded and does not void it.
- Out of memory, the wall cap and a spill are results, not voids (§2, notes 5 and 6).
Every other arm with no
step_plainpoint, and every arm that saw battery power, a suspend (two power samples more than 30 s apart) or a foreign reader, is void: kept under a_voidname and re-run once in the same position when the pre-arm checks hold again. A second void in the same position stops the sequence, and row 2 gives no verdict. - Skips. After an arm that does not fit (row 4; the driver applies (iv) live against the latest batch-1 step of that layout), every later position of that layout at that batch or above is not run; it is listed as skipped, not as missing. The verdict recomputes row 4 from the files with each round's own batch-1 step.
- Stopping early. After each round arm the driver applies rows 1 and 2, the two a single arm can settle, and stops if either fires. Row 3 does not stop the sequence: SINGLE can still be judged.
- Settle: 300 s between the build arm(s) and the first round arm, whichever order this probe and the sustained one run in: either the sustained probe has just read both drives for about 40 minutes, or this probe's build has just written the striped cache (about 36 GB over four minutes, the gate's §5.1). A judgement, not a measurement, and the same figure the sustained probe chose after its build.
Unchanged from the gate: AC on and commit headroom at least 8 GiB before every arm, polled every 30 s for at most 15 minutes, after which the sequence stops and the arms not run are listed. One process per arm, never two at once.
The run order between this probe and the sustained one. The commands below assume nothing: the driver's cache check decides the build. As of 20:16:35 on 2026-10-01 the striped cache under the PR's folder naming exists (the sustained probe's build made it), so the STRIPED build arm is one cold step. If the cache were missing, the same command would re-shard it first; the cache it writes is the one the sustained probe reads, from the same code and paths, and that probe's own build arm would then find a hit.
Estimated wall time, from the gate's own figures and §21's model, without voids or pre-arm waits (the driver prints the same estimate):
| part | estimate | from |
|---|---|---|
| STRIPED build arm | 0.7 min (hit) / 4.6 min (re-shard) | process start, build and one cold step; the gate's build, 275.9 s (§5.1) |
| settle | 5.0 min | above |
| 6 SINGLE arms | 110.5 / 131.2 / 151.9 s at batch 1 / 2 / 3, twice: 13.1 min | 15 s of process start (the gate's arms' wall time minus their steps, 13-15 s) + 4 steps at max(17.44 s, B) + 4 B steps |
| 6 STRIPED arms | 54.9 / 61.4 / 82.2 s, twice: 6.6 min | 15 s + 4 steps at max(9.97 s, B) |
| total | about 25 min, 29 with a re-shard | each void re-run adds 1-2.5 min, each pre-arm wait up to 15 |
The commands, in order — complete for whoever runs them (Git Bash, from the worktree, with
the base interpreter C:\Users\user\AppData\Local\Programs\Python\Python312\python.exe, not a
venv; the driver sets each arm's PYTHONPATH to Soup-r2/src itself):
# 0. Before anything: the laptop on AC with the lid open; the other Claude sessions asked by
# message to run no CUDA work, no test suite and no large copy on C: or D: until this probe
# reports done; the sustained probe, if it is running, finished.
cd C:/Users/user/projects/Soup-r2
git status --porcelain -- src benchmarks/harness # must print nothing
git log -1 --format='%h %s' # the rule commit (or a later records-only one)
PYTHONPATH="C:/Users/user/projects/Soup-r2/src" python -c "import soup_cli; print(soup_cli.__file__)"
# must print a path under Soup-r2\src
nvidia-smi --query-gpu=memory.used --format=csv,noheader # must print 0 MiB
# 1. Optional: the dry run, into a scratch folder only (stamps, the cache check, the estimate,
# a 4-s sampler self-test; no arm).
python benchmarks/harness/batch_control_probe.py --dry-run --out-dir "C:/Users/user/AppData/Local/Temp/r2-dry"
# 2. The probe: the cache check, the build arm(s), the 300 s settle, the twelve round arms; it
# prints --summarize's table at the end. About 25 minutes (29 with a re-shard).
python benchmarks/harness/batch_control_probe.py --plan all
# 3. Recompute §2 from the committed files alone, at any time after.
python benchmarks/harness/batch_control_probe.py --summarize
Step 2 exits 0 when the sequence completes, 4 when a row-1 or row-2 stop or a second void ended
it, and 5 when a pre-arm wait expired; in every case batch_driver.json says why and which
positions did not run. It has no resume: a stopped sequence is re-run whole with a new
--run-prefix (batch plus up to eight lowercase letters or digits), so the first one's files
stay as measured. The results to commit are everything under
benchmarks/results/probe-rtx5070/batch-control/, _void files included.
5. Results
5.1 Attempt 1 — 2026-10-01 23:35, batch_control_probe.py --plan all, NO VERDICT
Run from 5a4df9be (the rule commit; src/ and benchmarks/harness/ clean, soup_cli.__file__
under Soup-r2\src), with --run-prefix batch. AC online and charging at the first stamp
(73%, 19.15 GB available RAM, commit 29.91 of 60.91 GB, GPU 0 MiB, no compute apps); the four
other Claude sessions on the box had acknowledged a hold for 23:35-00:20 and were idle. Both
caches were a hit (SINGLE and STRIPED), so the build arm was one cold step.
| position | arm | wall | step_s (3 timed steps) |
judged |
|---|---|---|---|---|
| build | STRIPED b1, 1 step, 0 warm-up | 52.4 s | 23.44 | never (build arm) |
| 0, try 1 | SINGLE b1 | 173.2 s | 31.44 / 32.77 / 33.95 (mean 32.722) | VOID: foreign reader SearchIndexer.exe 4.94 GB |
| 0, try 2 | SINGLE b1 | 52,762.8 s | 18.22 / 30.08 / 31.42 (mean 26.575) | VOID: AC values seen [0, 1]; suspended, 52,599 s between two power samples |
The second void is a stop by the rule's own text (§4: "A second void in the same position stops
the sequence, and row 2 gives no verdict"). The 11 later positions did not run. --summarize
reads: NO VERDICT (row 2): the arms did not measure the rule (no valid round arm on record; row 1
does not fire on the two void arms' own steps, which the rule does not read). Every file of the
attempt is kept under results/probe-rtx5070/batch-control/ as measured, the two void arms under
their _void / _void2 names. The sequence has no resume; attempt 2 runs whole with a new
--run-prefix.
What happened, as far as the files say.
- The suspend. The laptop went to sleep seconds after try 2 started (23:44:32) and woke at 14:23:55 on 2026-10-02, on battery (AC 0, 79%). The Python process survived the sleep, so the arm finished and the driver stamped it. The cause of the sleep is not in the files; the box state log shows only the AC flip. Nothing of the driver can prevent it.
- The slow first arm. Try 1's three steps were 31.4-34.0 s, each with 157 + 2 loads, against
17.4 s for the same arm in the gate (§1). Compute was not the cause: in the same process the
read-free floor (
B_nocopy_r0) ran 6.658 s (mean of 3; §21: 6.425 s). The step is the read: 70,378,258,732 B in 31-34 s, and the 1-s PDH counters put C:'s read rate at a steady 2.0-2.2 GB/s through all three steps (per-step means 2.17 / 2.11 / 2.03 GB/s; the C: bytes read inside each step window, 69.6 / 67.4 / 69.1 GB, are the step's own), against ~4.1 GB/s averaged in §21. D: read 0. - The indexer is not shown to be the cause.
SearchIndexer.exeread 4,940 MB and wrote 643 MB in the arm (it read 258 MB during the build arm and 523 MB in the whole of try 2). Its share of C:'s traffic in the arm is about 1.8% (4.94 GB of the 275.0 GB the 1-s PDH counters summed for C:), and the C: rate did not move in bursts. Its small reads could still have cost latency; the files cannot say. What its reads were of (the layer cache, the many worktrees on this profile, anything else) is not recorded anywhere: the per-process counters carry bytes, not paths. - A pattern inside try 2. 18.22 s for its first step, which is inside the gate's band for this
arm, then 30.08 and 31.42 s: the rate halved during the arm, and the arm began after a 14.6-h
idle. One arm is not a pattern; it is the reason attempt 2 reads C:'s per-step rate before
anything else. The two void arms' power and clock samples (32.5 W / 795 MHz median in try 1) are
in
batch_driver.json, and are not judged.
Not concluded: why C: read at half rate; that the indexer, the drive's temperature (not readable without elevation on this box), a power state or something else is the cause. The rule's row 1 (a SINGLE batch-1 step outside 15.0-21.5 s) is exactly the check that would fire on this, and attempt 2 is bound by it as written.
5.2 Attempt 2 — 2026-10-07 10:15:46Z, batch_control_probe.py --plan all --run-prefix batch2, COMPLETE
Run from aef2d31a (attempt 1's records commit; src/ and benchmarks/harness/ clean,
soup_cli.__file__ under Soup-r2\src). src/ is byte-identical to 2c6ed087 and also to the
head PR #1536 merged at (0130e025, 2026-10-04): the PR's later commits touch only benchmarks and
docs (git diff --stat 2c6ed087 0130e025 -- src/ is empty). Run by session measure-28 for soup-28
inside the owner's machine window (10:07Z), with a hold listed in .claude/SESSIONS.md. Times
below are UTC; the box-state log stamps local time (UTC+5).
Box. AC online and battery 100% at every stamp; GPU 0 MiB before every arm; commit headroom 21.3-28.1 GiB. Two preconditions of the runbook were not met, and the run went ahead under the ruling recorded on 2026-10-04 (handoff, R2 section, UPDATE item 3: run once anyway after SearchIndexer.exe has read ~0 MB/s for 5 minutes in a row):
- Windows Search was RUNNING (no session is elevated; the owner had not stopped it). SearchIndexer.exe's cumulative read count stayed flat at 1,150.789 GB from 10:10:10 to 10:15:26Z (10-s samples), and the run started at 10:15:46Z. SearchIndexer.exe is not among the eight largest readers of any arm of this attempt.
- ChatGPT.exe was open. Its on-device-model process (PID 17064) is listed as a GPU compute app at every stamp, with 0 MiB in use, so the driver's pre-arm check (0 MiB) held.
Both caches hit; the STRIPED build arm was one cold step (wall 46.1 s, step 26.949 s,
shard_seconds 0.025 s), then the 300-s settle. The sequence ran 30.1 min against the 25.4-min
estimate (one void re-run, below).
One void, re-run once in the same position, as §4 prescribes. Round 0, position 4 (SINGLE b3),
try 1: foreign reader BackgroundDownload.exe (PID 3716, started during the arm) read 3.10 GB;
MoUsoCoreWorker.exe (Windows Update's orchestrator) also started inside it (45.4 MB). Its steps,
20.455 / 20.791 / 20.736 s (mean 20.661), are kept under batch2_r0_single_b3_void.* and are not
read by the rule. The re-run was valid. The largest foreign reader in any valid arm was
svchost.exe (PID 16716) at 376.3 MB, in round 1 SINGLE b1.
Every valid arm: direct_io and pinned true, AsyncDiskSource, read-ahead 2 (SINGLE) or 3
(STRIPED), 157 + 2 loads per timed step, 0 allocator retries, shard_seconds 0.004-0.008 s (cache
hits).
| round | pos | arm | timed steps (s) | step |
tok/s | peak alloc / reserved (GB) | in use (MiB) | B read-free (s) |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | SINGLE b1 | 18.712 / 19.179 / 19.547 | 19.146 | 26.74 | 4.378 / 4.798 | 4,750 | 6.511 |
| 0 | 1 | STRIPED b1 | 9.149 / 9.144 / 9.030 | 9.108 | 56.22 | 4.378 / 4.798 | 4,750 | — |
| 0 | 2 | SINGLE b2 | 17.781 / 18.077 / 17.910 | 17.923 | 57.13 | 5.345 / 5.545 | 5,462 | 12.074 |
| 0 | 3 | STRIPED b2 | 12.391 / 12.588 / 12.413 | 12.464 | 82.16 | 5.345 / 5.545 | 5,462 | — |
| 0 | 4 | SINGLE b3 (re-run) | 20.487 / 20.615 / 20.162 | 20.422 | 75.21 | 6.312 / 7.028 | 6,876 | 17.567 |
| 0 | 5 | STRIPED b3 | 17.748 / 17.842 / 17.760 | 17.784 | 86.37 | 6.312 / 7.028 | 6,876 | — |
| 1 | 0 | STRIPED b3 | 17.793 / 17.763 / 17.767 | 17.774 | 86.42 | 6.312 / 7.028 | 6,876 | — |
| 1 | 1 | SINGLE b3 | 20.322 / 20.274 / 20.405 | 20.334 | 75.54 | 6.312 / 7.028 | 6,876 | 17.503 |
| 1 | 2 | STRIPED b2 | 12.268 / 12.436 / 12.232 | 12.312 | 83.17 | 5.345 / 5.545 | 5,462 | — |
| 1 | 3 | SINGLE b2 | 17.640 / 17.948 / 17.743 | 17.777 | 57.60 | 5.345 / 5.545 | 5,462 | 11.913 |
| 1 | 4 | STRIPED b1 | 8.515 / 8.650 / 8.548 | 8.571 | 59.74 | 4.378 / 4.798 | 4,750 | — |
| 1 | 5 | SINGLE b1 | 17.491 / 17.626 / 17.408 | 17.508 | 29.24 | 4.378 / 4.798 | 4,750 | 6.490 |
What §2 note 5 asks for, for every batch-2 and batch-3 arm (the card is 8.518 GB; both rounds
read the same): process dedicated / shared GPU memory 5.731 / 2.229 GB (SINGLE b2), 7.214 / 2.229
(SINGLE b3), 5.733 / 3.303 (STRIPED b2), 7.216 / 3.303 (STRIPED b3); peak_reserved_gb 5.545 at
batch 2 and 7.028 at batch 3; num_alloc_retries 0 everywhere. The process's shared memory is the
same at batch 1 (2.229 GB SINGLE, 3.303 GB STRIPED), so it does not grow with the batch.
Power and SM clock over the timed steps of every batch-3 arm (median / minimum; utilisation 100% in all four): SINGLE b3 round 0 96.6 / 76.5 W, 2141 / 1620 MHz; round 1 106.7 / 11.3 W, 2362 / 1230 MHz. STRIPED b3 round 0 110.1 / 82.0 W, 2415 / 2340 MHz; round 1 112.8 / 79.3 W, 2422 / 2130 MHz.
The read-free floor, in session (mean of the two rounds' B): 6.500 s at 512 tokens (model 6.425 s), 11.993 s at 1024 (model 11.609 s), 17.535 s at 1536 (model 16.792 s). The least-squares line through them: 0.975 s + 0.010776 s x tokens (largest residual 0.016 s). Its crossover with this session's R: SINGLE 1,610 tokens, STRIPED 730 tokens.
The ids: row 0 of the batch-2 draw and row 0 of the batch-3 draw both equal the batch-1 draw.
The peak VRAM arithmetic of §2 (judged by nothing) predicted 5.345 GB allocated at batch 2 and 6.312 GB at batch 3; the arms read 5.345 and 6.312.
The rule's output, python benchmarks/harness/batch_control_probe.py --summarize --run-prefix batch2, recomputed from the committed files. It is identical to the table the driver printed at
the end of the run:
row 1 SINGLE b1 outside 15.0-21.5 s: no fire
row 2 off the path, or the protocol stopped: no fire
row 3 STRIPED b1 outside 7.69-12.0 s: no fire
row 4 SINGLE b2: fits
row 4 SINGLE b3: fits
row 5 SINGLE b2: NEAR-FREE: e 1.027 (r0 1.068, r1 0.985), tok/s ratio 2.053
row 5 SINGLE b3: PARTIAL: e 0.899 (r0 0.938, r1 0.861), tok/s ratio 2.698
row 6 SINGLE b2: AGREES: measured NEAR-FREE, model NEAR-FREE (e_pred 1.000)
row 6 SINGLE b3: DISAGREES AT b3: measured PARTIAL, model NEAR-FREE (e_pred 1.000)
row 6 SINGLE crossover: measured (1024, 1536] tokens; model 1688 tokens at R 18.327 s
row 4 STRIPED b2: fits
row 4 STRIPED b3: fits
row 5 STRIPED b2: PARTIAL: e 0.713 (r0 0.731, r1 0.696), tok/s ratio 1.427
row 5 STRIPED b3: PARTIAL: e 0.497 (r0 0.512, r1 0.482), tok/s ratio 1.492
row 6 STRIPED b2: AGREES: measured PARTIAL, model PARTIAL (e_pred 0.761)
row 6 STRIPED b3: AGREES: measured PARTIAL, model PARTIAL (e_pred 0.526)
row 6 STRIPED crossover: measured (512, 1024] tokens; model 750 tokens at R 8.839 s
row VERDICT: per layout: rows 4-6 above
Also recorded, not judged.
- Round 0's SINGLE b1 (19.146 s, steps rising 18.71 -> 19.55 s) is 9.4% slower than round 1's
(17.508 s). The reversed second round cancels a linear drift only in the mean, which is what row
5 reads (§2, note 1); both rounds'
eare quoted beside each class above. Attempt 1's half-rate C: read did not recur: every SINGLE batch-1 step of this attempt was 17.41-19.55 s, against 30.1-34.0 s for five of attempt 1's six (the sixth, try 2's first step, read 18.22 s). Why round 0's first arm was slower is not established. - The second drive at batch 3: SINGLE b3 20.378 s against STRIPED b3 17.779 s (two-round means), 1.146x, against 2.073x at batch 1 (18.327 / 8.839 s). §0's arithmetic expected about 4% at batch 3 (17.44 / 16.79 s). The STRIPED batch-3 step sits 1.4% above the in-session floor at 1536 tokens.
6. Verdict
From attempt 2 (§5.2), by the rule of §2 as committed, applied by batch_control_rule.py:
Rows 1-3 do not fire. SINGLE batch 1 read 19.146 and 17.508 s (band 15.0-21.5 s), no arm was off the path and the protocol completed, and STRIPED batch 1 read 9.108 and 8.571 s (band 7.69-12.0 s). Both layouts are judged.
SINGLE, the L2L control (R = 18.327 s; the code caveat of §1 applies):
| b | row 4 | row 5 class | ē (r0, r1) |
r̄ |
tok/s r0 / r1 | peak alloc | in use | labels | row 6 |
|---|---|---|---|---|---|---|---|---|---|
| 1 | — | (NEAR-FREE by definition) | — | 1 | 26.74 / 29.24 | 4.378 GB | 4,750 MiB | — | — |
| 2 | fits | NEAR-FREE | 1.027 (1.068, 0.985) | 2.053 | 57.13 / 57.60 | 5.345 GB | 5,462 MiB | none | AGREES (model NEAR-FREE, e_pred 1.000) |
| 3 | fits | PARTIAL | 0.899 (0.938, 0.861) | 2.698 | 75.21 / 75.54 | 6.312 GB | 6,876 MiB | none | DISAGREES AT b3 (model NEAR-FREE, e_pred 1.000) |
Measured crossover bracket (1024, 1536] tokens; the model's crossover at this session's R is 1,688 tokens, and the in-session floor line's is 1,610 tokens (§2 note 7). Neither label fires at batch 3: 6,876 MiB in use is below 7,620 MiB, and the steps (20.422 / 20.334 s) are below read plus compute in series (35.938 / 34.300 s). The largest batch that fits is 3, the largest one run. §2 note 4 named this outcome before the run: batch 3 sits on the model's edge, and a PARTIAL there is a DISAGREES that this record reports as such. In this session the floor at 1536 tokens (17.535 s) is 4.3% under R, and the step exceeds both.
STRIPED, an extra (R = 8.839 s):
| b | row 4 | row 5 class | ē (r0, r1) |
r̄ |
tok/s r0 / r1 | peak alloc | in use | labels | row 6 |
|---|---|---|---|---|---|---|---|---|---|
| 1 | — | (NEAR-FREE by definition) | — | 1 | 56.22 / 59.74 | 4.378 GB | 4,750 MiB | — | — |
| 2 | fits | PARTIAL | 0.713 (0.731, 0.696) | 1.427 | 82.16 / 83.17 | 5.345 GB | 5,462 MiB | none | AGREES (model PARTIAL, e_pred 0.761) |
| 3 | fits | PARTIAL | 0.497 (0.512, 0.482) | 1.492 | 86.37 / 86.42 | 6.312 GB | 6,876 MiB | none | AGREES (model PARTIAL, e_pred 0.526) |
Measured crossover bracket (512, 1024] tokens; the model's is 750 tokens at this session's R, and the in-session line's is 730 tokens.
Nothing in §2 was edited. The records are local, on probe/batch-control, and not pushed.