Skip to content

Official Inference Code of EAGLE-4

docs(pkg): reconcile release guides with qualified artifacts and engines

Seyfal Sultanov354f4f52d152
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1ecb6d6..166fb5a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,6 @@
- Serve EAGLE-4 drafts with SGLang on NVIDIA B300 and AMD MI355X.
- Serve EAGLE-4 drafts with vLLM and the experimental TokenSpeed integration.
- Use packaged cost profiles to select speculative decoding schedules.
-- Install the harness from wheels or source with the matching engine wheels and platform-specific quickstarts.
+- Install the `eagle-harness` package from wheels or source with the matching engine wheels and platform-specific quickstarts.
See the [0.1.0 release notes](docs/release-notes/0.1.0.md) for supported configurations and known limits.
diff --git a/CITATION.cff b/CITATION.cff
index adcd3d8..61dd01a 100644
--- a/CITATION.cff
+++ b/CITATION.cff
@@ -3,7 +3,7 @@ message: "If you use eagle-harness in your work, cite the EAGLE-4 paper below."
type: software
title: "eagle-harness"
abstract: >-
- EAGLE-Harness for sglang and vLLM: cost-model-driven dynamic-tree speculative decoding with EAGLE-4 drafts, as an
+ EAGLE-4 for SGLang, vLLM and experimental TokenSpeed: cost-model-driven dynamic-tree speculative decoding with EAGLE-4 drafts, as an
out-of-tree engine plugin.
authors:
- name: "Eagle AI Team"
@@ -19,7 +19,7 @@ preferred-citation:
type: generic
title: "EAGLE-4: One Flexible Drafter for All Batch Sizes and Hardware"
authors:
- - name: "Eagle AI Team"
+ - name: "EAGLE-4 Team"
year: 2026
notes: "Preprint"
url: "https://github.com/eagle-hq/EAGLE-4/blob/main/EAGLE4_Paper.pdf"
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 3057938..cedb464 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -37,7 +37,7 @@ python -m venv .venv && source .venv/bin/activate
pip install -e ".[test]"
```
-Install sglang, vLLM or TokenSpeed with the eagle-harness hooks ([Compatibility](docs/compatibility.md)) to run that engine's
+Install SGLang, vLLM or experimental TokenSpeed with the eagle-harness hooks ([Compatibility](docs/compatibility.md)) to run that engine's
tests. Each engine's tests skip without it.
## Test
@@ -52,11 +52,14 @@ A change to a kernel needs a test against a reference (torch or the kernel it re
needs a measurement with its hardware, model, workload and baseline. `tools/greedy_parity.py` compares a speculative
server token for token with a server without speculation. `tools/bench_kernels.py` times kernels.
-To build the release files:
+To build the release files on Linux, install the build tools and set the source and output paths.
+The example assumes CUDA 12.8 and 13.0 toolkits are installed at the paths shown:
```bash
-tools/build_wheel.sh <source tree> <out dir> /usr/local/cuda-12.9 /usr/local/cuda-13.0
-python tools/check_dist.py <out dir>
+python -m pip install build auditwheel patchelf
+export SOURCE_TREE="$PWD" OUT_DIR="$PWD/dist-release"
+tools/build_wheel.sh "$SOURCE_TREE" "$OUT_DIR" /usr/local/cuda-12.8 /usr/local/cuda-13.0
+python tools/check_dist.py "$OUT_DIR"
```
## Commit messages
diff --git a/SECURITY.md b/SECURITY.md
index 47fa033..aeb0b34 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -3,7 +3,7 @@
Report a vulnerability privately, never in a public issue, discussion or pull request. Use either:
- GitHub's private vulnerability reporting: the "Report a vulnerability" button on this repository's Security tab, or
-- email to [email protected].
+- email to <[email protected]>.
Include the affected version or commit, the steps to reproduce, the impact you expect, any proof-of-concept, and
whether you disclosed the issue anywhere else. Do not test beyond what you need to confirm the issue, and do not
@@ -13,7 +13,7 @@ Disclose publicly only after the maintainers have had a reasonable chance to inv
## Scope
-eagle-harness runs inside a serving engine (sglang or vLLM), in the engine's processes. Report a vulnerability of
+eagle-harness runs inside a serving engine (SGLang, vLLM or experimental TokenSpeed), in the engine's processes. Report a vulnerability of
the engine itself to that project. Report one in eagle-harness here, including one that the plugin makes reachable
in the engine. The native CUDA library loads from `$EAGLE_HARNESS_NATIVE_LIB` when that variable is set, so treat it
like `LD_PRELOAD`: anyone who can set it can run code in the server.
diff --git a/docker/sglang.Dockerfile b/docker/sglang.Dockerfile
index 7767318..f578012 100644
--- a/docker/sglang.Dockerfile
+++ b/docker/sglang.Dockerfile
@@ -15,7 +15,8 @@ ENV VIRTUAL_ENV=/opt/eagle
ENV PATH="/opt/eagle/bin:${PATH}"
RUN . /opt/eagle-release.env \
&& python3 -m pip install --no-cache-dir --extra-index-url "$EAGLE_CUDA_INDEX" \
- --extra-index-url "$EAGLE_TORCH_CUDA_INDEX" "sglang==$EAGLE_SGLANG_VERSION" 'flashinfer-python>=0.7.2' \
+ --extra-index-url "$EAGLE_TORCH_CUDA_INDEX" --extra-index-url "$EAGLE_NVIDIA_INDEX" \
+ "sglang==$EAGLE_SGLANG_VERSION" 'flashinfer-python>=0.7.2' \
&& python3 -m pip install --no-cache-dir --no-deps --force-reinstall /src \
&& python3 -m pip check \
&& eagle-harness calibrate --help
diff --git a/docs/README.md b/docs/README.md
index c2a6d03..2b784f8 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -6,7 +6,8 @@ last_verified: 2026-10-19
# eagle-harness documentation
-Install a pinned engine and check generation with live costs on SGLang or a fixed tree on vLLM.
+Install a pinned engine and check generation with live costs on SGLang or calibrated costs and the shipped
+batch-dependent tree schedule on vLLM.
TokenSpeed is experimental. Measure the setup you will deploy.
Start with the measured model pairs rather than treating config admission as proof of support.
diff --git a/docs/architecture.md b/docs/architecture.md
index 5311155..207f696 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -43,10 +43,10 @@ vLLM runs the method on its Model Runner V2. The plugin uses its speculative-met
`prepare_verify_inputs`, `make_rejection_sampler` and `post_accept`, the linear-attention verify extension for GDN
layers, and the `CUSTOM` attention backend slot.
-TokenSpeed runs the method as a drafter on its plugin registries. The plugin uses `register_drafter` and
-`register_model` from TokenSpeed main and generic hooks of the eagle-hq TokenSpeed build: the method's JSON
+Experimental TokenSpeed runs the method as a drafter on its plugin registries. The plugin uses `register_drafter` and
+`register_model` and generic hooks of the pinned TokenSpeed release build: the method's JSON
config (`--speculative-algorithm-config`, delivered to `BaseDrafter.configure_algorithm`), the target's aux hidden
-states for a plugin algorithm, and tree drafting by a plugin drafter. The same branch lets a plugin draft bind its
+states for a plugin algorithm, and tree drafting by a plugin drafter. That build lets a plugin draft bind its
cache under a hybrid target and adds sliding-window attention to tree lanes. See [TokenSpeed](#tokenspeed) below.
## One decode round
@@ -73,9 +73,11 @@ prepare the next round.
request it is the target's argmax. For a sampled request it is a draw from the target's distribution at that
node (temperature, top-k, top-p, min-p): on vLLM the draw vLLM's sampler makes at that position, with noise
keyed by the request's seed and the node's depth; on sglang, sglang's tree speculative sampling. The accepted
- path plus one bonus token from the target are committed. Every committed token is the target's own token given
- the accepted prefix, so the output has the target's distribution. Greedy output matches greedy decoding without
- speculation, up to near-ties in the target's logits that a different batch shape can flip. The accepted rows'
+ path plus one bonus token from the target are committed. Verification uses target scores at the accepted prefix.
+ This describes the algorithm, not proof of distribution equivalence or exact no-spec output on every
+ implementation and model. Different verification shapes can change target arithmetic, including cases outside
+ the near-tie gate. Check the [measured correctness coverage](support-matrix.md#correctness-and-request-features).
+ The accepted rows'
KV and GDN states are moved to the request's chain positions.
6. The draft extend runs the draft over the accepted tokens, which updates its KV cache and yields the next round's
first candidates.
@@ -103,8 +105,8 @@ eagle-harness supplies the tree policy and the draft model:
(`tests/test_tokenspeed_adapter.py`). `eagle_harness/tokenspeed/tree_lanes.py` turns the frontier choices into
TokenSpeed's 64-bit lane masks (each lane sees its ancestors' draft slots) and the verify window and parents.
-The EAGLE-4 draft layer has a sliding window. TokenSpeed main refuses a sliding-window layer in its tree lanes; the
-hooks branch adds that path (every lane of a request sits at one position, so all lanes see the same committed keys).
+The EAGLE-4 draft layer has a sliding window. The pinned TokenSpeed release build supports sliding-window
+attention in tree lanes (every lane of a request sits at one position, so all lanes see the same committed keys).
Serve mode captures the target, sampling and draft extend together. The host launches the first draft-depth graph.
The GPU cost policy decides whether to continue and tail-launches later depths and finalization. Calibration parsing,
diff --git a/docs/calibration.md b/docs/calibration.md
index 92d3b6b..836413b 100644
--- a/docs/calibration.md
+++ b/docs/calibration.md
@@ -8,8 +8,8 @@ last_verified: 2026-10-20
## Costs measured on the served rounds
-On SGLang the cost-model modes need no offline step: without `data_path`, the server prices the draft step and the
-target verify on its own served rounds. The first rounds at each batch bucket (a captured graph batch size) force
+On SGLang the cost-model modes need no offline step. Without `data_path`, a matching packaged MI355X profile
+is selected when available. Otherwise, the server prices the draft step and target verify on its own served rounds. The first rounds at each batch bucket (a captured graph batch size) force
every captured verify width a few times, timed with CUDA events around the exact kernels served at the real
context length; after that every round refines its cell (an exponential moving average per batch bucket, verify
width and 1024-token context interval) and the cost model reads the resulting tables. The cells are cached under
@@ -38,16 +38,20 @@ on your allocated GPUs first.
## Run the command
-SGLang requires the host graph contract with per-variant forward modes. The calibration driver measures
-native decode as well as speculative chains. An older host without that contract fails with an explicit error.
+The released offline calibration path is vLLM. SGLang served-chain calibration requires the host graph
+contract with per-variant forward modes because it also measures native decode. The pinned SGLang CUDA
+release engine does not expose that contract, so the CLI's SGLang calibration path fails with an explicit
+error on that build. Use the [SGLang live-cost serving recipe](quickstart-sglang.md) without an offline step.
+The SGLang commands below document the driver interface for a host with the required contract, not a
+working calibration recipe on the pinned release engine.
Set `MODEL`, `DRAFT` and `TP` to your target, compatible exported draft and tensor-parallel size:
```bash
-eagle-harness calibrate --model "$MODEL" --draft "$DRAFT" --tp "$TP"
+eagle-harness calibrate --engine vllm --model "$MODEL" --draft "$DRAFT" --tp "$TP"
```
-The command selects the sole installed engine. If both engines are installed, specify `--engine sglang` or
+Without `--engine`, the CLI selects the sole installed engine. If both engines are installed, specify `--engine sglang` or
`--engine vllm`. TP defaults to one. No private dataset or calibration directory is required.
| Engine | Default measurement | Default output |
@@ -55,8 +59,8 @@ The command selects the sole installed engine. If both engines are installed, sp
| vLLM | Batch-one tree, verify widths 8, 16, 32 and 64, with context lengths below 4096 | `eagle-calibration-vllm/` |
| SGLang | Served chains up to width 8, at concurrency 1 and 8, with 1024 new tokens per request | `eagle-calibration-sglang/` |
-SGLang requires a package build containing the served-chain calibration driver. It does not produce dynamic-tree
-cost tables. vLLM prices isolated whole steps and draft graph replays, not a full high-concurrency served round.
+The package includes the SGLang served-chain calibration driver. It requires the host graph contract above
+and does not produce dynamic-tree cost tables. vLLM prices isolated whole steps and draft graph replays, not a full high-concurrency served round.
Neither command establishes a speedup or replaces a correctness check against no-spec generation.
Preview the settings without importing an engine, loading a model or creating output:
@@ -193,7 +197,8 @@ The `a_d` come from `<data_path>/accept/chain.json`:
These are vLLM's `spec_decode_num_drafts` and `spec_decode_num_accepted_tokens_per_pos` counters after serving the
workload with the widest chain. Acceptance depends on the workload (short answers accept differently from long
-ones), not on the batch size, so measure it on the requests you serve. Widths past the measured positions are not
+ones). The policy reuses this profile across batch sizes, so measure representative requests and check that
+it remains appropriate at the concurrency you serve. Widths past the measured positions are not
considered. Without the file, or without timings that price a chain at every calibrated batch size, chains verify
`max_verify_width` tokens at every batch size.
diff --git a/docs/compatibility.md b/docs/compatibility.md
index 7197879..c9ecf65 100644
--- a/docs/compatibility.md
+++ b/docs/compatibility.md
@@ -20,7 +20,7 @@ packages include the hooks or that every configuration in the range has been mea
| vLLM | `>=0.32.0rc1.dev283,!=0.32.0` with hooks and Model Runner V2 | `EAGLE_VLLM_VERSION` |
| TokenSpeed, experimental | `>=0.1.1,<0.2` with hooks and matching native packages | `EAGLE_TOKENSPEED_VERSION` and `EAGLE_TOKENSPEED_AUX_REQUIREMENTS` |
| FlashInfer, CUDA SGLang | `>=0.7.2` | CUDA installation, not ROCm |
-| Python | Package metadata allows 3.10 and newer | Release recipes use 3.12 |
+| Python | Package metadata allows 3.10 and newer | CUDA recipes use 3.12, the ROCm wheel requires 3.10 |
| PyTorch and GPU runtime | Match the engine's dependency closure | [CUDA and ROCm requirements](install.md#requirements) |
| GPU and model pair | Configuration-specific qualification | [Support matrix](support-matrix.md) |
diff --git a/docs/configuration.md b/docs/configuration.md
index 9e6318b..4847871 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -15,7 +15,7 @@ For legacy environment variables and CLI flags, see [Migration](migration.md).
| key | type | default | meaning |
|---|---|---|---|
| `mode` | string | `"serve"` | `serve`: the cost model sizes every round. `eval`: cost sizing with evaluation settings (see the inactive-policy note below). `calibrate`: reserved for cost measurement (see [Calibration](calibration.md)). `static`: the fixed tree of the shape flags every round, no cost model (on SGLang the same EAGLE-4 round, graphs and kernels as the cost-model modes). |
-| `data_path` | string | none | Offline calibration directory whose `cost/` subdirectory holds the cost tables and whose `key.json` names the setup they were measured on (GPU, TP size, model pair, engine, dtype); a directory keyed for another setup is refused. Without it, the cost-model modes measure the costs on the served rounds ([Calibration](calibration.md#costs-measured-on-the-served-rounds)). |
+| `data_path` | string | none | Offline calibration directory whose `cost/` subdirectory holds the cost tables and whose `key.json` names the setup they were measured on (GPU, TP size, model pair, engine, dtype); a directory keyed for another setup is refused. On SGLang, omitting it uses a matching packaged profile when available, otherwise live served-round costs ([Calibration](calibration.md#costs-measured-on-the-served-rounds)). vLLM and TokenSpeed require the engine-specific inputs below. |
| `cost_cache` | string | `~/.cache/eagle-harness/costs` | Directory of the live-cost cache files (one per setup key). |
| `intensity` | string | `"high_intensity"` | Draft shape family of the cost tables: `high_intensity`, `low_intensity` (trees) or `chain_intensity` (one candidate per step). It sets the batch limit and the verify widths that get a CUDA graph. |
| `draft_family` | string | `"eagle4"` | `eagle4`. Sets the cost model's depth loss when `cost_model.depth_loss_ratio` is not given. |
@@ -115,7 +115,8 @@ Tree modes keep one physical verify width per server. Serve-mode trees pad unuse
Pass `--sampling-backend greedy` because the default FlashInfer sampler cannot verify draft trees.
For calibrated tree serving, provide `data_path` containing the calibration `cost/` directory.
-The README example uses a running cap of 32, not a general performance recommendation.
+The [TokenSpeed quickstart](quickstart-tokenspeed.md) distinguishes the single-client cap of 4 from the
+concurrent cap of 32. Neither is a general performance recommendation.
The calibration must match the target, draft, hardware, tensor-parallel setup and recurrent-state dtype.
`eval`, `calibrate` and live costs are not supported by this adapter. Chains use top-k 1 and `chain_intensity`.
An explicit `chain_width_table` contains increasing `[minimum_batch_size, verify_width]` pairs in `serve` mode.
@@ -129,7 +130,8 @@ not a measured high-concurrency speed claim. Sampled-tree distribution quality i
| `--speculative-num-draft-tokens` | Physical verify width, root included, large enough for the selected top-k |
The lane window `(steps - 1) * topk` must fit in 64 slots. Its KV reservation is independent of the verify width,
-so depth 12, top-k 4 and verify width 16 are supported in serve mode.
+so depth 12, top-k 4 and verify width 16 are supported in serve mode. Static mode also encodes a
+depth-first sort key and supports at most ten steps. This is distinct from the lane-window limit.
## Shape limits
diff --git a/docs/faq.md b/docs/faq.md
index 8ac56b6..adb6fd5 100644
--- a/docs/faq.md
+++ b/docs/faq.md
@@ -22,7 +22,8 @@ so only the draft depth adapts per round.
## Do I need cost tables?
-SGLang measures costs on the served rounds when no `data_path` is supplied. Offline tables remain an option.
+When no `data_path` is supplied, SGLang selects a matching packaged MI355X profile if available, otherwise
+it measures costs on served rounds. Offline tables remain an option.
vLLM static trees need no tables, while automatic policies require matching measured costs. Experimental
TokenSpeed needs offline costs for calibrated decisions, or a complete explicit chain schedule.
See [calibration](calibration.md).
@@ -58,10 +59,12 @@ SGLang and vLLM support sampled requests with the engine-specific restrictions b
`tools/sampling_check.py` compares a speculative server with a no-spec server. Sampling smoke tests alone
do not establish distribution equivalence for every model and workload.
-- vLLM: a seeded request decodes the same text as without speculation, up to the logits' numerics. vLLM itself
+- vLLM: seeded sampling uses the target sampler, but an identical seed is not a blanket guarantee of the same
+ no-spec text or distribution qualification. vLLM itself
refuses min-p and logit bias with any speculative method. A batch with
repetition, frequency or presence penalties, bad words, a thinking budget or a custom logits processor is
- verified as chains. That is exact but accepts fewer tokens.
+ verified as chains using the host rejection sampler, which can accept fewer tokens. This fallback is not a
+ claim of token-identical output across different numerical execution shapes.
- sglang: sampled requests go through sglang's own tree speculative sampling. min-p, and top-k together with top-p,
need an sglang build with the verify filter fix: older builds ignore min-p in verify and cut top-p after top-k.
Penalties use sglang's relaxed form (counts of committed tokens only), as in sglang's own EAGLE, so they are not
diff --git a/docs/install.md b/docs/install.md
index 9ca25a5..ca3fc10 100644
--- a/docs/install.md
+++ b/docs/install.md
@@ -6,7 +6,7 @@ last_verified: 2026-10-20
# Install eagle-harness 0.1.0
-Install one patched engine per environment, then install `eagle-harness==0.1.0` from PyPI.
+At release, install one patched engine per environment, then install `eagle-harness==0.1.0` from PyPI.
Stock engine packages do not provide the required plugin hooks. Choose the pip or Docker path below,
then follow that engine's quickstart.
@@ -14,11 +14,19 @@ then follow that engine's quickstart.
[Release settings](release.env) is the single source for package pins, wheel indexes and container tags.
The engine wheels belong to the GitHub release named by `EAGLE_RELEASE_ASSETS`. Its PEP 503 indexes
-separate CUDA and ROCm wheels. `eagle-harness` is a separate PyPI package.
-
-These are release destinations, not evidence that publication has completed. Before distribution,
-the release owner must publish the wheels, indexes and images and qualify a clean installation.
-The recipes have not been clean-installed or container-built as part of this documentation check.
+separate CUDA and ROCm wheels. The `eagle-harness` release path is PyPI, with its wheels and source distribution
+also attached to the same GitHub Release. Publishing the public release runs `publish.yml` with PyPI trusted publishing.
+
+These are release destinations, not evidence that publication has completed. The PyPI project and public
+artifacts are not yet available for this draft. Fresh CUDA environments installed private wheels and served
+real requests, and the `eagle-harness` wheel and source-distribution forms passed installation checks. Those checks
+used a private index, not the public URLs. The private SGLang image also passed serving and correctness
+checks. vLLM image qualification is pending, and the TokenSpeed image has not been built. Final release
+artifacts and registry publication require separate checks.
+If PyPI is unavailable after publication, download the matching `eagle-harness` wheel from the GitHub Release
+named by `EAGLE_RELEASE_ASSETS`, verify its release checksum, and install it with
+`python -m pip install /path/to/downloaded/eagle_harness-0.1.0-py3-none-any.whl` in the engine environment.
+Before the release assets exist, neither path is available.
Do not replace an unavailable artifact with the stock engine or an unpinned package.
Use the release checkout containing these docs. Run commands from its root:
@@ -52,12 +60,16 @@ python3.12 -m venv .venv-sglang
source .venv-sglang/bin/activate
python -m pip install --upgrade pip
python -m pip install --extra-index-url "$EAGLE_CUDA_INDEX" \
- --extra-index-url "$EAGLE_TORCH_CUDA_INDEX" "sglang==$EAGLE_SGLANG_VERSION" 'flashinfer-python>=0.7.2'
+ --extra-index-url "$EAGLE_TORCH_CUDA_INDEX" --extra-index-url "$EAGLE_NVIDIA_INDEX" \
+ "sglang==$EAGLE_SGLANG_VERSION" 'flashinfer-python>=0.7.2'
python -m pip install "eagle-harness==$EAGLE_VERSION"
eagle-harness-check
```
-For Docker, open the release image and check the hooks:
+The NVIDIA index supplies the pinned `cuda-tile==1.6.0rc5` wheel. It avoids relying on a source build
+from PyPI. The private install check also succeeded with that exact dependency built from its PyPI source.
+
+After the image is published and qualified, open it and check the hooks:
```bash
docker run --rm --gpus all --ipc host -it \
@@ -71,8 +83,8 @@ Inside the container, use `/models` paths and `--host 0.0.0.0` in the
## SGLang ROCm on AMD
-The fresh-install path is under qualification and deferred to a follow-up guide. No pip or Docker
-installation command is qualified here. The ROCm wheel requires Python 3.10 (`cp310`), not Python 3.12,
+The private offline pip closure is qualified, but its fresh-install commands are deferred to the AMD
+follow-up guide. No public pip or Docker installation command is qualified here. The ROCm wheel requires Python 3.10 (`cp310`), not Python 3.12,
and a complete compatible ROCm 7.2 runtime. A successful import or `eagle-harness-check` alone does not
qualify the dependency closure for serving.
@@ -94,7 +106,8 @@ python -m pip install "eagle-harness==$EAGLE_VERSION"
eagle-harness-check
```
-For Docker, mount a writable directory for the calibration output:
+The vLLM image has been built, but its serving and correctness checks are pending. After qualification
+and publication, mount a writable directory for the calibration output:
```bash
mkdir -p calibration
@@ -111,8 +124,10 @@ place calibration output under `/calibration`, and bind the server to `0.0.0.0`.
TokenSpeed requires the matching engine, scheduler and kernel wheels pinned in `release.env`.
Install all three from the CUDA release index. Stock scheduler and kernel wheels are not substitutes.
-Keep an nvcc-equipped CUDA 13.0 toolkit available: tree-round CUDA sources compile at first use even
-though the runtime wheel itself has a `py3-none-any` tag. The scheduler and kernel wheels contain native code:
+Tree-round CUDA sources compile at first use even though the runtime wheel has a `py3-none-any` tag.
+The passing serve-mode check uses a host-compatible CUDA 13.2 JIT on Ubuntu 26.04 / glibc 2.43,
+with `CUDA_HOME=/usr/local/cuda-13.2`; the original CUDA 13.0 JIT fails on that host. This does not
+change the `cu130` wheel channel. The scheduler and kernel wheels contain native code:
```bash
python3.12 -m venv .venv-tokenspeed
@@ -127,7 +142,8 @@ python -m pip install "eagle-harness==$EAGLE_VERSION"
eagle-harness-check
```
-For Docker, use the complete experimental runtime image:
+No TokenSpeed image has been built or qualified. The following is the planned command after an
+approved image is qualified and published, not an available installation path:
```bash
docker run --rm --gpus all --ipc host -it \
@@ -156,7 +172,10 @@ The [troubleshooting guide](troubleshooting.md) covers hook failures and startup
## Dockerfile sketches
-The files under `docker/` document how to layer the release wheels and this source checkout onto
-engine runtimes. They are not qualified image builds. The ROCm and TokenSpeed sketches require an
+The files under `docker/` document how to layer release wheels and this source checkout onto engine
+runtimes. The private image checks used a prebuilt `eagle-harness` wheel instead of the sketches' source install,
+so they do not qualify the Dockerfiles verbatim. The corrected SGLang TP4 image passed hook, dependency,
+request and twenty-prompt correctness checks. vLLM image qualification is pending. TokenSpeed has no
+built image, and ROCm has no container qualification. The ROCm and TokenSpeed sketches require an
explicit compatible `BASE_IMAGE`; do not infer a tested base from a successful syntax check.
-The release tags above are the user-facing Docker path. Building or publishing them is a separate release step.
+Final rebuilds and publication under the release tags are separate release steps.
diff --git a/docs/kernels.md b/docs/kernels.md
index 465794a..8a51fbc 100644
--- a/docs/kernels.md
+++ b/docs/kernels.md
@@ -23,8 +23,9 @@ python -m eagle_harness.kernels.arch
On a GPU outside these tiers, or without an importable Triton, the server refuses `EAGLE_HARNESS` at startup with the
reason, for example `... is older than NVIDIA sm_80 (A100)`.
-Runtime tests run on B300. The other architectures are compile-checked: nvcc builds the native library for each of
-them and Triton cross-compiles the kernels for each target.
+Runtime serving measurements cover B300 and SGLang on MI355X. Other listed architectures have compilation
+coverage, not serving qualification. Native CUDA builds apply only to NVIDIA targets. See the
+[support matrix](support-matrix.md) for the measured engine and model combinations.
## The kernels
diff --git a/docs/known-limits.md b/docs/known-limits.md
index bdafe4a..abffbe4 100644
--- a/docs/known-limits.md
+++ b/docs/known-limits.md
@@ -20,6 +20,26 @@ A loaded model, an admitted request cap or a compiled kernel does not establish
- Throughput depends on the model, draft, engine, precision, TP size, attention backend, context length and
workload. A result at one concurrency does not establish a result at another.
+## Performance and correctness gaps
+
+- SGLang TP4 single-request and 32K-context optimization remain open. TP8 concurrency-8 prefill
+ overhead and intermittent stalls remain unresolved. Repeated passes in one process are not
+ independent server starts.
+- SGLang automatic high-load policy, fixed-chain improvements and sampled-tree improvements are not
+ qualified release defaults. High-load and Nemotron first divergences remain open, as does TP4
+ native-OFF arbitration. A release-engine cap-512 startup and request smoke passed, but it did not
+ establish high-load acceleration or concurrent correctness.
+- The shipped vLLM TP4 rounds schedule improves the measured greedy concurrency-64 case. The
+ [support matrix](support-matrix.md#measured-high-load-tree-schedule) retains its no-spec-control and
+ correctness limits. The schedule has no TP1/TP8 profile, and higher concurrency and temperature-1
+ throughput are not qualified by that greedy result. Native-OFF is disabled by default and does not
+ reach native-decode throughput in the measured TP1 cases.
+- vLLM can drop EAGLE-4 INFO messages during plugin registration. An absent schedule-load line is
+ not proof that the schedule is inactive. See [troubleshooting](troubleshooting.md#slower-than-expected).
+- AMD measurements apply to the MI355X TP1 Qwen3.8 profile. HIP side-stream prelaunch stays disabled
+ because its replay check failed. Remaining target, draft and verify phase overheads are separate
+ from the measured-profile throughput results.
+
## Requests
- Client concurrency can exceed the server's running cap and include queued requests. Keep both values with
@@ -41,6 +61,12 @@ configurations, not every implemented path. The sampled-output distribution is u
Prompt logprobs, per-request sampled-tree overrides, live cost measurement and native no-speculation rounds
are not supported. A zero-draft speculative round is not a native no-speculation round.
Tree and chain cost data must match the served shape, TP size, target, draft and recurrent-state dtype.
+TP4 single-request optimization remains open. Release-stack TP1/TP8, long contexts, four-GPU chains,
+retraction under serving load and original-row output-logprob checks are incomplete. The reported sampled
+cap-64 path is refused. A CUDA static-mode install smoke generated text but retained one fp32-gate failure.
+The serve-mode install smoke passed its checked greedy prompts only with the host-compatible CUDA 13.2
+JIT. The original CUDA 13.0 JIT fails on Ubuntu 26.04 / glibc 2.43. Static mode is not qualified.
+These checks do not qualify arbitrary CUDA-toolkit and host-compiler combinations.
## Calibration
@@ -49,12 +75,16 @@ the model, draft, TP size, precision, attention backend or engine. Keyed tables
key does not encode every setting. A cache or table is not evidence of an optimal policy for every workload.
The vLLM public sweep measures batch-one tree forwards, not complete high-concurrency serving rounds.
-The SGLang served-chain driver measures a separate path from dynamic-tree calibration. Built-in smoke prompts
+The SGLang served-chain driver measures a separate path from dynamic-tree calibration. The pinned SGLang
+CUDA engine lacks its required per-variant native-decode graph contract, so the CLI path is unavailable
+on that engine. Live-cost tree serving does not require it. Built-in smoke prompts
check the command, not representative acceptance. A failed run retains partial output for diagnosis and does
not write a successful `serve.sh`. See [calibration](calibration.md).
## Distribution
-The [install guide](install.md) uses supplied wheel bundles and source builds. Its local index is not a claim
-of a public package service or anonymously accessible model weights. GPU serving and clean-machine container
-installation require their own checks.
+The [install guide](install.md) describes the release path: `eagle-harness` packages on PyPI and GitHub Release,
+patched engine wheels on the GitHub Pages indexes, and images on GHCR. These destinations do not establish
+publication or image qualification. Private-index CUDA installation and request checks do not test the public
+URLs. AMD has a qualified private offline pip closure, but its fresh-install commands remain deferred
+to the AMD guide update, and its Docker image is unqualified. Model and draft weights require separate access.
diff --git a/docs/new-model.md b/docs/new-model.md
index 5b39d30..bf84b22 100644
--- a/docs/new-model.md
+++ b/docs/new-model.md
@@ -20,8 +20,8 @@ First serve the target without speculation in your pinned engine. Confirm that i
precision and attention backend work. Keep those settings for the speculative comparison.
For the measured EAGLE4 ports, train a dense single-layer draft with rotary positions, full target vocabulary,
-three auxiliary hidden-state inputs and a low-rank candidate head. The draft query and KV head counts must match
-for the measured tree adapter. The target itself can use a hybrid architecture if the engine and EAGLE-4 support
+three auxiliary hidden-state inputs and a low-rank candidate head. Grouped-query draft attention is implemented, but the query and KV head counts must be compatible
+with the selected adapter and tensor-parallel layout. The target itself can use a hybrid architecture if the engine and EAGLE-4 support
its verify and state-commit paths.
Use the target's hidden size, vocabulary and token IDs when building your draft configuration. Keep its trained
@@ -161,9 +161,10 @@ with safe_open(root / "model.safetensors", framework="pt", device="cpu") as weig
PY
```
-The inspected Nemotron draft used this format, including a rank-128 candidate head, but also used MoE,
-attention normalization and no RoPE. Those structural fields are rejected by the measured release adapters.
-Do not remove them or claim Nemotron support from the format check.
+The inspected Nemotron draft used this format, including a rank-128 candidate head, latent MoE,
+attention normalization and no RoPE. SGLang and vLLM implement those structures. Preserve them in the
+export rather than removing fields to pass validation. Structural admission does not establish Nemotron
+release-stack performance or correctness qualification. See [known limits](known-limits.md#models-and-hardware).
## Check generation before calibration
@@ -187,11 +188,13 @@ feature out of the deployment contract.
Stop the test servers and measure the admitted pair:
```bash
-eagle-harness calibrate --model "$MODEL" --draft "$EXPORT" --tp "$TP" --output calibration-new-model &&
+eagle-harness calibrate --engine vllm --model "$MODEL" --draft "$EXPORT" --tp "$TP" --output calibration-new-model &&
sh calibration-new-model/serve.sh
```
-Specify `--engine` if both engines are installed. Supply a representative `--data` file for SGLang.
+This offline command is for vLLM. The pinned SGLang engine lacks the native-decode graph contract needed
+by the served-chain calibration CLI. Use its live-cost serving path instead. On a host exposing the required
+contract, SGLang chain calibration should use a representative `--data` file.
The [calibration guide](calibration.md) explains the different per-engine measurements. Benchmark against a
matched no-spec server at your deployment concurrency. Save the model and draft revisions, engine build,
GPU type, TP, precision and calibration settings alongside the result before calling the new pair supported.
diff --git a/docs/quickstart-tokenspeed.md b/docs/quickstart-tokenspeed.md
index 3c131aa..d92fd89 100644
--- a/docs/quickstart-tokenspeed.md
+++ b/docs/quickstart-tokenspeed.md
@@ -12,14 +12,19 @@ The release serving recipe uses Qwen3.8-27B BF16 and its EAGLE-4 draft on four B
## Before starting
Complete the [TokenSpeed installation](install.md#tokenspeed-cuda-experimental), including its scheduler
-and kernel wheels, and run `eagle-harness-check`. Keep the CUDA 13.0 toolkit and `nvcc` available for
-first-use tree-round compilation. Set `MODEL` and `DRAFT` to readable local directories.
+and kernel wheels, and run `eagle-harness-check`. Tree-round sources need `nvcc` at first use.
+On the tested Ubuntu 26.04 / glibc 2.43 host, the CUDA 13.0 JIT fails. The passing serve-mode
+check uses a host-compatible CUDA 13.2 JIT with `CUDA_HOME=/usr/local/cuda-13.2`. Set that path
+to the installed compatible toolkit before launching. The engine wheels still come from `cu130`.
+Set `MODEL` and `DRAFT` to readable local directories.
The measured cost-model recipe also requires matching calibrated target and draft tables under
`COST_ROOT/cost/`. EAGLE-4 does not include a TokenSpeed calibration CLI or automatically select
an installed TokenSpeed profile. Do not substitute vLLM or ROCm tables.
-If you do not have matching TokenSpeed costs, use the fixed-shape option below to check generation.
-It does not reproduce the cost-model benchmark or establish the same performance.
+Use `mode=serve` with matching costs for the checked path. Its twenty-prompt install check passed
+with 4 exact and 16 fp32-tolerance results. Static mode is not a qualified substitute when cost files
+are unavailable: its check has one fp32 gap of 0.178637, above the 0.125 tolerance.
+These are first-divergence checks, not bitwise output equality or sampled-distribution qualification.
## Start the server
@@ -33,23 +38,13 @@ test -d "$COST_ROOT/cost"
CONFIG=$(python -c 'import json, os; print(json.dumps({"mode":"serve","data_path":os.environ["COST_ROOT"]}))')
```
-Alternatively, select a fixed tree without cost files:
-
-```bash
-CONFIG='{"mode":"static"}'
-STEPS=10
-```
-
-The static path supports at most ten draft steps because the engine stores its depth-first sort key
-in 64 bits. Do not reuse the twelve-step cost-model bound with `mode=static`.
-
-Use the release launch flags with the selected configuration:
+Use the release launch flags with the serve-mode configuration:
```bash
: "${MODEL:?set the local Qwen3.8-27B BF16 target directory}"
: "${DRAFT:?set the local compatible EAGLE-4 draft directory}"
-: "${CONFIG:?select cost-model or fixed-shape mode above}"
-: "${STEPS:?set the step bound with the selected mode above}"
+: "${CONFIG:?construct the serve-mode configuration above}"
+: "${STEPS:?set the serve-mode step bound above}"
export HF_HUB_OFFLINE=1 TOKENIZERS_PARALLELISM=false OMP_NUM_THREADS=4
python -m tokenspeed.cli serve "$MODEL" --tensor-parallel-size 4 \
--attention-backend trtllm --max-model-len 65536 --max-num-seqs 4 \
@@ -63,8 +58,8 @@ python -m tokenspeed.cli serve "$MODEL" --tensor-parallel-size 4 \
The running-request cap is four in the single-client release recipe. The concurrent release recipe
uses `--max-num-seqs 32` with the same TP4 and context settings. Neither setting promises a speedup
-for your workload. The fixed-shape option changes the policy and draft-step bound, not the engine.
-In Docker, bind the server to `0.0.0.0` inside the container and keep the published host port on loopback.
+for your workload. A TokenSpeed release image has not been built or qualified.
+When a qualified image is available, bind the server to `0.0.0.0` inside the container and keep the published host port on loopback.
If using cost files in Docker, mount their directory read-only and set `COST_ROOT` to that container path.
## Generate text
diff --git a/docs/quickstart-vllm.md b/docs/quickstart-vllm.md
index 52c43e4..d589350 100644
--- a/docs/quickstart-vllm.md
+++ b/docs/quickstart-vllm.md
@@ -69,7 +69,9 @@ python -m vllm.entrypoints.cli.main serve "$MODEL" --tensor-parallel-size 4 \
Model Runner V2 and `CUSTOM` attention are required for this tree. The static shape supplies the bounds,
the calibrated costs supply depth stopping, and the rounds file selects width by batch size.
-Without `rounds/tree.json`, the configured width stays fixed. Do not point `data_path` at the packaged
+Registration-time INFO messages can be dropped before vLLM configures logging, so an absent schedule-load
+line does not prove the schedule is missing. Without `rounds/tree.json`, the configured width stays fixed.
+Do not point `data_path` at the packaged
rounds-only directory because it lacks the depth-stop cost tables.
## Generate text
diff --git a/docs/release-notes/0.1.0.md b/docs/release-notes/0.1.0.md
index b731317..b8b9883 100644
--- a/docs/release-notes/0.1.0.md
+++ b/docs/release-notes/0.1.0.md
@@ -7,7 +7,10 @@ last_verified: 2026-10-20
# Release notes for 0.1.0
This is a draft for the initial eagle-harness release, not a publication announcement. It describes merged source
-features. Final engine builds, install pins and release qualification remain pending. Performance evidence lives
+features. Engine pins are fixed in [release settings](../release.env). Private CUDA artifacts have installation
+and request evidence. The private AMD offline pip closure is qualified, but its guide update, container
+qualification and final release publication remain separate steps.
+Performance evidence lives
in the [measured support matrix](../support-matrix.md), not in release-wide speed claims.
## Serving and draft models
@@ -16,15 +19,17 @@ in the [measured support matrix](../support-matrix.md), not in release-wide spee
interface is missing or incompatible.
- Serve EAGLE-4 drafts with the low-rank candidate head and tree-aware GDN verification for hybrid targets.
Matching cost tables control SGLang draft depth and verify width, and vLLM draft-depth stopping.
-- Select vLLM chain widths from measured draft/verify costs and acceptance profiles. High-load behavior still needs a
- matched no-spec comparison.
-- Use SGLang Mamba2 tree-verification convolution through the engine extension. This alone does not include the
- real Nemotron draft integration or qualify its performance.
+- Select vLLM chain widths from measured draft/verify costs and acceptance profiles. The packaged TP4 tree
+ rounds schedule narrows verification at higher batch sizes. Its measured gains and qualification limits
+ are in the support matrix. Native-OFF remains disabled by default.
+- Use Mamba2 tree verification and latent-MoE Nemotron draft adapters on SGLang and vLLM. The adapters are
+ included, but release-stack Nemotron performance and correctness are not generally qualified.
+- Use the experimental CUDA-only TokenSpeed adapter with its matching engine, scheduler and kernel packages.
## Request correctness
- Return vLLM logprobs in accepted tree-path order rather than physical verify-row order.
-- Serve SGLang accepted-token, top-k and input logprobs with the matching host fixes or documented eager workaround.
+- Serve SGLang accepted-token, top-k and input logprobs with the matching release host fixes.
- Sample vLLM tree verification from the target distribution and preserve request-slot indexing during chunked
verification. SGLang sampled-filter correctness requires the corresponding host fix.
- Use root-only tree verification for vLLM grammar-constrained batches. This preserves constraints without
@@ -37,7 +42,8 @@ in the [measured support matrix](../support-matrix.md), not in release-wide spee
The installed `eagle-harness calibrate` command detects the engine, preserves raw measurements, refuses an
existing output directory and writes a serving command only after successful output checks. The public vLLM
-measurement/fitting driver is included. The SGLang served-chain driver remains a separate integration dependency.
+measurement/fitting driver and SGLang served-chain driver are included. The SGLang driver also requires
+the host native-decode graph contract described in [calibration](../calibration.md).
The source includes correctness checks for greedy output, sampling, mixed stop conditions, draft attention and
concurrent verification. GDN recurrence code has clearer tensor-layout and ownership documentation without an
@@ -52,5 +58,7 @@ Read [installation](../install.md), [known limits](../known-limits.md) and the m
The selected package versions and distribution destinations are in `docs/release.env`. Their presence does not
assert that artifacts have been published. AMD ROCm and CUDA-only TokenSpeed are included in the release source,
with qualification restricted to the configurations in the [support matrix](../support-matrix.md).
-Real Nemotron draft integration and the deferred optimizations in [known limits](../known-limits.md) remain
-outside that support contract. Final artifact installation and serving checks remain publication gates.
+Nemotron qualification and the deferred optimizations in [known limits](../known-limits.md) remain
+outside that support contract. AMD fresh-install commands are deferred, and TokenSpeed retains the
+experimental correctness and toolchain limits recorded there. Final artifact installation and serving checks
+remain publication gates.
diff --git a/docs/release.env b/docs/release.env
index b7b97ac..7509f0a 100644
--- a/docs/release.env
+++ b/docs/release.env
@@ -17,5 +17,7 @@ export EAGLE_ROCM_VERSION='0.5.23.dev252+eagle.4.rocm'
export EAGLE_VLLM_VERSION='0.32.0rc1.dev283+eagle.4'
export EAGLE_TOKENSPEED_VERSION='0.1.1+eagle.4'
export EAGLE_TORCH_CUDA_INDEX='https://download.pytorch.org/whl/cu130'
+# The pinned SGLang dependency cuda-tile==1.6.0rc5 has vendor wheels here.
+export EAGLE_NVIDIA_INDEX='https://pypi.nvidia.com'
# These are coupled native packages from the same TokenSpeed release source.
export EAGLE_TOKENSPEED_AUX_REQUIREMENTS='tokenspeed-scheduler==0.1.40+eagle.4 tokenspeed-kernel==0.1.4+eagle.4'
diff --git a/docs/support-matrix.md b/docs/support-matrix.md
index 9d76f40..f654934 100644
--- a/docs/support-matrix.md
+++ b/docs/support-matrix.md
@@ -35,8 +35,8 @@ Every timed pass follows a warm pass. EAGLE-4 measures live costs. Each EAGLE-4
| 8 | 1 | 1 | Irrelevant at batch 1 | 241.7 | 828.6 | 3.43× |
TP4 concurrency 8 measures 2749.5 and 2891.8 tokens/s across its two starts.
-TP8 concurrency 1 has a throughput result but its configuration-specific correctness check is incomplete.
-Correctness from another TP8 run is not substituted.
+TP8 concurrency 1 and 8 each give 5 exact / 14 near tie / 1 fp32-tolerance pass, with zero failures
+in the configuration-specific twenty-prompt checks. This does not resolve TP8 concurrency-8 throughput variability.
Concurrency 64 queues into 32 running slots. The measured padding-disabled settings identify those results,
not a recommendation to disable padding. Concurrent serving commands keep SGLang's default padding enabled.
@@ -96,23 +96,63 @@ The two TP8 concurrency-8 starts differ by 11%, and both are included in the mea
### Measured high-load tree schedule
-A one-start TP4 qualification of the served-round schedule reaches 2906.7 tokens/s at concurrency 8,
-compared with the fixed-width tree's two-start mean of 2883.9 above. At concurrency 64 it reaches
-7036.3 tokens/s, 1.025× its matched no-speculation control. These one-start results are not pooled with
-the fixed-width results. The twenty-prompt check gives 5 exact / 14 near tie / 1 previously arbitrated
-fp32-tolerance pass; the last case has the same common prefix as the earlier passing arbitration.
-This is not bitwise equality to no-speculation decoding.
+A short-workload check uses two independent starts of EAGLE-4,
+128 requests with a 256-token cap, a running cap of 64 and a warm pass before measurement. The shipped rounds
+schedule reaches 8288.6 tokens/s.
+The shipped starts reach 8318.5 and 8258.7 tokens/s. Their mean is 0.970× an earlier matched-workload no-spec
+control at 8541.1 tokens/s on the same machine. That control was not repeated inside this check, so the
+ratio does not establish a new two-start no-spec qualification. An earlier one-start smoke reached only
+6872.4 tokens/s. None of these short-workload results is pooled with the longer fixed-width results above.
+
+The final-source twenty-prompt check gives 5 exact / 14 near tie / 1 fp32-tolerance pass and no failures.
+It checks first divergences against no-speculation decoding, not bitwise equality or the concurrent
+high-batch width-4 path. Missing registration-time INFO logs do not imply the schedule failed to load:
+runtime counters show the reduced verification width.
Use the [packaged rounds file with matching calibrated costs](../eagle_harness/core/profiles/vllm-qwen38-tp4/README.md).
The schedule narrows the verify width from 16 to 4 at batch size 20 and above, without enabling native-OFF.
No rounds file means no width change.
+## Short-workload C8 checks
+
+These B300 TP4 checks use two independent starts, 16 requests with a 256-token cap and a matching
+warm pass. They report absolute rates for the final runtime source on its pinned release engine.
+They do not replace the longer-workload tables or establish a speedup over no-spec decoding.
+
+| Engine | Tokens/s | Running cap | Policy |
+|---|---:|---:|---|
+| SGLang | 2586.7 | 32 | Live costs, graph padding enabled |
+| vLLM | 2708.8 | 64 | Width 16 at this batch size |
+
+Each measured pass lasts roughly 1.5 seconds with substantial start-to-start spread. No matching
+no-spec mean is reported for these short checks.
+
## TokenSpeed (experimental)
The adapter implements static and calibrated trees, chains and sampled trees. This is implementation coverage,
not a performance or distribution-quality claim for every configuration.
Prompt logprobs, live cost measurement and native no-speculation rounds are not supported.
-No TokenSpeed throughput number is claimed for an unmeasured or non-matching workload.
+No TokenSpeed throughput number is claimed for an unmeasured or non-matching workload. The CUDA install
+checks completed real static and cost-model requests, but a static-mode first divergence remains above the
+fp32 gate. Successful installation or generation is not blanket correctness qualification. TP1/TP8, sampled
+distributions, long contexts and release-stack Nemotron need separate qualification.
+
+## Artifact installation checks
+
+These checks use private CUDA artifacts on B300 TP4, twenty prompts with up to 256 generated tokens,
+and one server start. The rate is total generated tokens divided by summed request latency, not a
+multi-start concurrent throughput benchmark. Publication and final-artifact rebuilding are separate gates.
+
+| Artifact and mode | Correctness result | Tokens/s | Qualification limit |
+|---|---|---:|---|
+| TokenSpeed wheel, serve | 4 exact / 16 fp32-tolerance passes | 786.33 | Host-compatible CUDA 13.2 JIT; CUDA 13.0 JIT fails on Ubuntu 26.04 / glibc 2.43 |
+| TokenSpeed wheel, static | 5 exact / 14 fp32 passes / 1 failure | 706.36 | Not qualified: failing gap 0.178637 exceeds 0.125 |
+| SGLang corrected private image | 4 exact / 16 near ties | 628.19 | Hook, dependency and request checks pass; prebuilt `eagle-harness` wheel replaces the Dockerfile source install |
+| vLLM private image | Pending | — | Image loaded; serving and correctness qualification pending |
+| TokenSpeed image | Not built | — | No approved base image qualification |
+
+A near-tie or fp32-tolerance pass is not an exact-token match. TokenSpeed lacks full-decode native
+top-logprobs, so each divergent prefix in these install checks goes directly to fp32 arbitration.
## Correctness and request features
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
index 58c0170..c62953b 100644
--- a/docs/troubleshooting.md
+++ b/docs/troubleshooting.md
@@ -46,7 +46,7 @@ no supported host installed (one of: sglang, vllm, tokenspeed)
| `Unknown speculative algorithm name: EAGLE_HARNESS` (sglang), vLLM rejects the method `eagle_harness`, or TokenSpeed does not know the algorithm `EAGLE_HARNESS` | the plugin did not register | run `eagle-harness-check` and read the error the engine logged at startup |
| `EAGLE_HARNESS needs --speculative-algorithm-config (JSON or a path to JSON), ...` | sglang got no config | pass `{}` (live costs), `{"data_path": "<calibration dir>"}` or `{"mode": "static"}` |
| `Cost tables under ... were measured on another setup (stored vs this server): {...}` | `data_path` tables keyed for another GPU, TP size, model pair, engine build or dtype | calibrate for this setup, or drop `data_path` to measure the costs on the served rounds |
-| `EAGLE-Harness mode 'serve' needs data_path (the calibration directory).` | a cost-model mode without cost tables (TokenSpeed) | set `data_path` ([Calibration](calibration.md)) or use `"mode": "static"` |
+| `EAGLE-Harness mode 'serve' needs data_path (the calibration directory).` | a cost-model mode without cost tables (TokenSpeed) | set `data_path` to matching TokenSpeed costs ([Calibration](calibration.md)); static mode is not a correctness-qualified substitute |
| `config: unknown keys ['intensty']; valid keys are [...]` | a misspelt key | use a key from the list ([Configuration](configuration.md)) |
| `mode: 'fast' is not one of static, serve, eval, calibrate.` | a bad value | use one of the listed values |
| `EAGLE_HARNESS needs --speculative-num-steps, --speculative-eagle-topk and --speculative-num-draft-tokens ...` | sglang got no tree shape | pass all three |
@@ -56,7 +56,7 @@ no supported host installed (one of: sglang, vllm, tokenspeed)
| `Unexpected FlashInfer page size 128 without trtllm-gen GQA` (vLLM) | the draft runs on FlashInfer, which serves a multi-head draft only on pages up to 64 tokens; a hybrid target needs 128 or more | set `"attention_backend": "FLASH_ATTN"` in `--speculative-config` (the draft's backend only) |
| `num_speculative_tokens=... disagrees with method_config.max_verify_width=...` (vLLM) | both were set and do not differ by one | set only `max_verify_width` |
| `the draft tree has fewer nodes than the verify width needs; raise tree_depth or node_top_k` (vLLM) | the tree cannot fill the verify width | raise `tree_depth` or `node_top_k`, or lower `max_verify_width` |
-| `EAGLE_HARNESS on TokenSpeed supports static and serve modes` (TokenSpeed) | an unsupported evaluation or calibration mode | use `static`, or `serve` with matching cost tables |
+| `EAGLE_HARNESS on TokenSpeed supports static and serve modes` (TokenSpeed) | an unsupported evaluation or calibration mode | use `serve` with matching cost tables for the checked path; static remains unqualified |
| `tree drafting topk=... exceeds verify width ...` (TokenSpeed) | a draft step needs more write locations than the verify-sized buffer holds | raise `--speculative-num-draft-tokens` or lower top-k |
| `FlashInferSamplingBackend cannot verify draft trees; use --sampling-backend greedy or triton` (TokenSpeed) | TokenSpeed's default sampling backend | pass `--sampling-backend greedy` |
| `--speculative-num-draft-tokens N verifies more nodes than a tree of depth ... holds` (TokenSpeed) | the tree cannot fill the verify width | raise `--speculative-num-steps` or `--speculative-eagle-topk`, or lower the width |
@@ -70,9 +70,12 @@ inputs.`). Serve them from a server without the method.
## Slower than expected
- Check that the cost tables match your GPU, TP size, tree shape and engine build ([Calibration](calibration.md)).
- Tables from another setup load without an error and make worse choices.
+ A mismatched `key.json` is refused. Unkeyed legacy tables can load with a warning, and the key does not
+ encode every setting, including the attention backend.
- Look for the startup warning `native CUDA kernels unavailable`. The Triton fallback gives the same results more
slowly.
-- On vLLM, look for the startup line about the depth stop: `no cost tables price a draft step ...` means the stop
- drafts every step that could still change the verify set.
+- On vLLM, `no cost tables price a draft step ...` means the lossless stop drafts every step that could
+ still change the verify set. Registration-time INFO messages can be suppressed before logging is configured.
+ An absent schedule-load message alone does not show that `rounds/tree.json` was ignored. Verify the
+ configured `data_path` and runtime draft-width counters rather than relying only on that message.
- Run `python -m eagle_harness.kernels.arch` to see which kernel tier each GPU gets.