Open-source AI browser agent for Chrome and Firefox (monorepo) ðŸ§
Reproducibility runbook
This runbook reconstructs the data, training, verification, packaging, and inference state for the DeepSeek V4 Flash Vision and Laguna XS 2.1 Vision experiments. Commands are examples with explicit placeholders; adapt storage paths but do not change pinned revisions, seeds, token budgets, or hashes when reproducing the reported run.
1. Scope and evidence classes
Treat evidence in this order:
- A downloaded file whose SHA-256 matches
sources/ARTIFACTS.json. - An immutable Hugging Face commit URL.
- The private Laguna run archive at its pinned commit.
- The copied local source snapshot and historical notes.
- A mutable Hugging Face model page or current local working tree.
The local DeepSeek source snapshot is base commit
285a35a1c5cbff3251418013a0671b25ef78488a plus working-tree changes. Its published
SGLang runtime subset is synchronized to the reviewed public package revisions below,
but the larger snapshot is not represented as one clean source release. See
sources/SOURCE_SNAPSHOT.json.
2. Immutable revision ledger
| Resource | Revision |
|---|---|
deepseek-ai/DeepSeek-V4-Flash routing reference |
60d8d70770c6776ff598c94bb586a859a38244f1 |
sgl-project/DeepSeek-V4-Flash-FP8 training reconstruction source |
ae01d80c06cdfe30581edfd0e1c5449dc7ed7f17 |
nvidia/DeepSeek-V4-Flash-NVFP4 package source |
e3cd60e7de98e9867116860d522499a728de1cf9 |
| SGLang source used by DeepSeek package | fdebc938f7f4d16fe6b9f55dcd9a767cf0899ea1 |
poolside/Laguna-XS-2.1 BF16 source |
e9df9a59996d790b94b70f3fef343fe1d9e34bdf |
poolside/Laguna-XS-2.1-NVFP4 source |
d32afde8b09af1539b49ff96ff5551c674485f8e |
moonshotai/Kimi-K2.6 MoonViT source |
7eb5002f6aadc958aed6a9177b7ed26bb94011bb |
HuggingFaceM4/the_cauldron data source |
847a98a779b1652d65111daf20c972dfcd333605 |
| DeepSeek public NVFP4 final | 12b653e63329ac3c20395f9aeeb1bb8264d2db8b |
| DeepSeek public BF16 overlay final | 60c441aa1c7386387c89ddaff703136395ad8d8b |
| Laguna public NVFP4 final | ce108f0f3764a18a1f5f7d14ecefa90485ea6e52 |
| Laguna public BF16 final | b06063d00b73f7713ef1c28f8247eed488c73faf |
| Laguna NVFP4 SGLang integration candidate | sglang-integration at 740a6ccc05441a7ea0426c1d2536d5cd031adb86 |
| Laguna private 100K archive | 4f5d6359867fa55315a2bec4b568901bfdeca5e5 |
3. Security and storage prerequisites
- Keep Hugging Face and RunPod tokens in the host's credential store. Do not write them to source files, shell history, plist files, pod volumes, logs, or process arguments.
- Use a persistent volume. DeepSeek conversion needs roughly 274 GiB of FP8 source plus 542 GiB of BF16 output; 1 TiB is a practical minimum.
- Keep the MoonViT environment separate from the DeepSeek environment. The recorded stack used Transformers 4.57.6, PyTorch 2.8, and FlashAttention 2.8.3.post1 for Kimi's remote MoonViT code, and PyTorch 2.11, Triton 3.6, CUDA 12.8, and a pinned Transformers 5 development build for DeepSeek.
- Use a fixed absolute budget deadline and a verified-work liveness guard. Never convert it into a blind rolling timer that silently extends the budget.
- Preserve stopped-pod storage until all final artifacts have been copied and hashed.
4. Source setup
The curated source is in sources/deepseek-v4-flash. To run it
outside the documentation tree, copy it into a clean work directory and create both pinned
environments using the included scripts:
rsync -a sources/deepseek-v4-flash/ /workspace/deepseek-vision/
cd /workspace/deepseek-vision
./scripts/preflight_cloud.sh
./scripts/bootstrap_cloud.sh
./scripts/bootstrap_moonvit_env.sh
PYTHONPATH=src python3 -m deepseek_vision.cli validate-config configs/model/moonvit.json
python3 -m unittest discover -s tests -v
For a strict rerun, first convert this working-tree snapshot into a reviewed source commit and record that commit. The copied snapshot itself is sufficient for audit, but not an ideal long-term release boundary.
5. Build the deterministic 100K data manifest
The shared fetcher writes per-configuration checkpoints, so a retry reuses completed source configs instead of downloading them again.
cd /workspace/deepseek-vision
.venv-moonvit/bin/python scripts/fetch_cauldron.py \
artifacts/data/pilot100k \
--preset pilot100k \
--revision 847a98a779b1652d65111daf20c972dfcd333605 \
--seed 20260802 \
--shuffle-buffer 512
shasum -a 256 artifacts/data/pilot100k/manifest.jsonl
For the archived Laguna run, expect:
1d1961eee90cb3bad1ee3fb75f79e38d636ef406bb2a2539581c442a3c0ae4bb manifest.jsonl
Also verify provenance.json reports exactly 100,000 examples, the eleven quotas in the
research report, the train split, and evaluation_splits_included: false. A hash mismatch
means the input bytes, data library behavior, seed, shuffle buffer, or fetch code changed;
do not label the rerun equivalent.
6. Extract and cache MoonViT
Extract only the vision component:
.venv/bin/deepseek-vision extract-component moonshotai/Kimi-K2.6 \
'vision_tower.' artifacts/towers/moonvit-k2.6.safetensors \
--revision 7eb5002f6aadc958aed6a9177b7ed26bb94011bb
Run one cache worker per GPU. Each process receives a disjoint rank, uses its local GPU as
cuda:0, and atomically reuses existing features/examples:
WORLD_SIZE=4
for RANK in 0 1 2 3; do
CUDA_VISIBLE_DEVICES="$RANK" .venv-moonvit/bin/python scripts/cache_moonvit_features.py \
artifacts/data/pilot100k/manifest.jsonl \
artifacts/cache/deepseek-moonvit \
--model-config configs/model/moonvit.json \
--max-sequence-length 2048 \
--max-image-tokens 512 \
--rank "$RANK" --world-size "$WORLD_SIZE" --device cuda:0 &
done
wait
Laguna uses its archived
cache_moonvit_features.py
because its chat template and token IDs differ:
WORLD_SIZE=4
for RANK in 0 1 2 3; do
CUDA_VISIBLE_DEVICES="$RANK" python \
sources/laguna-xs-2.1/private-training-archive/code/cache_moonvit_features.py \
artifacts/data/pilot100k/manifest.jsonl \
artifacts/cache/laguna-moonvit \
artifacts/towers/moonvit-k2.6.safetensors \
--max-sequence-length 2048 \
--max-image-tokens 512 \
--rank "$RANK" --world-size "$WORLD_SIZE" --device cuda:0 &
done
wait
Before training, load every cached file with torch.load(..., weights_only=True) and verify:
- all vision features are finite BF16 and shaped
[tokens, 4, 1152]; - the image mask contains exactly one entry per merged feature group;
- input IDs, image mask, attention mask, and labels have equal sequence length;
- supervised labels form the intended answer suffix;
- no sequence exceeds 2048 and no image exceeds 512 merged tokens.
7. DeepSeek training path
7.1 Reconstruct a gradient-compatible backbone
.venv/bin/hf download sgl-project/DeepSeek-V4-Flash-FP8 \
--revision ae01d80c06cdfe30581edfd0e1c5449dc7ed7f17
.venv/bin/deepseek-vision dequantize-checkpoint \
/path/returned/by/hf-download \
artifacts/models/deepseek-v4-flash-bf16 \
--resume
Required post-conversion invariants:
46 shards
34,123 scaled weights converted
581,975,531,080 output bytes
35,020 indexed tensors
34,600 BF16 weight tensors
0 FP8 tensors
0 scale tensors
0 partial files
Do not delete the FP8 source until a real forward/backward gate has loaded the BF16 output.
7.2 Validate routing and backward
The model config must retain:
DeepSeek routing reference: 60d8d70770c6776ff598c94bb586a859a38244f1
routing policy: palette_cycle
palette size: 64
projector output width: 4096
projector parameter count: 40,119,040
Pass gates in order: source/component validation, synthetic backward, 256-example overfit, 1,000-example timed calibration, then 100K. A gate passes only if loss is finite, all six projector tensors receive finite gradients, no language/tower parameter receives a gradient, frozen hashes remain unchanged, text routing IDs remain exact, and image slots match features.
7.3 Run the 100K projector
.venv/bin/deepseek-vision train-cache \
configs/model/moonvit.json \
configs/train/pilot100k.json \
artifacts/cache/deepseek-moonvit/examples \
artifacts/runs/moonvit-pilot100k \
--max-examples 100000
The checked-in config specifies one epoch, batch/accumulation 128, AdamW, LR 1e-3, no
weight decay, 3% warm-up, cosine decay, BF16, clip norm 1.0, and checkpoints every 100 steps.
DeepSeek's local --resume-checkpoint restores projector weights and schedule position but
does not restore AdamW optimizer state. Its log explicitly reports
optimizer_state_restored: false. A strict reproduction should either avoid interruption or
extend the runner to atomically archive optimizer/scheduler/RNG state before claiming exact
resume equivalence.
8. Laguna training path
Use the archived trainer, which pins Laguna BF16 revision
e9df9a59996d790b94b70f3fef343fe1d9e34bdf and validates the 30,679,808-parameter
projector:
torchrun --standalone --nproc_per_node=4 \
sources/laguna-xs-2.1/private-training-archive/code/train_projector_ddp.py \
artifacts/cache/laguna-moonvit \
artifacts/runs/laguna-projector-100k \
--max-examples 100000 \
--epochs 1 \
--global-gradient-accumulation 128 \
--learning-rate 0.001 \
--weight-decay 0 \
--warmup-ratio 0.03 \
--gradient-clip 1.0 \
--save-every 10 \
--seed 20260802 \
--resume
The first launch prints a harmless resume found: false event. Later launches restore the
projector, full optimizer state, scheduler state, example count, and step from
training-state.json. The world size must remain four. Checkpoint writes are atomic and the
state file points only to complete projector/optimizer files.
The historical run reached step 384, restarted from durable step 380, replayed steps 381-384, and finished at step 782. Therefore:
- use the line for step 1 as the true first loss (
4.908374495338649); - use the final line as the final loss (
0.7318046561413212); - the logged minimum is step 476 (
0.6835096142294574); - interpret the final
elapsed_secondsand throughput as resumed-segment metrics; - do not divide cumulative 100K examples by resumed-segment time.
The private archive should contain 170 files totaling 14,543,120,986 bytes, including
training.log, run-summary.json, training-state.json, SHA256SUMS, checkpoint pairs,
and code snapshots. Verify its own SHA256SUMS after download.
9. Component verification
| File | Size | SHA-256 |
|---|---|---|
DeepSeek vision_tower.safetensors |
833,765,768 | 1382c41f1a4afc91791ade630e2b1e1cef68cc5a1e09668a45970a5d5e1b8f15 |
DeepSeek mm_projector.safetensors |
80,238,568 | 7024d9d5c9714c7abbc09abda015f083b7d7b107745eb78879f019bf4721577a |
Laguna vision_tower.safetensors |
833,765,768 | befe801bd7dfe8bf5630fef56a7f53c2235065599ca9eea4d995040e2e6fd183 |
Laguna mm_projector.safetensors |
61,360,104 | 7837384f18be69a4f875ca44a8ed69ec186501d70896daf4b305fa77547974de |
| Laguna final optimizer step 782 | 122,726,045 | c5879b03b6f86b5ddabc3eb05380915b1b1aa4c0058592ae6a73f6220624c9f7 |
Example verification:
shasum -a 256 /external/model/mm_projector.safetensors
Tensor-level checks should assert six BF16 projector tensors, the exact parameter count, finite values, and expected input/output dimensions. Do not treat equal byte size as equal content; the two tower files have different hashes.
10. Publication and package identity
Published packages must include their source model ID/revision, component SHA-256 and size, licenses, model card, and an explicit serving-status declaration. Copying text weights and adding a projector does not by itself create a runnable vision-language endpoint.
DeepSeek package rules:
- NVFP4 is a complete pinned quantized text package with custom SGLang source.
- BF16 is a source overlay and must say
complete_text_checkpoint: false. - Stock SGLang is unsupported; the integration requires the pinned source patch.
- The packaged native target is SGLang
/generate, not OpenAI multimodal chat. - The pinned B200 profile passed loader/startup and two live image requests; text-only parity, broad evaluation, and production validation are still required.
Laguna package rules:
- BF16 and NVFP4 repositories each contain their complete pinned text backbone plus tower and projector.
- Both public main revisions must say that the multimodal processor/serving integration is absent from main.
- BF16 image inference, NVFP4 equivalence, and end-to-end image serving remain unvalidated.
- A processor/server candidate exists only on the NVFP4
sglang-integrationbranch at740a6ccc05441a7ea0426c1d2536d5cd031adb86; it has no recorded full-model or image smoke.
The public model cards and adapter manifests copied into
sources/published-artifacts are the templates for these
statements.
11. DeepSeek image smoke test
The local prototype can test the BF16 reference path with a single image:
.venv/bin/deepseek-vision infer-image \
configs/model/moonvit.json \
artifacts/runs/moonvit-pilot100k/projector-step-000782.safetensors \
/path/to/image.jpg \
--prompt 'Describe this image in one sentence.' \
--max-tokens 64
The local non-streaming test server binds to loopback by default:
.venv/bin/deepseek-vision serve \
configs/model/moonvit.json \
artifacts/runs/moonvit-pilot100k/projector-step-000782.safetensors \
--host 127.0.0.1 --port 8000
Forward it without exposing the unauthenticated prototype:
ssh -N -L 8010:127.0.0.1:8000 \
-p REMOTE_SSH_PORT -i ~/.ssh/id_ed25519-runbod root@REMOTE_SSH_HOST
curl http://127.0.0.1:8010/health
python3 scripts/test_vision_api.py /path/to/image.jpg \
--url http://127.0.0.1:8010/v1/chat/completions \
--prompt 'Bu goruntude ne var?'
The left side of -L is the local port. If local port 8000 is occupied, changing only the
remote side does not help; use -L 8010:127.0.0.1:8000 as above.
The prototype accepts one base64 data:image/... URL, rejects remote image URLs, limits
decoded images to 20 MiB, serializes generation, and buffers responses even when stream
is requested. These local API semantics are separate from the public custom SGLang package.
Pinned B200 SGLang package smoke
On 2026-08-04, the public DeepSeek integration was exercised on NVIDIA B200 with four-way
tensor parallelism using SGLang source
fdebc938f7f4d16fe6b9f55dcd9a767cf0899ea1. The server remained healthy across two
consecutive requests to its native endpoint, http://127.0.0.1:30000/generate:
| Probe | Image tokens | Total prompt tokens | Observed answer content | Elapsed |
|---|---|---|---|---|
| Street/taxi scene | 294 | 308 | A man, yellow vest, and taxi were identified | 6.95 s |
| Cat/person scene | 532 | — | A cat sitting on a person's shoulder was identified | 2.52 s |
This is bounded evidence of end-to-end image serving, not a broad quality benchmark or a
production-readiness claim. The extension accepts one image per request, requires CUDA
image preprocessing, targets tensor parallelism, and does not support stock SGLang or the
OpenAI /v1/chat/completions image_url path. Pipeline parallelism and encoder data
parallelism remain blocked or unvalidated, and text-only parity remains a separate gate.
Laguna has no validated equivalent endpoint in the recorded artifact. The NVFP4
sglang-integration candidate is packaging evidence only until its tokenizer/image
placeholder contract, processor, embedding scatter, loader, and generation path pass a
real smoke test. Do not point a generic text server at the projector and call that an image
test.
12. Completion checklist
- Published source, model, tower, and serving revisions are immutable.
- Dataset manifest and component SHA-256 values match this record.
- All feature-cache invariants pass.
- Synthetic backward changes only projector gradients.
- Text-only DeepSeek parity and routing-ID preservation pass.
- Overfit loss falls and timed calibration remains within the approved budget.
- Final checkpoint, optimizer state where applicable, state file, log, and hashes are copied before the pod is stopped.
- Public cards distinguish full checkpoints, overlays, and serving limitations.
- BF16 reference image inference passes.
- A bounded quantized DeepSeek B200 end-to-end image smoke passes.
- Text-only parity, held-out evaluation, and broader regression gates pass before declaring either NVFP4 package production ready.
- The Laguna NVFP4 integration candidate passes a full-model loader and image smoke before any serving change is promoted to a public main revision.