Running FP4-Local on 4 × V100 32 GB (Volta) with ReVolta
Hi all! Sharing a setup for running nvidia/Qwen3.8-Flash-Next-FP4-Local on older hardware: 4 × V100 32 GB (Volta, SM70), with ReVolta, a vLLM-based engine built for 4 × V100 servers. V100 has no FP4, FP8 or BF16 units, but ReVolta serves this checkpoint's NVFP4 and MXFP8 weights on it, and of the checkpoints ReVolta supports, this one needs the least host memory and leaves the largest KV pool.
What it takes
- ReVolta 0.1.4:
pip install revolta==0.1.4(Linux x86-64, Python 3.12, CUDA 12.8), ordocker pull ghcr.io/leonccaa/revolta:0.1.4. - For an E4M3 KV cache, a small scale pack: leoncca/Qwen3.8-Flash-Next-FP4-Local-QSA-FP8-E4M3-KV-Scales, tag
v1.1-base-b9ce18a6. It holds the 24 target K/V scales and the two scales of the built-in MTP layer, so both the target and the MTP draft can use E4M3 KV. It is matched to revisionb9ce18a6of this repository. - The steps (download, materialize the scales, a 512K YaRN view, launch command) are in the FP4-Local setup guide. The built-in MTP layer is used as the draft; no separate draft download is needed.
- Recommended interconnect: NVLink, or all four GPUs under one PCIe switch with P2P.
Numbers
Measured with ReVolta 0.1.3 on 4 × V100 32 GB, TP4 + DCP2, 512K context, 8 sequences, E4M3 target and draft KV:
| Download | 102 GB |
| PLE table in host memory | 27 GiB (the AWQ and standard NVFP4 checkpoints need 48 GiB) |
| GPU KV pool | 1,474,106 tokens (2.81× at 512K) |
| MTP acceptance | 63.1% greedy, 62.0% sampled |
| Quality vs its BF16 parent | KL 0.0380, top-1 agreement 95.9% (66 English, Chinese and code records, E4M3 KV) |
With an FP16 draft cache instead, the GPU KV pool is 1,321,612 tokens (2.52×). The checkpoint passed ReVolta's serving acceptance on the reference setup: 512K context, 4 and 8 concurrent requests, images and video. Keep the vision tower on TP4, the default.
Thanks to NVIDIA for publishing this checkpoint. Questions and results from other V100 setups are welcome here or in the ReVolta issues.