Models · DeepSeek
DeepSeek MLA latent cache

A third smaller than FP8. Up to 2.5× more accurate.

Serving stacks keep DeepSeek's latent cache at about 8 bits because int4 costs too much quality. weights.zip matches FP8 quality in 1.4 to 1.5× fewer bits at 100k context, and for sessions paged out to CPU memory or SSD it stores the cache in two thirds of FP8's bytes with 2.5× lower error.

For labs and inference providers serving DeepSeek and other MLA models Pilots open
Annual GPU cost for 10,000 users at 100k context: FP8 mix 17.5 million dollars, weights.zip 11.8 million
Annual GPU cost for 10,000 users at 100k context · reserved H100s
2.5×lower error than FP8 in 0.67× the bytes, for offloaded cache
1.48×fewer bits than FP8 at equal quality (100k tokens), on the GPU. Reproduced on a second GPU type (H100, 2026-10-13)
3 to 5×lower error than int4 at the same size
100ktokens of context, every result independently re-checked
What it saves

int4 memory. Better than FP8 quality.

The cache is the memory that grows with every token of context. weights.zip stores it close to int4 size while beating today's FP8 format on accuracy, so every GPU holds more long-context users and every parked session costs less to keep.

The FP8 mix is what serving stacks use today: FP8 per token for the latent part of the cache and int8 for its position part, 8.06 bits per value. Deployments that keep the position part in bf16 use 8.92 bits per value, which makes the saving larger.

Capacity · a provider on 1,000 H100s serving a DeepSeek MLA model at 100k context, 10,000 concurrent users with the FP8 mix
GPU money · the same 10,000 users
Annual GPU cost for 10,000 concurrent 100k-context users
Cache formatH100 GPUsPer year, reservedSaving, reservedSaving at on-demand rates
FP8 mix1,000~$17.5Mbaselinebaseline
weights.zip676 to 704~$11.8M to $12.3M~$5.2M to $5.7M~$9.4M to $10.2M

GPU figures use the GPU format at FP8 quality: 1.42× fewer bits on code and 1.48× on prose; charts use 1.48×. Where cache memory sets how many users each GPU serves, users scale with cache capacity, assuming decode runs at FP8 speed. Reserved H100 capacity at about $2.00 per hour ($17,500 per GPU-year); on-demand at about $3.61 per hour.

What you get

  • About a third less cache memory than today's FP8 format, at FP8 quality or better.
  • More concurrent long-context users on every GPU.
  • Parked sessions in CPU memory or on SSD that are smaller and more accurate than FP8.
  • No retraining and no changes to the model weights.

Good fit when

  • You serve DeepSeek or other models with a latent attention cache.
  • Cache memory limits how many users each GPU can serve.
  • You page long chats and agent sessions out of GPU memory.
  • int4 would save the memory but costs too much quality.
Benchmarks

Verified live at 100k context against today's FP8 format.

Offloaded cache · 0.67× FP8's bytes
Test windowError vs FP8Tokens where ours beats FP8
Prose, window 10.32×81%
Prose, window 2 (never used for tuning)0.34×85%
Python code0.61×57%
All three0.41× (95% CI 0.36 to 0.47)57 to 85%

5.43 bits per value against FP8's 8.06. Against int4, the format most often deployed at 4 bits, the error is 11 to 16× lower for 10% more bytes. Each window scores 1,024 tokens at 100k context, and the window marked was never used to choose any setting.

GPU cache formats · DeepSeek-V2-Lite, 100k context
FormatBits per valueQuality
FP8 mix (today's practice)8.06baseline
int4 (group 64) with int8 position part4.92about 4× FP8's error
weights.zip4.973 to 5× lower error than int4 at the same size
weights.zip5.44level with FP8 on code, half FP8's error on prose
weights.zip6.312 to 6× lower error than FP8

Reproduced on a second GPU type (H100, 2026-10-13): the H100 rows match the L40 run configuration by configuration. At FP8 quality, weights.zip needs 1.42× fewer bits on code and at least 1.48× fewer on prose. Quality is KL divergence of the model's next-token predictions against the uncompressed model, on the live model in fp32. Every number was re-checked by an independent verifier, with sizes counted from the real stored bytes and the test text kept separate from any tuning data.

Deployment

No retraining. No changes to the model.

  1. Prepared once per modelA one-time offline step per model. The model weights are untouched and nothing extra is stored per token.
  2. Swap the cache formatThe cache is written and read in the weights.zip format on the GPU, and in the higher-accuracy format when sessions are paged out.
  3. Pilot on your trafficWe integrate with your serving stack and measure memory, quality and capacity on your models and prompts.
Start with your own models

Free benchmark on your models and traffic.

  1. Tell us your stackThe models you serve, your context lengths and your serving engine.
  2. Get your numbersMemory saved, quality against your current format and capacity per GPU, measured on your prompts.
  3. PilotRun on one workload for 30 days. Pricing is a share of the measured saving.
Request a pilot
Models

Built for the models you serve.

Model names and logos are trademarks of their owners and are used only to identify the models tested. weights.zip is not affiliated with or endorsed by DeepSeek, NVIDIA, Z.ai, Meta or MiniMax.