weights.zip KV Cache

1.6× more cached context in the memory you have.

Hold 1.6× more cached context in the same memory and SSD. More returning requests find their context already cached, so the GPU skips reprocessing the prompt and answers sooner.

For long-context, agent and chat platforms that keep conversation state between requests Pilots open · vLLM

Three exploded tiers, GPU memory, CPU memory and SSD, packed with green cache blocks that move up and down between them.

1.59×more context per GB, Llama-3.1-70B on H200

1.5×faster restore of cached context from NVMe

0slowdown restoring from CPU memory

100%identical model output: cache restored bit for bit

On a $1 billion inference budget, 1.59x more cached context per GB saves an estimated $55.7M, $37.1M or $18.6M a year if 15%, 10% or 5% of GPU time goes to recomputing evicted prompts.

Modes

Mode	Capacity gain	Output	Notes
Standard	1.58 to 1.60×	identical, bit for bit	Llama-3.1 8B and 70B on H200, 8k to 128k tokens
Extended exact	3.1 to 3.3×	identical, bit for bit	long-term and archival storage of cached context
High capacity	5 to 9×	as accurate as an fp8 cast, or better	Llama-3.1 8B and 70B on H200; opt-in

What you get

More cache hits from the memory and SSD you already pay for.
Fewer prompts reprocessed from scratch, which frees GPU time and cuts time to first token.
Park thousands more idle agent sessions without buying more storage.
Plugs into vLLM as a cache tier. Responses do not change.

Good fit when

Users return to long conversations, documents or codebases.
You already offload KV cache to CPU memory or SSD.
Prompt reprocessing is a large share of your GPU bill.

Benchmarks

Measured on NVIDIA H200, up to 128k tokens of context.

Capacity · cached context per GB, exact bytes back

Model	Context	Uncompressed cache	Capacity gain	Tensors identical
Llama-3.1-70B	128k	43 GB	1.59×	160 / 160
Llama-3.1-70B	32k	10.7 GB	1.59×	160 / 160
Llama-3.1-70B	8k	2.7 GB	1.59×	160 / 160
Llama-3.1-8B	128k	17.2 GB	1.58×	64 / 64
Llama-3.1-8B	32k	4.3 GB	1.58×	64 / 64
Llama-3.1-8B	8k	1.07 GB	1.58×	64 / 64

Against zstd on the same cache · Qwen2.5-0.5B, 8k tokens

Codec	Capacity gain	Output
zstd, level 19	1.31×	identical
weights.zip	1.60×	identical

Restoring 128k tokens of context

Model	Cache	From GPU memory	From CPU memory	From NVMe	Recompute the prompt
Llama-3.1-8B	17.2 GB	0.065 s	0.31 s	1.76 s	18.8 s
Llama-3.1-70B	43 GB	0.14 s	0.74 s	4.56 s	248.6 s

Recompute is a full prefill with Hugging Face Transformers on the same H200. From CPU memory, a restore takes the same time as copying the uncompressed cache over PCIe while moving 37% fewer bytes. From NVMe it is 1.5× faster than reading the uncompressed cache. Every restore was checked tensor by tensor against the original.

Inside a serving engine

In vLLM, with chunks aligned to its 16-token block, requests restored from weights.zip generated exactly the same tokens as requests served from the engine's own GPU cache: 8 of 8. Recomputing the prompt instead changed the output on 4 of the 8.

Integration

A cache tier for vLLM. No fork, no patches.

weights.zip KV Cache plugs into vLLM through its standard KV connector interface, the same one used by LMCache. Completed chunks of context are stored as they are produced, and a returning request restores its longest stored prefix before the forward pass, so vLLM computes only the new tokens.

Supported

vLLM v1 engine, through the standard KV connector API.
Prefix reuse across turns, sessions and process restarts.
Chunk size from vLLM's 16-token block up to 256 tokens and beyond.
Storage on local NVMe or a shared file system.
Llama and Qwen model families, including grouped-query attention.

Enable it

from vllm import LLM
from vllm.config import KVTransferConfig

llm = LLM(
    model="meta-llama/Llama-3.1-8B-Instruct",
    kv_transfer_config=KVTransferConfig(
        kv_connector="KVZipConnector",
        kv_role="kv_both",
        kv_connector_extra_config={
            "store_path": "/mnt/kvcache",
            "chunk_tokens": 256,
        },
    ),
)

Verify it yourself

Every restore is checked before the model sees it.

Checked on every restoreEach stored chunk carries a checksum of the exact bytes it held. A restored tensor must match it before it enters the cache; any mismatch stops the restore with an error, never partial data.
Compare the outputRun the same prompts with and without the connector and compare the generated tokens. With chunks aligned to vLLM's block size they match token for token.
Benchmark your trafficSend us a KV cache dump or run our benchmark in your own account. You get capacity gain, restore time on your GPUs and a byte for byte match report.

Start with your own data

Free benchmark on your files and hardware.

Send a sampleA few models, a checkpoint series or a KV cache dump, or run our benchmark inside your account.
Get your numbersBytes saved, restore speed on your GPUs, and a byte-for-byte match report.
PilotRun on one workload for 30 days. Pricing is per terabyte managed or a share of the measured saving.

Request a pilot

The product line

Ten products for the data inside AI models.

How these numbers were measured

All figures are measured on public models (Qwen2.5 0.5B and 1.5B, Llama-3.1 8B and 70B, Pythia-160M, GPT-2 and a 27B FP8 model) on NVIDIA H100 SXM, H200 and RTX 4060 GPUs. Every restore was checked byte for byte against the original. zstd comparisons use zstd level 19 or NVIDIA nvCOMP Zstd on the same files. Dollar figures assume S3 Standard-class storage at about $24 per TB-month and typical cloud egress pricing. Results depend on the model and data type, so the free benchmark measures yours.