Configurator

Build your setup, without overflowing

Pick your card, the memory to leave to the system, the model and the context size: we check that everything fits in video memory and estimate the speed.

Speeds at context 0 are measured. Anything depending on context or concurrency is an estimate: every formula is shown, nothing is written to the database.

Graphics card

24 models measuredReserved for the OS

display, browser, system

Model283 tok/s · 3,7 Go measured · 2 runsContext window

tok

max 131 072

KV cache

Video memory of the RTX 5070 Ti

4,0 Go used of 15,0 Go usable (16 GB)

11,0 Go left to add a model or grow a context.

▮ model (measured)▨ KV cache□ Reserved for the OS

Llama 3.2 3B InstructQ4_K_M · 4 096 tok

VRAM: 4,0 Go; 3,7 Go + 0,4 Go KV
Alone: 239 tok/s; Estimated
Concurrent: —; add a 2nd model