Mistral AI

Mistral 7B

Pick a size: the video memory shown is the Q4_K_M file (the protocol's quantisation) plus about 1.5 GB for a 4,000-token context.

7B

8 GB card
Parameters
7,2B
Q4_K_M file
4,1 Go
Recommended VRAM
≥ 5,6 Go
Max context
32 768

Measured on 1 card · 98,1 tok/s (RTX 4070 Ti)

MoE (“30B-A3B”): all the memory of a 30B, but the speed of a model with 3B active parameters. “> 32 GB”: several cards, or part of the model in system memory (much slower).