Card × model benchmark

Meta Llama 3.1 8B Instruct on RTX 4070 Ti Q4_K_M

Smooth

On an RTX 4070 Ti (12 GB), Meta Llama 3.1 8B Instruct in Q4_K_M generates 89,6 tokens/s and reads the prompt at 5 960 tokens/s, using 5,6 Go of video memory.

Median of 2 runs · latest on 08/08/26

89,6 tok/s

Generation

5 960 tok/s

Prompt processing

5,6 Go

VRAM used (peak)

158 W

Average power

0,57 tok/s/W

Efficiency

0,12 €/M tok

Electricity / M tokens · 0,25 €/kWh

Estimate your monthly cost from your electricity price and usage: sign in

RTX 4070 Ti page All cards on Meta Llama 3.1 8B Instruct Feel the speed Rent in the cloud

Where to buyLDLC Materiel.net TopAchat Cybertek Grosbill Rue du Commerce Amazon

What about a longer context?

An estimate, not a measurement: the KV cache grows with context, takes VRAM and slows generation down.

Context	KV cache	Estimated VRAM	Fits in 12 GB?	Estimated speed
4 096	0,5 Go	6,1 Go	yes	≈ 82,5 tok/s
8 192	1,0 Go	6,6 Go	yes	≈ 75,5 tok/s
16 384	2,0 Go	7,6 Go	yes	≈ 64,4 tok/s
32 768	4,0 Go	9,6 Go	yes	≈ 49,9 tok/s
65 536	8,0 Go	13,6 Go	no	—
131 072	16,0 Go	21,6 Go	no	—

KV cache per token = 2 × 32 layers × 8 KV heads × 128 × 2 bytes (f16). Estimated VRAM = measured VRAM + added KV cache. Speed ≈ measured speed × weights ÷ (weights + added KV cache) — attention compute is not modelled, the real drop is slightly larger. Fine-tune it in the “Configurator”

Meta Llama 3.1 8B Instruct on other cards

Card	VRAM	Generation	Difference	Price
RTX 5070 Ti	16 Go	151 tok/s	+68,6 %	1 400 €
RTX 4090demo	24 Go	132 tok/s	+47,3 %	—
RTX 5080demo	16 Go	124 tok/s	+38,4 %	1 850 €
RTX 4080demo	16 Go	103 tok/s	+15,0 %	780 €
RTX 4070 Ti	12 Go	89,6 tok/s	this card	927 €
RTX 4070	12 Go	82,3 tok/s	−8,2 %	595 €
RTX 3060demo	12 Go	48,0 tok/s	−46,4 %	260 €

Other models measured on the RTX 4070 Ti

Llama 3.2 3B Instruct Q4_K_M 186 tok/s Qwen2.5 7B Instruct Q4_K_M 94,9 tok/s DeepSeek R1 Distill Qwen 14B Q4_K_M 49,3 tok/s Qwen2.5 14B Instruct Q4_K_M 49,2 tok/s Phi-4 Q4_K_M 48,8 tok/s

All measurements

Date	Source	Type	Generation	Prompt	VRAM	Max temp.	Power	llama.cpp
08/08/26	measured	standard	88,9	5 925	5,6 Go	47 °C	157 W	llama.cpp 96278e39f
04/08/26	measured	standard	90,3	5 996	5,6 Go	50 °C	159 W	llama.cpp 96278e39f

Protocol: llama-bench (llama.cpp), prompt 512 + generation 128 tokens, model fully loaded in VRAM. VRAM used = system peak (includes the OS, ~0.5 to 1 GB more than the model alone). Measure my card