Card × model estimate

Granite 4.0 H Tiny 7B-A1B on RTX 5070 Ti Q4_K_M

should fit

On an RTX 5070 Ti (16 GB), Granite 4.0 H Tiny 7B-A1B in Q4_K_M needs about 5,5 Go of video memory: it should fit. Estimated generation speed: ≈ 191–287 tokens/s.

Not measured yet: the figures below are estimates, computed from the models already measured on this card. Measure my card

≈ 191–287 tok/s

Generation (estimated)

≥ 5,5 Go

Recommended VRAM (16 GB avail.)

≈ 60 W

Power (ref. model)

0,02 €/M tok

Electricity / M tokens · 0,25 €/kWh

Calculation: Qwen3.5 0.8B measured at 514 tok/s on this card, 0,5 Go read per token → × 0,5 Go ÷ 0,6 Go ≈ 478 tok/s (generation is memory-bound: speed follows the amount of weights read). MoE: only ~14% of the weights are read per token, but this is a theoretical upper bound; in practice llama.cpp reaches about 40 to 60% of it (expert routing, dense attention layers) → indicative range 191 to 287 tok/s.

What about a longer context?

Context	KV cache	Estimated VRAM	Fits in 16 GB?	Estimated speed
4 096	0,0 Go	4,5 Go	yes	≈ 229 tok/s
8 192	0,1 Go	4,6 Go	yes	≈ 217 tok/s
16 384	0,1 Go	4,6 Go	yes	≈ 198 tok/s
32 768	0,3 Go	4,8 Go	yes	≈ 168 tok/s
65 536	0,5 Go	5,0 Go	yes	≈ 129 tok/s
131 072	1,0 Go	5,5 Go	yes	≈ 88,0 tok/s

Estimated VRAM = file + KV cache (full-attention layers, f16) + ~0,5 Go of buffers, excluding memory used by the system — hence the slightly higher “recommended VRAM”, which keeps that margin. Speed ≈ estimated speed × weights read ÷ (weights read + added KV cache).

Granite 4.0 H Tiny 7B-A1B on other cards

should fit

Granite 4.0 H Tiny 7B-A1B page RTX 5070 Ti page Feel the speed Download the GGUF (4,0 Go)

Estimates based on real measurements of other models on this card and on the model's public data (IBM Granite). As soon as a measurement exists, this page shows the real figures.