Card × model estimate

Granite 4.0 H Small 32B-A9B on RTX 5090 Q4_K_M

should fit

On an RTX 5090 (32 GB), Granite 4.0 H Small 32B-A9B in Q4_K_M needs about 19,9 Go of video memory: it should fit.

Not measured yet: the figures below are estimates, computed from the models already measured on this card. Measure my card

—

Generation (estimated)

≥ 19,9 Go

Recommended VRAM (32 GB avail.)

—

Power (ref. model)

—

Electricity / M tokens · 0,25 €/kWh

No model has been measured on the RTX 5090 yet: no speed estimate possible.

What about a longer context?

Context	KV cache	Estimated VRAM	Fits in 32 GB?	Estimated speed
4 096	0,1 Go	19,0 Go	yes	—
8 192	0,1 Go	19,0 Go	yes	—
16 384	0,3 Go	19,1 Go	yes	—
32 768	0,5 Go	19,4 Go	yes	—
65 536	1,0 Go	19,9 Go	yes	—
131 072	2,0 Go	20,9 Go	yes	—

Estimated VRAM = file + KV cache (full-attention layers, f16) + ~0,5 Go of buffers, excluding memory used by the system — hence the slightly higher “recommended VRAM”, which keeps that margin. Speed ≈ estimated speed × weights read ÷ (weights read + added KV cache).

Granite 4.0 H Small 32B-A9B on other cards

should fit

Granite 4.0 H Small 32B-A9B page RTX 5090 page Feel the speed Download the GGUF (18,4 Go)

Estimates based on real measurements of other models on this card and on the model's public data (IBM Granite). As soon as a measurement exists, this page shows the real figures.