IBM
Granite 4.0 H
MoE
Pick a size: the video memory shown is the Q4_K_M file (the protocol's quantisation) plus about 1.5 GB for a 4,000-token context.
Micro 3B
8 GB card- Parameters
- 3,2B
- Q4_K_M file
- 1,8 Go
- Recommended VRAM
- ≥ 3,3 Go
- Max context
- 131 072
Not measured yet
Tiny 7B-A1B
8 GB card- Parameters
- 6,9B · 1B active
- Q4_K_M file
- 4,0 Go
- Recommended VRAM
- ≥ 5,5 Go
- Max context
- 131 072
Not measured yet
Small 32B-A9B
24 GB card- Parameters
- 32,2B · 9B active
- Q4_K_M file
- 18,4 Go
- Recommended VRAM
- ≥ 19,9 Go
- Max context
- 131 072
Not measured yet
MoE (“30B-A3B”): all the memory of a 30B, but the speed of a model with 3B active parameters. “> 32 GB”: several cards, or part of the model in system memory (much slower).