Meta
Llama 3.1 405B
405,9B
Parameters
228,8 Go
Q4_K_M file
≥ 230,3 Go
Recommended VRAM
131 072
Max context
Not measured yet
No card has run the TokenBench protocol on this model yet. The current protocol covers a selection of models; benchmarking any catalog model from the app is coming soon.
Measure my card →Which cards should it fit on?
Cards on the site whose video memory covers the recommended 230,3 Go (file + 4,000-token context). An estimate, not a measurement. Click a card for the detailed estimate.
Technical sheet
- Published
- 2024-07-16
- Architecture
- 126 layers · 8 KV heads · dimension 128
- Attention
- Standard: every layer keeps the whole context in memory.
- Context cache (f16)
- ≈ 492 MB per 1,000 tokens · 15,8 Go for 32,768 tokens
- GGUF file
- bullerwins/Meta-Llama-3.1-405B-Instruct-GGUF · 6 files Download (228,8 Go)
- Official repo
- meta-llama/Llama-3.1-405B-Instruct
Public Hugging Face data (official config.json, GGUF repo). The context cache only counts full-attention layers.