Llama 3.1 405B

Model Overview

Llama 3.1 405B is Meta’s flagship open-weights model, released on July 23, 2024. With 405 billion parameters and a 128K context window, it is the largest openly available foundation model to date. Meta reports that Llama 3.1 405B closes the gap with closed-source flagships like GPT-4o and Claude 3.5 Sonnet on MMLU, HumanEval, and MATH, enabling organizations that require full model control — for regulatory, privacy, or customization reasons — to deploy a near-frontier model on their own infrastructure. The weights are released under the Llama 3 Community License, which permits commercial use for organizations with fewer than 700 million monthly active users. The model is available via major cloud providers (AWS, Azure, Google Cloud) and self-hosted on clusters of 8× H100 GPUs or equivalent.

Key Specifications

Benchmark Performance

Pricing

Pricing source: https://www.together.ai/pricing (as of 2024-08-01). Self-hosted pricing depends on GPU infrastructure cost.

Strengths

• Largest open-weights model available, with near-frontier performance.
• Full model control for regulatory-sensitive industries (defense, healthcare, finance).
• Llama 3 Community License permits commercial use for most organizations.
• Mature ecosystem with vLLM, TGI, and TensorRT-LLM serving stacks.

Weaknesses

• Requires significant GPU infrastructure (8× H100 80GB recommended) to self-host.
• No native multimodal support (text-only).
• Self-hosting introduces operational complexity for inference, scaling, and monitoring.

Use Cases

• On-premise deployments for regulated industries.
• Custom fine-tuning and domain-specific adaptation.
• Sovereign AI initiatives requiring full data control.
• Research workloads needing weight-level access.

References

• Meta Llama 3.1 Announcement
• Llama Documentation
• Together AI Pricing