🚀 Llama Large Language Model

Llama, released by Meta AI starting in February 2023, is a family of large language models (LLMs). The latest version, Llama 3.3, was released in December 2024.

Models are trained at different parameter sizes ranging from 1B to 405B. Starting with Llama 2, instruction fine-tuned versions were released alongside foundation models.

🔐 Access & Distribution

The initial model weights were available to researchers under a non-commercial license. Unauthorized copies were shared via BitTorrent, leading to subsequent versions being made accessible outside academia with licenses permitting some commercial use.

📱 Applications & Integration

With the release of Llama 3, Meta integrated virtual assistant features into Facebook and WhatsApp in select regions, using the Llama 3 model for these services.

📈 Scaling Laws & Performance

Llama 3 models exhibit log-linear performance scaling when trained on data exceeding "Chinchilla-optimal" amounts. For example, an 8B model showed continued performance improvements up to a 15 trillion token dataset.

🔍 Background & Impact

Llama's development followed the success of GPT-3 and ChatGPT, focusing on up-scaling models for emergent capabilities. Meta's Chief AI Scientist Yann LeCun emphasized its utility in aiding writing tasks.