Foundation Model
A large AI model trained on broad data at massive scale that can be adapted to a wide variety of downstream tasks, forming the 'foundation' for specialized applications.
The term 'foundation model' was coined by Stanford researchers in 2021 to describe large AI models trained on internet-scale data that serve as a general starting point for diverse applications. GPT-4, Claude, Gemini, and Llama are all foundation models, each capable of writing, coding, reasoning, summarizing, and translating without task-specific training.
What makes foundation models powerful is their emergent capabilities: abilities that appear at scale that weren't explicitly trained for. As models grow larger, they spontaneously acquire new skills like arithmetic, analogical reasoning, and multilingual translation, behaviors that weren't seen in smaller models trained on the same data.
Foundation models represent a fundamental shift in AI development. Instead of training separate models for each task, organizations can fine-tune a single foundation model for multiple applications. This dramatically reduces the cost and expertise required to deploy AI, which is why foundation models have catalyzed an explosion of AI-powered products since 2022.
Read next
What Is a Large Language Model?
A clear, jargon-free introduction to large language models, what they are, how they work at a high level, and why they're transforming software and society.
How LLMs Work: A Technical Overview
A clear technical explanation of how large language models actually process text, generate responses, and represent knowledge, from tokenization to sampling.
The Scaling Laws of LLMs: Why Bigger Often Means Better
The mathematical relationship between model size, training data, compute, and capability, and what the scaling laws predict about the future of AI.