AIME: Why Competition Math Is the New Benchmark for AI Reasoning

What AIME Is

The American Invitational Mathematics Examination (AIME) is a prestigious 15-problem mathematics competition for top US high school students, specifically, those who have qualified through the AMC 10 or AMC 12 by placing in the top 2.5-5% of participants. AIME problems require multi-step mathematical reasoning across number theory, algebra, geometry, combinatorics, and probability. There are no multiple-choice options: each answer is an integer from 0 to 999, which eliminates lucky guessing entirely.

A perfect AIME score is 15 (one point per correct answer). Students who qualify for AIME, the top few percent of high school math students in the country, typically score 3-7. The top 250-270 AIME scorers each year qualify for the USA Mathematical Olympiad, an even harder competition. This context establishes the human difficulty baseline: AIME is not just 'hard math', it's math that consistently stumps even the most mathematically talented high school students in the United States.

Why AIME Tests Genuine Reasoning

What makes AIME valuable as an AI benchmark is the nature of its problems. AIME questions cannot be solved by retrieving a formula and substituting values, they require discovering the right approach, which is non-obvious and often requires insight specific to the problem. The same topic area (combinatorics, say) can be addressed by radically different techniques depending on problem structure, so pattern-matching to problem type is insufficient. A model that has memorized competition problem solutions will fail on novel AIME problems because each requires genuine reasoning from mathematical first principles.

AIME problems also resist contamination better than many benchmarks. While individual AIME problems from past years are published online and likely appear in training data, the AIME is administered annually with new problems each year. Using AIME 2024 and AIME 2025 (rather than problems from many years ago) provides some contamination resistance, the more recent the exam year, the less likely problems appear in training data. This makes recent AIME years the most reliable signal of genuine mathematical reasoning ability.

The integer-answer format is a further anti-contamination property. Multiple-choice benchmarks can be gamed by models that learn answer distributions or apply process of elimination without understanding. With AIME, there are 1,000 possible answers per question; guessing is statistically useless. A model that answers 10 out of 15 correctly has genuinely solved those problems.

The Scoring Structure and Evaluation Protocol

Each AIME has 15 problems. Scores are reported as the number correct out of 15, equivalently, as a percentage of 15. In AI evaluation, scores are sometimes reported as the fraction correct across an entire year's exam (pass rate) or averaged across multiple years. The distinction matters: AIME 2024 Problem 1 is substantially easier than AIME 2024 Problem 15, so a model that correctly solves Problems 1-10 but not 11-15 has a different capability profile than one with a scattered pattern across difficulties.

Evaluation protocol also matters. Some evaluations use temperature 0 (greedy decoding, single attempt), others use temperature > 0 with multiple samples (best-of-N). Because AIME problems have single integer answers, automated verification is straightforward, no rubric or LLM judge is needed. This makes AIME evaluation cleaner than many open-ended benchmarks where scoring subjectivity is a concern.

What AIME Scores Mean

Interpreting AIME performance requires knowing the human baseline. Typical AIME-qualifying high school students score 3-7 (20-47%). The average score among all AIME participants is approximately 5-6. Students who score 10+ are in exceptional territory, these students typically become top university mathematicians or professional researchers in quantitative fields. A perfect score of 15 is extremely rare even among the most gifted competitors.

In AI terms: a model scoring below 3 (20%) is below the average AIME qualifier, not distinguishing itself on hard math. A model scoring 5-8 is in the range of a typical high-performing qualifier. A model scoring 10-13 is performing at the level of exceptional students who go on to compete at national olympiad level. A model consistently scoring above 13 is performing beyond what most professional mathematicians could do on demand, in competition conditions, without verification tools.

How Reasoning Models Transformed the AIME Leaderboard

AIME performance is the benchmark where the reasoning model inflection point is most dramatically visible. Standard instruction-tuned frontier models score 30-55% (4.5-8.25 out of 15) on AIME 2024, respectable but within the range of strong human competitors. Reasoning models, specifically o1, o3, and their successors, score 70-90% (10.5-13.5 out of 15) on the same exam, well above the range of all but the most exceptional human competitors.

The mechanism is extended chain-of-thought reasoning: reasoning models are trained to produce long internal reasoning sequences before committing to an answer. For AIME problems, this means the model can work through multiple sub-steps, check its work, backtrack when it detects errors, and try alternative approaches, exactly the systematic process that human competitors use. Standard models don't have enough inference-time compute allocated to this iterative process.

The jump from o1 to o3 on AIME is also notable: o1 scored approximately 74% on AIME 2024; o3 scores approximately 87%. This improvement came from scaling reasoning compute and from improvements to the training process for reasoning models. The continued rapid improvement suggests that mathematical reasoning is one of the capabilities most amenable to the reasoning model training paradigm.

Current Frontier Scores

As of mid-2025 on AIME 2024: o3 achieves approximately 87% (13/15), o4-mini approximately 93% (14/15), GPT-5 approximately 80% (12/15), Claude Opus 4 approximately 74% (11/15), and Gemini 2.5 Pro approximately 92% (13.8/15). Standard instruct-tuned models without reasoning training cluster around 40-55%. The difference between reasoning and non-reasoning models is typically 30-40 percentage points, larger than on almost any other benchmark.

AIME 2025 provides a partially fresh challenge: scores are generally 5-15 percentage points lower than AIME 2024, both because the problems are newer (less likely to appear in training data) and because 2025 problems were deliberately designed to be harder. As of early 2025, o3 scores approximately 75% on AIME 2025, Gemini 2.5 Pro approximately 88%, and o4-mini approximately 86%. The gap between AIME 2024 and 2025 scores is an imperfect but useful signal of how much training data contamination inflates AIME 2024 results.

Limitations and What Comes After AIME

AIME's saturation is approaching. When leading models consistently score 13-15 out of 15, the benchmark will lose discriminating power among frontier reasoning models, the same pattern that claimed MMLU and HumanEval. The research community is already developing harder mathematical benchmarks: FrontierMath (research-level mathematics, currently below 10% for frontier models), USA Mathematical Olympiad (USAMO) problems, and the International Mathematical Olympiad (IMO), the highest level of pre-university mathematical competition.

Frontier models score very low on IMO problems in general solution form: not because they can't produce steps, but because complete mathematical proofs require logical rigor and creativity that current models still lack. FrontierMath, developed by Epoch AI, contains problems sourced from active research mathematicians and designed to take expert mathematicians hours to days to solve. Current frontier models score under 10% on FrontierMath, suggesting substantial headroom remains at the highest levels of mathematical reasoning.

For practitioners: AIME performance is the most predictive available signal for mathematical reasoning capability in applications like quantitative finance modeling, physics simulation, formal verification, and scientific computing support. A model that scores well on AIME can follow complex multi-step mathematical arguments, check their own reasoning, and catch errors in proposed solutions. For applications requiring serious mathematical rigor, checking AIME and MATH-500 scores together is the recommended starting point.