The Most Misunderstood Graph in AI Gets a Major Update from METR
- METR's updated time horizon plot evaluates AI models based on the human completion time of assigned tasks.
- The original METR plot showed that the length of tasks frontier models could successfully complete doubled every four months.
- None of the AI-designed evaluations in METR's new tests were deemed useful enough for human researchers to use in their work.
Background & Context Analysis
A new report from the AI evaluation nonprofit METR updates its widely shared and misinterpreted time horizon plot, revealing that while frontier AI capabilities continue to improve exponentially on verifiable tasks, models struggle with judgment-heavy tasks. Using human completion time to measure task difficulty, the original plot showed that the length of tasks frontier models could successfully complete doubled every four months. The updated plot incorporates longer and more open-ended tasks that are essential for automating research and development. The findings highlight a critical gap in current AI capabilities. While reinforcement learning has made systems exceptionally proficient at verifiable domains like coding and math, applying this technique to tasks with less defined success criteria remains difficult. In METR's new tests, advanced models failed to produce a single AI evaluation useful for human researchers, and showed mixed performance in scoring philosophical arguments, demonstrating that fully autonomous AI research and development remains limited despite rapid progress in software engineering.
✅ Source Verification
- Original source
- MIT Technology Review (🇺🇸 United States)
- Original language
- English
- Originally published
- 2026-11-20 18:15 (11h ago)
- How this briefing was made
- This briefing was structured and translated by AI from the original report. Key facts (names, numbers, dates) come from the source; synthesis may contain errors — please check the original.
Was this AI briefing helpful?
✨ Ask AI About This Article
Ask follow-up questions grounded strictly in this briefing