He Wants AI Benchmarking To Be a Science, Not a Dark Art
Moritz Hardt, author of a new book on benchmarks, believes these “standardized tests” for artificial intelligence are fundamentally broken — but not unfixable.
Moritz Hardt, author of a new book on benchmarks, believes these “standardized tests” for artificial intelligence are fundamentally broken — but not unfixable.
The short version
- The post He Wants AI Benchmarking To Be a Science, Not a Dark Art first appeared on Quanta Magazine.
- An editorially independent publication supported by the Simons Foundation.
- Create a reading list by clicking the Read Later icon next to the articles you wish to save.
What happened
The computer scientist Moritz Hardt calls benchmarks “the mistake that made machine learning.” He’s studied where they went wrong, and why they’ve managed to drive progress anyway. Benchmarks are the rulers of AI, in both senses of the word.
Why it matters
As measuring sticks, they work according to an extremely simple logic: If you want to know what an AI model can do, devise a standardized test to gauge it.
Summary by Nerd News Network. Read the full article at Quanta Magazine via the links above and below.