The World’s New Fastest Supercomputer Is an Exascale Machine for AI

Twice a year, the world’s fastest supercomputers take a test to see which is top of class.
These hundred-million-dollar machines usually run on hundreds of thousands of processors, occupy warehouse floors, gobble up copious amounts of energy, and crunch numbers at an ungodly pace. All that computing is directed at some of humanity’s toughest challenges with the likes of advanced climate modeling or protein simulations to help cure diseases.
For the last two years, the US’s Summit was the fastest supercomputer on the planet. But this week, a new system took the crown . Running 2.8 times faster than Summit, Japan’s Fugaku notched a blistering 415 petaflops as measured by Top500’s high-performance linpack benchmark (HP-L).
That means Fugaku completes a simple mathematical operation 415 quadrillion times a second. You’d need every person on Earth to complete a calculation a second for 20 months—no bathroom breaks—to match what Fugaku does in a heartbeat.
Japan last claimed the top spot with its K computer in 2011. Developed by the Riken Insitute and Fujitsu, Fugaku took a billion dollars and the better part of the ensuing decade to build. It’s notable because it doesn’t use graphics processing units (GPUs), like many of its competitors, and it’s the first top supercomputer to use Arm processors—an efficient chip design commonly used in mobile devices.
In addition to all that, Fugaku is insanely fast at machine learning.
World’s Top AI Brainiac
While supercomputers have historically been mostly about military and scientific research—and Fugaku has already been crunching coronavirus data—they’re also increasingly being tailored to run machine learning algorithms. Indeed, Fugaku’s predecessor, Summit, was designed from the ground up with AI in mind.
Likewise, Fugaku will be an unparalleled AI brainiac.
By a new measure, HPL-AI, Fugaku was able to do the kind of calculations used in today’s machine learning algorithms at a speed of 1.4 exaflops . That mark is fastest in the world.
Exascale computing by more traditional measures (that is, by HP-L, not HPL-AI) is the next big computing milestone, anticipated for over a decade. The first such systems are expected next year or the year after. But for machine learning, Fugaku is already there.
That’s significant because AI researchers are scaling up machine learning algorithms at a quick pace. OpenAI, for instance, recently pulled back the curtains on a massive new machine learning algorithm for natural language processing called GPT-3 . The algorithm is notable for its size, 175 billion parameters, as well as its ability to learn and perform a range of tasks.
OpenAI also partnered with Microsoft to fund and build a supercomputer dedicated to its machine learning efforts . Microsoft claimed (unofficially) that it would be the fifth fastest supercomputer in the world, though the system has...