The new Green500 list of the most energy-efficient supercomputers demonstrates some significant progress from last year. Thanks to the new manycore processors from Intel and NVIDIA that are starting to penetrate the top systems, performance per watt numbers are on the rise.
The top two systems, DGX SATURNV and Piz Daint, both use NVIDIA’s P100 Tesla GPU, which exhibits excellent energy efficiency. The DGX SATURNV, NVIDIA’s first petascale in-house supercomputer, earned the top spot on the Green500 list with a rating of 9.46 gigaflops per watt. That’s more than a 40 percent jump from the number one system just six months ago on the June 2016 list. It owes its superior efficiency to the server upon which it is based: NVIDIA’s DGX-1, which houses eight P100 GPUs.
Piz Daint, a Cray XC50 system, took second place with a mark of 7.45 gigaflops per watt. That system raised its stature on the list due to an upgrade from its original K20x graphics processors to the new P100 GPUs. Although it’s based on the same processor technology as the DGX SATURNV, it uses a lower ratio of GPUs to CPUs. The former number one Green500 system, the Shoubu supercomputer, equipped with the PEZY-SCnp manycore accelerator, slipped into third place with an energy efficiency of 6.67 gigaflops per watt.
With the exception of the XStream system at number eight, which uses the NVIDIA K80 accelerator, the remainder of the top 10 systems relied on homogeneous manycore processors in a standalone configuration. Five of those machines are powered by Intel’s new “Knights Landing” Xeon Phi processors, which exhibited energy efficiencies of between 3.84 and 5.61 gigaflops per watt. The outlier here is the Sunway TaihuLight supercomputer, the number one system on the TOP500 list, which earned the number four spot on the Green500 with a value of 6.05 gigaflops per watt. Taihulight is powered by the SW26010, a 260-core ShenWei processor, designed and manufactured in China.
Overall, heterogeneous systems continue to demonstrate better energy efficiency than homogeneous systems, but that gap appears to be narrowing with the appearance of standalone manycore processors like Xeon Phi and ShenWei. One interesting trend to watch will be how these manycore architectures continue to evolve, and if the self-hosted ones will be able to catch or overtake coprocessors with regard to energy efficiency.
There has been progress in other areas as well. For the first time in the 10-year history of the Green500, the mean power consumption of all systems on the list actually went down. In the past, due to the growth in the size of the top supercomputers, power consumption rose even as the energy efficiency of those machines dropped. It remains to be seen if the reversal is a one-time event or will start a new trend.
It’s not expected that the topmost systems on the list will make such large gains in energy efficiency over the next six to twelve months as they did during this cycle. The next big jump is apt to occur when the next generation of GPUs, Xeon Phi processors, and perhaps the Chinese manycore processors are available to supercomputer makers in the late 2017/early 2018 timeframe.
If we use the most efficient system on the current list (the DGX SATURNV) as a baseline, an exascale system with that level of energy efficiency would draw 105.7 MW. That is about five times the desired goal of 20 MW, but only a little more than three times the 30 MW upper limit of the initial systems targeted under the US exascale effort.
The good news is that we can expect two or three more generations of manycore processors before the first exascale systems are built, so the wattage for the initial systems should be considerably less than 100 MW. Green500 founder Wuchun Feng is optimistic that the numbers are starting to line up. “While reaching a 20 MW exascale is highly unlikely by 2020, China has unofficially indicated that their target power for an exascale supercomputer in 2020 to be 40 MW,” Feng told TOP500 News. “This 40 MW projection, I think, is attainable.”
Regardless of the exascale milestone, HPC systems continue to become more energy efficient with each passing year, thanks in part to Moore’s Law, but also due to a concerted effort on the part of component makers and system engineers, who are squeezing the maximum amount of performance from a given power envelop.
The next Green500 list will be announced in June 2017 in conjunction with the ISC High Performance conference, which will take place in Frankfurt, Germany.
Listed below are the November 2016 The Green500's energy-efficient supercomputers ranked from 1 to 10. For more information about the sites and systems in the list, click on the links or view the complete list.
| Green500 Rank | MFlops/watts | Site | System | Total Power(kW) |
|---|---|---|---|---|
| 1 | 9462.1 | NVIDIA Corporation | NVIDIA DGX-1, Xeon E5-2698v4 20C 2.2GHz, Infiniband EDR, NVIDIA Tesla P100 | 349.5 |
| 2 | 7453.5 | Swiss National Supercomputing Centre (CSCS) | Cray XC50, Xeon E5-2690v3 12C 2.6GHz, Aries interconnect , NVIDIA Tesla P100 | 1312 |
| 3 | 6673.8 | Advanced Center for Computing and Communication, RIKEN | ZettaScaler-1.6, Xeon E5-2618Lv3 8C 2.3GHz, Infiniband FDR, PEZY-SCnp | 150.0 |
| 4 | 6051.3 | National Supercomputing Center in Wuxi | Sunway MPP, Sunway SW26010 260C 1.45GHz, Sunway | 15371 |
| 5 | 5806.3 | Fujitsu Technology Solutions GmbH | PRIMERGY CX1640 M1, Intel Xeon Phi 7210 64C 1.3GHz, Intel Omni-Path | 77 |
| 6 | 4985.7 | Joint Center for Advanced High Performance Computing | PRIMERGY CX1640 M1, Intel Xeon Phi 7250 68C 1.4GHz, Intel Omni-Path | 2718.7 |
| 7 | 4688.0 | DOE/SC/Argonne National Laboratory | Cray XC40, Intel Xeon Phi 7230 64C 1.3GHz, Aries interconnect | 1087 |
| 8 | 4112.1 | Stanford Research Computing Center | Cray CS-Storm, Intel Xeon E5-2680v2 10C 2.8GHz, Infiniband FDR, Nvidia K80 | 190 |
| 9 | 4086.8 | Academic Center for Computing and Media Studies (ACCMS), Kyoto University | Cray XC40, Intel Xeon Phi 7250 68C 1.4GHz, Aries interconnect | 748.1 |
| 10 | 3836.6 | Thomas Jefferson National Accelerator Facility | KOI Cluster, Intel Xeon Phi 7230 64C 1.3GHz, Intel Omni-Path | 111 |