BLACKSBURG, Va., June 28, 2013 -- Today’s release of the Green500 List shows that the top end of the list is again dominated by heterogeneous supercomputers, those that combine two or more types of processing elements together, such as a traditional processor or central processing unit (CPU) combined with a graphical processing unit (GPU) or coprocessor.
Two heterogeneous systems, based on NVIDIA’s Kepler K20 GPU accelerators, claim the top two positions and break through the three-billion floating-point operations per second (gigaflops or GFLOPS) per watt barrier. Eurora, located at Cineca, debuts at the top of the Green500 at 3.21 gigaflops/watt, followed closely by Aurora Tigon at 3.18 gigaflops/watt. The energy efficiency of these machines, manufactured by Eurotech, improves upon the previous greenest supercomputer in the world by nearly 30%. Two other heterogeneous systems, Beacon with an efficiency of 2.449 gigaflops/watt* and SANAM with an efficiency of 2.35 gigaflops/watt, come in at numbers 3 and 4 on the Green500. The former is based on Intel Xeon Phi 5110P coprocessors while the latter is based on AMD FirePro S10000 GPUs.
Rounding out the top five is CADMOS BlueGene/Q, which is based on a previously list-leading custom design of the IBM BlueGene/Q architecture with PowerPC-based CPU processors. In fact, the next swath of supercomputers down to No. 28 are dominated by IBM BlueGene/Q systems with one set at approximately 2.30 gigaflops/watt and another at approximately 2.18 gigaflops/watt. Overall, the greenest fifty systems are either heterogeneous systems, which incorporate accelerators (GPUs) or coprocessors, or custom BlueGene/Q systems. The exceptions are a pair of homogeneous systems at 40 and 41, which are the only homogeneous systems based on Intel’s new Ivy Bridge processors and three more at 45-47 based on Intel Sandy Bridge processors.
The current fastest supercomputer in the world, Tianhe-2, uses heterogeneous computing elements based on Intel Xeon Phi. It delivers 1.9 gigaflops/watt, which is commensurate with the majority of Xeon Phi systems that are ranked between No. 30 and 35, inclusive.
With the DARPA’s goal of an exascale supercomputer within a power envelope of 20 megawatts (MW), extrapolating Beacon, the previous No. 1 supercomputer on the Green500, to exascale results in a 408 MW machine. However, due to the improved efficiency of Eurora, this exascale power envelope comes down to 312 MW, a sizeable 24% drop in power consumption for an exascale machine -- a move in the right direction. Nevertheless, the electricity bills for such a system would still be more than 300 million U.S. dollars per year.
This 13th edition of the Green500 List also marks the Green500’s first use of new energy measurement methodologies developed in tandem with the Energy Efficient High Performance Computing Working Group, Top500, and The Green Grid. In addition to the current Green500 submission requirements, denoted as Level 1, the Green500 also accepts higher-precision measurements, denoted as Level 2 and 3. The Green500 received Level 2 and 3 submissions from several systems, including a superset of the Sequoia BlueGene/Q supercomputer at LLNL called Sequoia-25 and SuperMUC from LRZ. “While the higher quality measurements taken for these systems show lower energy efficiency, they provide a much better picture of the real-world costs of running each system as well as a more in-depth picture of how the system handles a Linpack run,” noted Wu Feng, founder of the Green500. “Additionally, the Sequoia-25 Level 1 submission includes the networking infrastructure, as required for higher levels.”
Since the launch of the Green500 in 2007, the energy efficiency of the highest-ranked machines on the Green500 has been improving much more rapidly than the mean and the median. For instance, while the energy efficiency of the greenest system improved by nearly 30%, the median improved by 14% and the mean by only 11%. “Overall, the performance of machines on the Green500 List has increased at a higher rate than their power consumption. That's why the machines' efficiencies are going up," says Feng. For commodity-based machines -- machines built with off-the-shelf components -- a great deal of the efficiency gains can be attributed to heterogeneous designs, i.e., using traditional CPUs along with GPUs or coprocessors. Such a design allows these systems to keep pace and in some cases even outpace custom systems like IBM’s Blue Gene/Q. "While the gains at the top end of the Green500 appear impressive, overall the improvements have been much more modest," says Feng. “This clearly indicates that there is still work to be done.”
The Green500 has provided a ranking of the most energy-efficient supercomputers in the world since November 2007. For decades, the notion of supercomputer "performance" has been synonymous with "speed" (as measured in FLOPS, short for floating-point operations per second). This particular focus has led to the emergence of supercomputers that consume egregious amounts of electrical power and produce so much heat that extravagant cooling facilities must be constructed to ensure proper operation. In addition, when there is an emphasis on speed as the ultimate metric, it often comes at the expense of other performance metrics, such as energy efficiency, reliability, availability, and usability. As a result, there has been an extraordinary increase in the total cost of ownership (TCO) of a supercomputer. Consequently, the Green500 seeks to raise the awareness in energy efficiency of supercomputing and to drive it as a first-order design constraint on par with speed.
ERRATA: We had a typo in the original release that reported 2.49 gigaflops/watt for the Beacon machine when it should have read 2.449 as it does now. The numbers for the MVS-10P at the Joint Supercomputer Center were revised and the rankings adjusted. The TOP500 data was used in lieu of submitted data for the MVS-10P machine causing the error in ranking. As a result the MVS-10P machine was moved from 34 to 30 in the rankings and the others adjusted to reflect the updated numbers.
Listed below are the June 2013 The Green500's energy-efficient supercomputers ranked from 1 to 10. For more information about the sites and systems in the list, click on the links or view the complete list.
| Green500 Rank | MFlops/watts | Site | System | Total Power(kW) |
|---|---|---|---|---|
| 1 | 3208.8 | CINECA | Eurotech Aurora HPC 10-20, Xeon E5-2687W 8C 3.100GHz, Infiniband QDR, NVIDIA K20 | 30.7 |
| 2 | 3179.9 | Selex ES Chieti | Eurotech Aurora HPC 10-20, Xeon E5-2687W 8C 3.100GHz, Infiniband QDR, NVIDIA K20 | 31.0 |
| 3 | 2449.6 | National Institute for Computational Sciences/University of Tennessee | Appro GreenBlade GB824M, Xeon E5-2670 8C 2.600GHz, Infiniband FDR, Intel Xeon Phi 5110P | 45.1 |
| 4 | 2351.1 | King Abdulaziz City for Science and Technology | Adtech, ASUS ESC4000/FDR G2, Xeon E5-2650 8C 2.000GHz, Infiniband FDR, AMD FirePro S10000 | 179.2 |
| 5 | 2299.1 | IBM Thomas J. Watson Research Center | BlueGene/Q, Power BQC 16C 1.60 GHz, Custom | 82.2 |
| 6 | 2299.1 | DOE/SC/Argonne National Laboratory | BlueGene/Q, Power BQC 16C 1.600GHz, Custom Interconnect | 82.2 |
| 7 | 2299.1 | Ecole Polytechnique Federale de Lausanne | BlueGene/Q, Power BQC 16C 1.600GHz, Custom Interconnect | 82.2 |
| 8 | 2299.1 | Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw | BlueGene/Q, Power BQC 16C 1.600GHz, Custom Interconnect | 82.2 |
| 9 | 2299.1 | DOE/SC/Argonne National Laboratory | BlueGene/Q, Power BQC 16C 1.60GHz, Custom | 82.2 |
| 10 | 2299.1 | University of Rochester | BlueGene/Q, Power BQC 16C 1.60GHz, Custom | 82.2 |