Research

Tokyo Tech News

TSUBAME-KFC is Ranked No. 1 on both Green500 and Green Graph 500 Lists

December 4, 2013

  • Overwhelming performance by the TSUBAME-KFC (glossary term 1), the next-generation TSUBAME 3.0 test system featuring an oil cooling system, led to its being ranked No. 1 in the world on the Green500 list (for power efficiency), a first for a Japanese supercomputer. It was also ranked No.1 on the Green Graph 500 list (for power-efficient big data processing), for a total of two top awards in power efficiency.
  • TSUBAME 2.5 was ranked No. 6 in the world on the Green500 list (for power efficiency). It ranked No. 11 in the world and No. 2 in Japan on the TOP500 list (for processing speed), showing a strong overall performance.

TSUBAME-KFC, a supercomputer developed by the Global Scientific Information and Computing Center (GSIC) at Tokyo Institute of Technology (Tokyo Tech) in cooperation with NEC Corporation, NVIDIA Corporation in the U.S. and other domestic and overseas companies, went into operation in October 2013 and was recognized as the world's most power-efficient supercomputer. It was ranked No.1 in the world in the November 2013 edition of the Green500 list (glossary term 2) with an efficiency of 4503.17 MegaFlops/Watt, which put great distance between it and its closest rivals, according to the announcement on November 20 at the SC13 world supercomputing conference in Denver, USA. It is the first time a Japanese supercomputer has obtained the top position on the Green500 list and can be called a display of joint Japan-US technological leadership in the realization of a low-carbon society.

At the same time, TSUBAME-KFC was also ranked No.1 in the world in the big data division of the Green Graph 500 list (glossary term 3). This division was started this year as a competition for energy efficiency in big data processing. To receive the prize the TSUBAME-KFC had to outperform IBM's Blue Gene/Q supercomputer, which held the No. 1 position on the previous Green Graph 500 list of May 2013, with its high-level big-data/graph processing capabilities. This was the second top prize the TSUBAME-KFC received for power efficiency this time around.

Additionally, GSIC's TSUBAME 2.5 supercomputer, which is an upgrade from September 2013, recorded 3068.71 MegaFlops/Watt, which gave it a ranking of No. 6 in the world on the Green500 list. TSUBAME 2.5 also made a comeback on the TOP500 list, being ranked No. 11 in the world and No. 2 in Japan behind the K computer.

TSUBAME-KFC
TSUBAME-KFC

TSUBAME2.5
TSUBAME2.5

The TSUBAME-KFC was designed and developed as a test-bed system for TSUBAME 3.0, the successor of TSUBAME 2.0 and subsequent models, through an ultra-green supercomputing project, which is part of a budget request made by GSIC to the Ministry of Education, Culture, Sports, Science and Technology. The project aims to reduce both supercomputer power consumption and the associated power necessary for cooling. The TSUBAME-KFC is designed to enable cooling with extremely low power consumption through a combination of oil-submersion cooling technology, in which the data processing system is immersed in an oil-based cooling solvent fluid that circulates around the compute nodes, and air-cooling with cooling towers.

The TSUBAME-KFC system is made from 40 compute nodes and an FDR InfiniBand network that connects them. Each compute node has 2 Intel Xeon E5-2620 v2 processors (Ivy Bridge EP) and 4 NVIDIA Tesla K20X GPUs on a 1U-size server, making for an extremely high density. The compact design features 40 nodes per oil-submersion rack. The entire system's theoretical peak operating performance is 217 TeraFlops (double-precision).

Its predecessor, TSUBAME 2.0, was ranked No. 4 in the world on the November 2010 Top500 list and No. 2 in the world on the Green500 list. It was also the recipient of the Greenest Production Supercomputer in the World award. A September 2013 upgrade of TSUBAME 2.0, the TSUBAME 2.5 uses the latest NVIDIA Tesla K20X GPUs (glossary term 4) replacing the formerly used NVIDIA Tesla M2050s. As a result of this switch, TSUBAME 2.5 has drastically increased its operating performance to 5.7 PetaFlops (glossary term 5) at double-precision and 17 PetaFlops at single-precision. Its power efficiency also improved by more than a factor of three and it was recognized again as one of the world's top supercomputers for power efficiency on the most recent Green500 list.

The current results are the fruit of a variety of research targeting power conservation conducted at GSIC. Based on these results, numerous joint developments with industry have been carried out, primarily with NEC and NVIDIA, but also with other companies such as Green Revolution Cooling, Inc., Super Micro Computer, Inc., Intel Corporation and Mellanox Technologies Ltd. In addition to the research for the ultra-green supercomputing project, other research is continuously being pursued at the center. Some of these other projects include a basic research program in ULPHPC (Ultra Low Power High Performance Computing) as one of the Japan Science and Technology Agency Strategic Basic Research Programs (also known as JST-CREST), research into wide-scale applications of the latest GPUs in supercomputers, and power conservation in HPC systems as part of an ongoing joint research project with NVIDIA Corporation.

Evolution and Future of TSUBAME Supercomputers: How TSUBAME will evolve and be used

A commemorative symposium, Evolution and Future of TSUBAME Supercomputers: How TSUBAME will evolve and be used, will be held on Tuesday, December 10, 2013 from 1 p.m. The symposium will cover a wide range of topics, including TSUBAME 2.5, research and development for TSUBAME 3.0, applications for ultra-large-scale applications, and use in industry. The symposium language is Japanese.

Glossary Term 1 TSUBAME-KFC

TSUBAME Kepler Fluid Cooling, like TSUBAME 2.5, has NVIDIA Kepler generation GPUs. The name comes from the fact that the compute nodes are submerged in fluid for cooling.

Glossary Term 2 The Green500 List

Comparable to the TOP500 list, where the benchmark speed performance of the world's supercomputers is ranked from 1 to 500 every 6 months, the Green500 list, in keeping with the 'green' trend of recent years, ranks the power efficiency (speed performance / power consumption) of the TOP500 supercomputers every 6 months.
http://www.green500.orgouter

Glossary Term 3 The Green Graph 500 List

Launched in May 2013, the Green Graph 500 list ranks the Graph 500 supercomputers every six months for power efficiency (analysis performance / power consumption) in big data analysis performance.
http://green.graph500.org/outer

Glossary Term 4 GPU (Graphics Processing Unit)

Originally a dedicated processor for computer graphics, they have become more general-purpose and higher-capacity as graphics processing has become more complex. They are currently used as generic vector processors for HPCs.

Glossary Term 5 PetaFlops, TeraFlops

Flops is a performance measure indicating the number of floating-point operations possible per second. Giga (109), Tera (1012) and Peta (1015), etc.