Supercharging a supercomputer

Tokyo Tech's continually evolving TSUBAME supercomputer cluster is a perennial performance leader among the world's supercomputers. And a recent upgrade with graphics processing units (GPUs) has raised the system's performance further. Tokyo Tech has employed 170 Nvidia Tesla S1070 1U systems to increase TSUBAME's theoretical peak computing speed to a mind-boggling 170 teraflops. That has helped maintain the system's standing at the high end of the much-watched Top500 Supercomputers ranking.

The upgrade, carried out in November 2008, optimizes the first-generation TSUBAME, which Tokyo Tech installed in 2006. It is in step with Tokyo Tech's preparations to design and install a brand-new version of the supercomputer, TSUBAME 2.0, in 2010.

"We work constantly to position TSUBAME in the vanguard of computing platforms," explains Tokyo Tech's Satoshi Matsuoka (photo), who heads the TSUBAME project. "Our system had reached a juncture where GPUs were the best technology for achieving the next leap in performance.

"Modern GPUs, despite their name, are actually massively parallel processors and accommodate general-purpose computing. Deploying them was therefore a more-effective solution for our needs than just adding conventional CPUs. Nvidia's Tesla GPUs have delivered unprecedented speedups--more than a hundredfold with some applications. That means huge gains in computing power for our scientists and engineers. And we have achieved those gains at low cost, at low power consumption, and in minimal space.

"We completed the entire upgrade for the whole machine in merely two weeks, and TSUBAME remained live throughout. The fast-turnaround upgrade has raised TSUBAME's measured LINPACK performance to 77.48 teraflops. That is double the system's original performance, two and a half years ago. What has amazed me is that there is little visible evidence of the upgrade. The Teslas are barely noticeable because their footprint is extremely small, compared with the rest of TSUBAME."

Matsuoka is referring to LINPACK benchmarks used by Top500, a joint undertaking by the University of Mannheim, the University of Tennessee, and the National Energy Research Scientific Computing (NERSC) Center at Lawrence Berkeley National Laboratory. Top500 ranks the world's 500 fastest supercomputers twice annually on the basis of those benchmarks. In the November 2008 results, TSUBAME overtook the University of Tsukuba's machine, which had overtaken TSUBAME earlier in the year, to achieve the No. 2 spot among Japanese supercomputers. Matsuoka's team is working feverishly to reclaim the top spot in Japan from a system at the University of Tokyo in the June 2009 rankings.

This graphic, which comprises 19.7 billion polygons, is a visualization of turbulent flow, such as occurs behind a glider's wing. TSUBAME's massive processing capacity has allowed for elucidating previously undetectable micromotion inside air currents. The supercomputer supports research in numerous fields, including geophysics, chemistry, material sciences, mechanical engineering, meteorology, and bioinformatics.

"We're proud to be a world leader in computing performance," acknowledges Matsuoka. "But we're even more proud of TSUBAME for another reason. This system is a huge step toward our goal of making supercomputing available to nonspecialist users.

"TSUBAME's user population has grown annually and now consists of more than 1,500 users. Any of the more than 10,000 members of the Tokyo Tech community, including undergraduate students, can secure user accounts. That is in contrast with the common pattern among supercomputer systems, which tend to be available to just a few specialists. We also have hundreds of users outside Tokyo Tech, including users in industry. The addition of GPUs to TSUBAME will make supercomputing available to a growing range of technologies. Their massively parallel processing is likely to become a mainstream computer resource."

Matsuoka, a professor in Tokyo Tech's Global Scientific Information and Computing Center, earned his doctorate in computer science from the University of Tokyo in 1993. A pioneer in grid computing and cluster computing in Japan, he has been at Tokyo Tech since 1996. Recognition for his work has included the 2005 JSPS Prize from the Japan Society for the Promotion of Science, presented by Prince Akishinonomiya. Matsuoka served for five years as a subleader of Japan's National Research Grid Initiative (NAREGI) for creating middleware for next-generation e-science infrastructure, and he is the technical papers chair for Association for Computing Machinery/IEEE (ACM/IEEE) Supercomputing 2009. ACM/IEEE Supercomputing, the world's preeminent supercomputing conference, draws more than 10,000 participants from around the world each year, and it is the venue for the autumn announcement of the Top500 rankings.

TSUBAME's unprecedented accessibility to researchers in diverse fields is partly attributable to the system's PC-based architecture. It is also a tribute to Matsuoka's commitment to what he calls "supercomputing for everyone." That commitment has yielded results through extensive research by Matsuoka and his team, through evangelical outreach to potential users inside and outside Tokyo Tech, and through conscientious orientation and support.

Matsuoka and his team have achieved two-tier accessibility: ready access for nonspecialist users on their PCs, and direct access to supercomputing resources. Users who proceed from PC access to direct access as they become proficient enjoy a seamless usage and programming environment for the same set of applications.