GPUs for atmospheric simulations

Atmospheric simulations can be run much faster by employing computer processors that were originally designed for high-quality computer graphics

GPU (Graphics Processing Unit)—processors originally developed for graphic displays—are now being actively used for scientific calculations and simulations.

Tokyo Institute of Technology’s TSUBAME2.0 was ranked 4^th in the November 2010 supercomputer Top 500 for supercomputers using GPU together with CPU. Notably, the Tokyo Institute of Technology pioneered GPU-based supercomputers when it introduced GPU in the TSUBAME1.2. The advantages of the GPU supercomputer are high performance, low cost, low power consumption, and a small ‘footprint’. The disadvantage is the portability of the existing code.

There used to be a difficulty with atmospheric simulations due to the requirement of the vast quantity of calculations. Takayuki Aoki of Tokyo Institute of Technology’s Global Scientific Information and Computing Center, Takashi Shimokawabe of the Interdisciplinary Graduate School of Science and Engineering, and co-workers have made tremendous advances in this field by employing over 4000 GPUs in the TSUBAME 2.0 supercomputer, thereby overcoming the aforementioned difficulty associated with GPUs. This new machine was constructed by the Tokyo Institute of Technology in collaboration with private companies NEC and HP. TSUBAME2.0 was launched in autumn 2010 and is capable of doing calculations at speeds measured in petaflops, where one petaflop is 10¹⁵ FLoating point OPerations per Second. By using this new supercomputer, researchers at Tokyo Institute of Technology have succeeded in running atmospheric simulations faster than ever before.

GPUs are dedicated to performing floating point operations—which means performing basic calculations on large numbers— and they can do so very fast. In recent years they have been finding more uses in general purpose computing and science. For example the atmosphere simulation model WRF (Weather Research and Forecasting) used worldwide has been speeded up by running some GPUs alongside a conventional host CPU (Central Processing Unit)—but the benefit was limited because of partial GPU computing and slow data transfer between the GPU and CPU.

The researchers focused on using TSUBAME 2.0 to run ASUCA, a high-resolution atmospheric model developed by the Japan Meteorological Agency. Unfortunately, it wasn’t as easy as just porting all the original programs to the GPUs and running the model. ACUSA was originally written in the Fortran programming language, which is not compatible with GPUs, so the researchers had to rewrite the entire program from scratch in the GPU language, called CUDA.

In their first tests, using single a GPU for small model runs, the researchers achieved computation speeds 12 times higher than on 1 CPU with 6 CPU-cores. They focused on optimizing some of the model code to make it more ‘streamlined’: for example they improved calculations of atmospheric pressure and wind advection by making use of shared memory rather than transferring too much data between processors and video memory.

Finally, by employing 3990 GPUs together the researchers measured calculation speeds more than 50 times faster than the equivalent number of CPU cores could achieve.

The major result this time is that ASUCA—a calculation code developed for operational use by Japan Meteorological Agency —was run with high performance by TSUBAME2.0, a GPU supercomputer.

GPU supercomputing is still in its infancy, and is only just emerging in many countries. It is a credit to Tokyo Institute of Technology, and to Japan as a whole, that its potential has now been demonstrated for such a universally relevant and important application as atmospheric simulations.

• T. Shimokawabe, T. Aoki, C. Muroi, J. Ishida, K. Kawano, T. Endo, A. Nukada, N. Maruyama & S. Matsuoka. "An 80-Fold Speedup, 15.0 TFlops, Full GPU
• Acceleration of Non-Hydrostatic Weather Model ASUCA Production Code,"
• Proceedings of the 2010 ACM/IEEE conference on Supercomputing (SC'10),
  New Orleans, LA, USA, Nov 2010.

• 

  TSUBAME 2.0

• 

  A Weather Prediction using ASUCA with horizontal 500m mesh