Conference Report: 14th ECMWF Workshop on the Use of High Performance Computing in Meteorology

The European Centre for Medium-Range Weather Forecasts (ECMWF, Reading, UK) organized from November 1 to 5, 2010 the 14th Workshop on the Use of High Performance Computing (HPC) in Meteorology. The aim of this workshop is to bring together worldwide key players providing meteorological and climate services running numerical models. Some major HPC vendors also attended the conference. ECMWF is a European institution providing global weather forecasts for the next 10 days for all his member states. MeteoSwiss is a member and uses ECMWF forecast results to drive his high-resolution COSMO model for the Swiss forecasts. The workshop offered a unique opportunity to understand the needs of the weather and climate community and the resulting technological challenges for HPC.

Weather Forecast and Climate: Towards Higher Resolution

The general trend in the weather forecast and climate community is to improve the spatial resolution of the numerical models being used. According to Erland Källén, Director of Research at ECMWF, the horizontal resolution for the ECMWF global weather forecast model will go from 16 km today to 10 km by the year 2015 and 5 km around the year 2020. The spatial resolution of the forecasting system is critical for the forecast quality; the available computing power determines the resolution that can be achieved. In the last years it was possible to increase the forecast quality by roughly one day every 10 years (today 8.5 days of “usable” forecast). In this frame the scalability of the model on future HPC platforms is a major concern.

Another trend is the usage of Ensemble Prediction Systems (EPS) where the same forecast is repeated many times perturbing the initial conditions as well as the model physics and statistically evaluating the different results. This approach has the benefit to keep constant the scalability requirement of the code.

The different products based on numerical forecasts are impressive and can range from forecasts for cities, regions, flooding, hurricanes, extreme events, particle dispersion etc. Tim Pug of the Australian Bureau of Meteorology listed (the long list) of products forecasts of his institute and concluded that the resulting weather applications will require sustained teraflops and gigabyte I/O on HPC systems to meet the operational forecast timewindows.

Especially USA and Japan are investing large resources in Earth System Modelling. Richard Loft of the National Center for Atmospheric Research (NCAR, USA) reported on recent advancements in high-resolution climate modeling on the NCAR next generation HPC infrastructure to be hosted in a petascale facility dedicated to the Atmospheric Sciences. Hirofumi Tumita of the Japan Agency for Marine-Earth Science and Technology (JAMSTEC, Japan) presented the K-computer, the next HPC system with a peak performance of 10 PF that will be operational by mid 2012.

HPC Vendors: Already Preparing for Exascale Computing

As expected, the size of the HPC systems is increasing yearly and the vendors are expecting to reach the Exascale mark by 2019/20. Several centers will be running Petaflops systems already in 2011/12 deploying hundred of thousand of cores.

Don Grice of IBM Corporation argues that rethinking the way applications are structured will be required to deal with new limits. At this scale already moving data inside the HPC system will be a major issue. In his opinion a view of exascale machines as a memory system with embedded computing resources may be useful.

It is expected that new HPC systems will be hybrid using CPUs and coprocessor accelerators like GPUs. Researchers, like Mark Govett of the National Oceanographic and Atmospheric Agency (NOAA, USA) already reported about their experiences using GPUs. The initial results were more than promising, although it is not clear what accelerator architecture will have a future and how to manage source code for different environments. The compiler support for accelerators is at the beginning but first companies are already offering products. For example Dave Norton of The Portland Group presented the version 11 of PGI providing support for GPUs and for the AVX X64 vector extension.

Challenges for Numerical Weather Forecasts and HPC

The numerical weather community is realizing that their need for larger computations will not be satisfied by just increasing the size of the HPC systems being used. Many speakers were addressing different aspects where the existing codes are not able to scale up for different reasons. Different benchmarks demonstrated that most of the applications do not scale up to more than a few thousands of cores. Attempts to increase the speed of existing codes often increase the efficiency of only a few percents. The community is not ready to rewrite the code and to port it to new programming languages. Most scientists seem already to be overwhelmed by the complexity of the newest integrated developing environments. The hope is that the hardware and the compilers will still be able to fill the gap. Few others groups, like Yannick Trémolet (ECMWF, UK) are seriously investigating the possibility of at least re-engineering the code in order to improve maintenance, further developments, as well as general code scalability.

Additional computing resources and higher spatial resolution is connected with increased requirements for Input/Output. The technologies and methods currently used have already reached their limits. It is necessary to think again how the application will handle I/O. One alternative is to reduce the I/O (often re-computing will be less expensive than performing I/O) or to apply highly parallel technologies also for I/O. Luis Kornblueh of the Max Plank Institute, Germany reported about promising attempts to implement a highly parallel I/O library with GRIB2 output. Thus this will able to mitigate but not to solve the problem on the long term.

It is interesting to remark that almost all operational weather forecasting services are not using clusters but integrated solutions provided by the major HPC vendors.

Consequences for MeteoSwiss and CSCS

The presentation of MeteoSwiss and CSCS was concentrating on the fruitful cooperation between the two organizations and the plans of the HP2C initiativeto port part of the COSMO code to emerging platforms using accelerators. The proposed solution is based on the co-design approach where scientists, software engineers and hardware specialists are together designing a new application able to keep on with the increasing size of HPC systems. At this moment the COSMO consortium seems to be one of the few really being ready to rewrite at least part of the code. The new code is targeting to be able to support CPUs and different accelerator technologies in a transparent way. This should ensure a better acceptance by the COSMO community. The I/O is also addressed in an innovative manner by using concept already developed in the context of in-situ visualization as VisIt.

The echo to the overall approach of HP2C has been very positive and one could see that it conceals methods used by different participants.