NASA and IBM have launched a new AI foundation model for a variety of weather and climate use cases, available in open source to the scientific, developer and business communities. Developed with contributions from Oak Ridge National Laboratory, the model offers a flexible, scalable way to address a variety of challenges related to short-term weather as well as long-term climate projection.
Open-source AI in meteorology
Because of its design and training regime, the weather and climate foundation model can tackle more applications than existing weather AI models, as outlined in a paper recently published on arXiv, “Prithvi WxC: Foundation Model for Weather and Climate”. Potential applications include creating targeted forecasts based on local observations, detecting and predicting severe weather patterns, improving the spatial resolution of global climate simulations, and improving how physical processes are represented in numerical weather and climate models. In one experiment in the paper, the foundation model accurately reconstructed global surface temperatures from a random sample of only 5% original data, suggesting a broader application to problems in data assimilation.
This model was pre-trained on 40 years of Earth observation data from NASA’s Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2). As a foundation model, it has a unique architecture that enables it to be fine-tuned to global, regional and local scales. This flexibility makes it suited for a range of weather studies.
The foundation model is available for download on machine learning community platform Hugging Face, along with two fine-tuned versions of the model that tackle specific scientific and industry-relevant applications. These are climate and weather data downscaling and gravity wave parameterization.
“Advancing NASA’s Earth science for the benefit of humanity means delivering actionable science in ways that are useful to people, organizations and communities. The rapid changes we’re witnessing on our home planet demand this strategy to meet the urgency of the moment,” said Karen St Germain, director of the Earth Science Division of NASA’s Science Mission Directorate. “The NASA foundation model will help us produce a tool that people can use: weather, seasonal and climate projections to help inform decisions on how to prepare, respond and mitigate.”
“This space has seen the emergence of large AI models that focus on a fixed data set and single use case – primarily forecasting. We have designed our weather and climate foundation model to go beyond such limitations so that it can be tuned to a variety of inputs and uses,” said Juan Bernabe-Moreno, director of IBM Research Europe (UK and Ireland) and IBM’s accelerated discovery lead for climate and sustainability.
“For example, the model can run both on the entire Earth as well as in a local context. With such flexibility on the technology side, this model is well-suited to help us understand meteorological phenomena such as hurricanes or atmospheric rivers, reason about future potential climate risks by increasing the resolution of climate models, and finally inform our understanding of imminent severe weather events.”
Weather forecasting use cases
“As a premier research institution and computing facility, we’re focused on supporting teams to make research breakthroughs across many areas of science,” said Arjun Shankar, director of the National Center for Computational Sciences at Oak Ridge National Laboratory. “Our collaboration with IBM and NASA to support the creation of the Prithvi weather and climate foundation model was a key part of our goal to bring advanced computing and data to problems of national importance, in this case, for weather and climate applications, which need continued computational science and model skill improvements to be impactful.”
IBM has already collaborated with Environment and Climate Change Canada (ECCC) with a view to testing the flexibility of the model with additional weather forecasting use cases. With the model, ECCC is exploring very short-term precipitation forecasts using a technique called precipitation nowcasting that ingests real-time radar data as input. The team is also testing the downscaling approach from global model forecasts at 15km- to km-scale resolution.
This weather and climate model is part of a larger collaboration between IBM Research and NASA to use AI technology to explore our planet and joins the Prithvi family of AI foundation models. In 2023, IBM and NASA made the Prithvi geospatial AI foundation model the largest open-source geospatial AI model available on Hugging Face. This geospatial foundation model has since been used by governments, companies and public institutions to examine changes in disaster patterns, biodiversity, land use and other geophysical processes. The foundation model and the gravity wave parameterization model can be accessed through the NASA-IBM Hugging Face page and the downscaling model can be accessed through the IBM Granite Hugging Face page.
In related news, researchers at the Penn State Institute for Computational and Data Sciences (ICDS) recently launched a research project to better understand and predict extreme weather events on both Earth and Mars, using computational tools and insights from traditional meteorology and funded by a three-year, US$973,396 grant from the US National Science Foundation (NSF). Click here to read the full story.