Recommended Practices for Data Preprocessing in AI-Driven Climate Prediction
This article establishes protocols and recommended practices for preprocessing input data for AI/ML models used in climate prediction across subseasonal-to-decadal time scales. It covers three main aims: educating researchers on the effects of data preprocessing, providing best practices for handling climate data (including creating standardized anomalies, dealing with nonstationarity, spatiotemporal correlations, and extreme values), and empowering end users to evaluate model design. Two case studies demonstrate how different preprocessing techniques can produce different predictions from the same model, highlighting the importance of proper data preparation for robust, transparent, and trustworthy AI-driven climate predictions.
Key quotes
The skill and confidence in the forecasts produced by data-driven models are directly influenced by the quality of the datasets and how they are treated during model development, yielding the colloquialism, 'garbage in, garbage out.'
Implementing the recommended practices set forth in this article will enhance the robustness and transparency of AI/ML in climate prediction studies.
We offer several recommended steps to properly preprocess input data for AI models used for climate predictions (i.e., time scale ranging from few weeks to many years).
Case studies will illustrate how using different preprocessing techniques can produce different predictions from the same model, which can create confusion and decrease confidence in the overall process.
Following these recommendations will help make such studies more transparent, reproducible, and trustworthy.
From the article
You might also wanna read
How reanalysis became essential training data for AI weather forecasting
This article explores how reanalysis — the reconstruction of past weather and climate data by combining observations with models — has becom
ecmwf.int·6d agoMaster Data Preprocessing Steps for AI Success
Autodata: Using AI agents as data scientists to generate high-quality synthetic training data
This paper introduces Autodata, a method that uses AI agents as data scientists to create high-quality synthetic training and evaluation dat
arxiv.org·13d agoAutodata: Using AI agents as data scientists to generate high-quality synthetic training data
This paper introduces Autodata, a method that uses AI agents as data scientists to create high-quality synthetic training and evaluation dat
arxiv.org·13d ago
Autodata: Using AI agents as data scientists to generate high-quality synthetic training data
This paper introduces Autodata, a method that uses AI agents as data scientists to create high-quality synthetic training and evaluation dat
arxiv.org·12d agoAutodata: Using AI agents as data scientists to generate high-quality synthetic training data
This paper introduces Autodata, a method that uses AI agents as data scientists to create high-quality synthetic training and evaluation dat
arxiv.org·12d ago
I have just published a new research paper titled: “AI Driven Urban Heat Equity Atlas Integrating Machine Learning for Climate Risk Assessment.” This research applies artificial intelligence and machi
Data Scarcity as the Emerging Bottleneck in AI Scaling and Intelligence Development
The article discusses the asymmetry between compute and data growth in AI development, arguing that while compute capacity grows rapidly, da
Comments
Sign in to join the conversation.
No comments yet. Be the first.