ChaosBench

We propose ChaosBench, a large-scale, multi-channel, physics-based benchmark for subseasonal-to-seasonal (S2S) climate prediction. It is framed as a high-dimensional video regression task that consists of 45-year, 60-channel observations for validating physics-based and data-driven models, and training the latter. Physics-based forecasts are generated from 4 national weather agencies with 44-day lead-time and serve as baselines to data-driven forecasts. Our benchmark is one of the first to incorporate physics-based metrics to ensure physically-consistent and explainable models. We establish two tasks: full and sparse dynamics prediction.

🔗: https://leap-stc.github.io/ChaosBench/

📚: https://arxiv.org/abs/2402.00712

Getting Started

Step 1: Clone the ChaosBench Github repository

Step 2: Install package dependencies

cd ChaosBench
pip install -r requirements.txt

Step 3: Initialize the data space by running

cd data/
wget https://huggingface.co/datasets/LEAP/ChaosBench/resolve/main/process.sh
chmod +x process.sh

Step 5: Download the data

# NOTE: you can also run each line one at a time to retrieve individual dataset

./process.sh era5            # Required: For input ERA5 data
./process.sh climatology     # Required: For climatology
./process.sh ukmo            # Optional: For simulation from UKMO
./process.sh ncep            # Optional: For simulation from NCEP
./process.sh cma             # Optional: For simulation from CMA
./process.sh ecmwf           # Optional: For simulation from ECMWF