Deep learning-based super-resolution for downscaling climate data Carlos Gómez-Gonzalez Collaborators: Lluís Palma Garcia, Llorenç Lledó, Raül Marcos, Nube Gonzalez-Reviriego, Giulia Carella, Albert Soret and Kim Serradell BSC Doctoral Symposium 2021 [email protected] May 13th, 2021 

Cross-disciplinarity and data science Climate Science Computer vision AI engineering Statistical downscaling aims at improving the resolution of coarse climate data, without running very expensive high-resolution dynamical climate models Super-resolution solves a similar problem. Numerous DL-based approaches have been proposed in the last few years thanks to the current wave in AI The design of novel DL algorithms, and the development and deployment of such models on a supercomputing facility is far from trivial 

Cross-disciplinarity and data science Climate Science Computer vision AI engineering ● Problem definition (need) ● Background and baseline approaches ● Data sources identification ● Validation metrics ● (Re)framing the problem ● Cutting edge DL approaches for super-resolution tasks ● Smart testing and model design/tuning ● Development of robust and efficient code ● Reusability/reproducibility 

Objectives and data sources 

Objectives ● Statistical downscaling aims at learning empirical links between the large-scale and local-scale climate, i.e., a mapping from a low-resolution gridded variable to a higher-resolution grid ● We aim at improving the coarse spatial resolution of the SEAS5 ECMWF seasonal forecast of temperature over the north-east of the Iberian Peninsula (Catalunya) ● 20x scaling factor: taking SEAS5 data from its native 1º grid to a 0.05º resolution Curious fact: Downscaling (climate) == Upscaling (computer vision) ~100 km per gridpoint ~5 km per gridpoint 

Data ... Other ERA5 variables: humidity, geopotential, wind, etc Static fields: elevation and land-ocean mask Seasonal forecast SEAS5 tas 1º (1981-2018) Observational reference UERRA tas 0.05º (1979-2018) Observational reference ERA5 tas 0.25º (1979-2018) 

Experimental setup Historical observations Seasonal forecasts training dataset (32 years) holdout dataset (8 years) Train / hypertune downscaling methods Verify downscaling methods on unseen observations ● field RMSE ● spatial Correlation quality of the downscaling method on daily mean fields Downscale SEAS5 predictions: ● Nov start dates for DJF ● 1981-2017 ● member by member Perfect Prognosis: The models are trained with obs, but used later with seasonal forecasts hindcast @daily scale field RMSE Verify SEAS5 downscaled hindcast @seasonal scale ● EnsCorr ● RPSS quality of the downscaled SEAS5 DJF forecasts issued in November 

Deep learning-based methods 

Computer Vision for Earth Sciences ● Spatio-temporal processes ● Common structural prior: Gridded climate (or any scientific) data can be treated as arrays of pixels or images ● Tasks in CV that relate to a problem in ES: ○ Next frame video prediction (regression) ○ Super-resolution ○ Object recognition ○ Inpainting (gap filling) ○ Image to image translation (transfer functions) ● Approaches from CV can be adapted to the specific challenges in ES research 

CNNs in a nutshell 2D convolution using a kernel size of 3 (sliding shadow) with stride of 1 and padding 96 convolutional kernels of size 11×11×3 learned by the first convolutional layer of an image classification CNN. From Krizhevsky et al. 2012 Residual block, learning F(x) converges faster than learning H(x). Used in super-resolution in the EDSR model (Lim et al. 2017) Input Activation map (output) 

Taken from Wang, et al. 2020 ● Several super-resolution approaches have been proposed in the field of computer vision ● These ideas have inspired deep learning-based downscaling methods in climate science, e.g., Vandal et al. 2017, Leinonen et al. 2020, Stengel et al. 2020 Upscaling methods Model families Model architectures Transposed convolutions Pixel shuffle Meta upscale module 

DL-based methods ● Several deep neural network architectures were implemented and trained to learn the mapping between the coarse and fine spatial resolutions ● We compared: ○ single (20x) and progressive (4x + 5x) models as depicted on panels (a) and (b) ○ adversarial training, as conditional generative adversarial networks (CGANs), and non-adversarial supervised training ○ pre and post-upsampling strategies ● Deep residual networks (He et al. 2015) are used as a backbone model ○ w/o batch norm (Lim et al. 2017) ○ with a channel attention mechanism (Woo et al. 2018) 

● Panel (a): only one of the blocks “pre-upsampling” and “post-upsampling” is active for a given model depending on where the upscaling happens in the model ○ ResNet-INT uses pre-upsampling ○ ResNet-SPC uses post-upsampling ● The purely supervised models are trained with a mean absolute error loss (MAE) ● Panels (a) and (b) show the generator (“counterfeiter”) and discriminator (“policeman”) or our CGAN models ● Either the ResNet-SPC or ResNet-INT are used as generators ● Our CGAN models are trained as standard conditional GANs (Isola et al. 2016) 

DL4DS ● These algorithms were implemented in Python using the Tensorflow library ● The implementations were modularized in a package called DL4DS (Deep Learning for statistical DownScaling) along with the data loading and training procedures ● The models were trained on the POWER-CTE cluster ● DL4DS is capable of distributed training with Horovod) ● While the training time varies depending on the model complexity (from 30 min to 12 hours on a single V100 GPU), the inference takes only a few seconds and can be applied to arbitrary domain sizes 

Results and validation 

Visual comparison of SEAS5 downscaled products Ok so we obtain realistic or at least plausible images, but that is not enough... 

Holdout verification RMSE and correlation measured using the holdout UERRA dataset as a reference 

Downscaled SEAS5 verification Start dates: Nov Lead time: 1 month Valid period: DJF 

● AI and deep learning-based solutions show great promise for Earth Science tasks ● The DL-based downscaling techniques presented here are efficient at generating high-resolution gridded temperature fields ● In terms of RMSE and correlation, the conditional GAN with a ResNet-SPC generator outperforms the other approaches ● In terms of RPSS and ensemble correlation, all the models behave similarly (with the exception of a single CGAN ResNet-INT model) ○ It is important to notice that even a simple method such as analogs-KNN provides good results ○ improving the skill and correcting the bias of the seasonal prediction require fine-tuned loss functions and training procedures. This is something we are currently exploring ● The DL4DS python package with a clear API is planned for release and hopefully will be a useful tool for the community Final thoughts 

Gracias Acknowledgements: The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement H2020-MSCA-COFUND-2016-754433 and the EU H2020 Framework Programme under grant agreements n° GA 823988 (ESiWACE-2), GA 869575 (FOCUS-Africa) and GA 869565 (VitiGEOSS). [email protected]