Using volunteered weather observations to explore urban and regional patterns in the Netherlands

Slides used during our presentation at the AGU Fall Meeting 2019 in San Francisco (USA), in the session "IN22A - Making Data Usable and Accessible: Gaining Insight from Citizen Science Applications". You can check the information of that session in the following link: https://agu.confex.com/agu/fm19/meetingapp.cgi/Session/88576

Transcript

1. Using volunteered weather observations to explore urban and regional weather patterns in the Netherlands Irene Garcia-Marti Marijn de Haij Jan-Willem Noteboom Gerard van der Schrier Cees de Valk AGU Fall Meeting 2019 IN22A - Making Data Usable and Accessible: Gaining Insight from Citizen Science Applications 10th December 2019
2. › Weather observations are crucial! › Spatial sparsity is a challenge for high-res weather forecasts › Increasing number of weather-related citizen science projects – WOW, Wunderground, Netatmo, Meteoclimatic Motivation 2
3. 1st September 2019: 1,400 million observations and 17K stations worldwide
4. › 2015: KNMI partner of WOW › Contributors: 400+ CWS › Data NL+BE: 3.7M obs/month › Devices: semi-professional – Manufacturers: Davis, Oregon scientific, Ventus, Alecto… – Expected “reasonable” quality of the observations WOW-NL 4 Province of Utrecht 94 CWS
5. › Quality not only related to device: – Good with respect to what? What variables are (not) properly monitored? – Local processes: radiation, shadowing, siting › Classical challenges of citizen science data: – Gaps in data – Noisy observations WOW-NL 5
6. WOW-NL 6 › Quality not only related to device: – Good with respect to what? What variables are (not) properly monitored? – Local processes: radiation, shadowing, siting › Classical challenges of citizen science data: – Discretization – Scale of the phenomena
7. 7 What is the quality of WOW-NL? R2 correlation between citizen and official weather stations
8. 8 What is the quality of WOW-NL?
9. Preprocessing WOW json csv 11.6M observations 65 features SVF Feature engineering
10. Quality control › Based on (Napoly, 2018) › Variable: temperature – Feasible to implement on WOW-NL – Levels have been compacted – Each of the 11.6M observations is labeled with a quality level M0: incorrect metadata M1: insufficient Z-score (presence outliers) M2: insufficient day/mon coverage M3: insufficient (Pearson) correlation M4: OK (Napoly et al., 2018) Development and Application of a Statistically-Based Quality Control for Crowdsourced Air Temperature Data Frontiers in Earth Science
11. Overview of the quality of WOW-NL (Each square represents 10K observations) M0: incorrect metadata (not shown) M1: insufficient Z-score (presence outliers) M2: insufficient day/mon coverage M3: insufficient (Pearson) correlation M4: OK
12. › Heat wave 27-07-2018 › Methodology: – Kriging interpolation: ▪ WOW-NL observations per hour ▪ Calibrated for this day ▪ No external drift – Visual comparison with HARMONIE ▪ Regional numerical model ▪ Provides forecast up to 48h in advance Exploring regional temperature 12
13. › Results: – WOW-NL captures the daily temperature cycle – Spatial patterns are different: ▪ Radiation: proximity to buildings ▪ Cooling: shadowing of trees ▪ Meaning: • Predicted weather in the column different to what happens at ground level • More work to reduce this gap Exploring regional temperature 13
14. › If good enough: – Open the door for new research: ▪ Fine-grained interpolated layers ▪ Nowcasting / hi-res weather ▪ Crowdsourced NWP – Governance level: ▪ Lower cost for administration ▪ Better weather forecast for underrepresented areas ▪ Bottom-up initiatives might work in developing regions Why the quality of citizen science weather data is important? 14 Imperfect data, but volume is difficult to ignore Big data problem! We are here
15. Questions? ☺ garciamarti@knmi.nl Thanks! 15