[ACCV22] Flare Transformer Solar Flare Prediction using Magnetograms and Sunspot Physical Features

Transcript

1. Flare Transformer: Solar Flare Prediction using Magnetograms and Sunspot Physical Features Kanta Kaneda1, Yuiga Wada1, Tsumugi Iida1, Naoto Nishizuka2, Yûki Kubo2, Komei Sugiura1 1 Keio University, Japan 2 National Institute of Information and Communications Technology, Japan
2. Introduction: Large solar flares can cause extensive damage - 2 - Solar Flares：Explosion occurring near sunspots on the solar surface ⇒ Monitored and forecasted by national agencies (USA : NOAA) If solar flare can be predicted in advance, damage can be minimized Disasters caused by Solar Flares 1989 Massive blackout at Quebec 2003 Damaged spacecraft HAYABUSA 2022 Damaged SpaceX’s satellites NASA, https://svs.gsfc.nasa.gov/4491 Estimated damage $160 billion at North America
3. Related Works: Existing solar flare prediction models are insufficient - 3 - Task Typical Models Overview Time-series Forecasting DeepAR [Salinias+, IJF20] Time-series forecasting model using an autoregressive RNN Informer [Zhou+, AAAI21] Prediction model using ProbSparse self-attention Solar Flare Forecasting [Park+, ApJ18] CNN-based model using full-disk images DeFN [Nishizuka+, ApJ18] Prediction model with sunspot physical features [Zhou+, AAAI21] [Park+, ApJ18] [Nishizuka+, ApJ18]
4. Problem Statement: Evaluation methods are important because of the imbalance in the class - 4 - ▪ Solar Flare Prediction ▪ Predicting the class of largest solar flare that will occur within 24 hours from time t ▪ Input 1. Magnetograms ▪ Magnetic images of the sun taken hourly 2. Sunspot Physical Features ▪ Physical features including time-series features extracted from solar images ▪ Output ▪ Solar flare classes (X, M, C, O) Class Frequency Damage X Low High Large Small M C O
5. Problem Statement: Solar Flare Prediction using Magnetograms and Sunspot Physical Features - 5 - Characteristics of Solar Flare Prediction Task 1. It is important to predict X/M class solar flares that can cause significant damage 2. X/M class solar flares occur rarely ▪ Evaluate using standard metrics ▪ Gandin–Murphy–Gerrity score (GMGS) ▪ True skill statistics (TSS) ▪ Brier skill score (BSS) 1% 8% 32% 59% X-class M-class C-class O-class Breakdown of solar flare classes that occurred from 2010 to 2017
6. Proposed Method: Flare Transformer - 6 - ▪ Introduced Magnetogram Module which handles magnetograms ▪ Sunspot Feature Module which handles physical features ▪ Introduced transformer attention mechanism to both modules ▪ Introduced GMGS loss and BSS loss to balance the major metrics Novelty
7. Proposed Method: Predicts flare classes from magnetograms and physical features - 7 - Input ▪ Magnetograms and 90 physical features from time to time Output ▪ A 4 dimensional vector that denotes the predicted probabilities for each solar flare class
8. Proposed Method: Introduced MM and SFM to handle both magnetograms and physical features - 8 - Magnetogram Module / Sunspot Feature Module ▪ Extract features from magnetograms/physical features ▪ Model the temporal relationship by / transformer layers
9. Loss Function: Weighted sum of CE loss, GMGS loss and BSS loss - 9 - ▪ Loss function: Trained to minimize Cross Entropy Loss BSS Loss GMGS Loss Cross Entropy Loss BSS Loss GMGS Loss BSS Loss GMGS Loss GMGS Loss Number of samples Number of classes Predicted probability Ground truth label Cross entropy between predicted labels and ground truth labels
10. Loss Function: Weighted sum of CE loss, GMGS loss and BSS loss - 10 - ▪ Loss function: Trained to minimize Cross Entropy Loss BSS Loss GMGS Loss Cross Entropy Loss BSS Loss GMGS Loss BSS Loss GMGS Loss GMGS Loss Number of samples Number of classes Predicted probability Ground truth label BSS is a differentiable metric Introduce a loss function that maximizes BSS
11. Loss Function: Weighted sum of CE loss, GMGS loss and BSS loss - 11 - ▪ Loss function: Trained to minimize Cross Entropy Loss BSS Loss GMGS Loss Cross Entropy Loss BSS Loss GMGS Loss BSS Loss GMGS Loss GMGS Loss Number of samples Number of classes Predicted probability Ground truth label Introduce weights that directly maximizes GMGS
12. Experiment Setting: Dataset containing magnetograms and physical features - 12 - ▪ Total of 61315 samples from 2010-2017 ▪ Contains magnetograms and a set of 90 physical features ▪ Division based on time-series cross-validation [Tashman+, 00] Magnetogram Physical Features Training Set Test Set Period Num of Samples Period Num of Samples 2010-2013 29247 2014 8127 2010-2014 37374 2015 8155 2010-2015 45529 2016 7795 2010-2016 53324 2017 7991
13. Quantitative Results: Outperformed human experts in terms of GMGS and TSS - 13 - GMGS ↑ TSS≧M ↑ BSS≧M ↑ DeFN [Nishizuka+, AsJ18] 0.38±0.14 0.41±0.15 -0.02±0.78 DeFN-R [Nishizuka+, AsJ20] 0.30±0.06 0.28±0.16 0.04±0.98 Proposed Method 0.50±0.06 0.53±0.11 0.08±0.97 Human Experts [Kubo+, AsJ18][Murray, SW17] 0.48 0.50 0.16 ▪ Our method outperformed the baseline methods (DeFN, DeFN-R) in terms of GMGS, BSS, and TSS +0.20 +0.02 +0.03 +0.25 +0.04 ▪ Flare Transformer gives better predictions than can be achieved by human experts in terms of GMGS and TSS
14. Ablation Studies: Scores improved by handling both magnetograms and physical features as input - 14 - ▪ Ablation (a) : Removed physical features from input Scores improved by handling both magnetograms and physical features as input ▪ Ablation (b) : Reduced a transformer layer in SFM ▪ Ablation (c) : Added a transformer layer in MM GMGS ↑ TSS≧M ↑ BSS≧M ↑ (a) w/o Ft-k+1:t 0.22±0.12 0.19±0.37 -1.77±0.23 (b) NV = 1, Nf = 1 0.52±0.09 0.49±0.08 0.05±1.05 (c) NV = 2, Nf = 2 0.56±0.07 0.55±0.12 0.01±0.97 (d) NV = 1, Nf = 2 0.50±0.06 0.53±0.11 0.08±0.97 Scores fluctuated slightly +0.28 +0.34 +1.85
15. ▪ Investigate the impact on GMGS from confusion matrices by GMGSInfluence ▪ The main bottleneck is the misprediction of X-class flares for M-class flares Error Analysis: Main bottleneck is the misprediction of X-class flares - 15 - (Observed Class➙Predicted Class) 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 X➙M M➙C C➙O M➙O X➙O Element from the confusion matrix Element from the GMGS score matrix
16. ▪ We proposed the Flare Transformer, a solar flare prediction model which handles heterogenous data (magnetograms and physical features) ▪ Flare Transformer gives better predictions than can be achieved by human experts in terms of GMGS and TSS (standard metrics in the field of solar flare prediction) Conclusion - 16 -