[ACCV22] Flare Transformer Solar Flare Prediction using Magnetograms and Sunspot Physical Features

Transcript

1. Flare Transformer: Solar Flare Prediction using
   Magnetograms and Sunspot Physical Features
   Kanta Kaneda1, Yuiga Wada1, Tsumugi Iida1, Naoto Nishizuka2,
   Yûki Kubo2, Komei Sugiura1
   1 Keio University, Japan
   2 National Institute of Information and Communications Technology, Japan
   

   View Slide

2. Introduction:
   Large solar flares can cause extensive damage
   - 2 -
   Solar Flares：Explosion occurring near sunspots on the solar surface
   ⇒ Monitored and forecasted by national agencies (USA : NOAA)
   If solar flare can be predicted in
   advance, damage can be minimized
   Disasters caused by Solar Flares
   1989 Massive blackout at Quebec
   2003 Damaged spacecraft HAYABUSA
   2022 Damaged SpaceX’s satellites
   NASA, https://svs.gsfc.nasa.gov/4491
   Estimated damage
   $160 billion at North America
   

   View Slide

3. Related Works:
   Existing solar flare prediction models are insufficient
   - 3 -
   Task Typical Models Overview
   Time-series
   Forecasting
   DeepAR
   [Salinias+, IJF20]
   Time-series forecasting model using an autoregressive RNN
   Informer
   [Zhou+, AAAI21]
   Prediction model using ProbSparse self-attention
   Solar Flare
   Forecasting
   [Park+, ApJ18] CNN-based model using full-disk images
   DeFN
   [Nishizuka+, ApJ18]
   Prediction model with sunspot physical features
   [Zhou+, AAAI21] [Park+, ApJ18] [Nishizuka+, ApJ18]
   

   View Slide

4. Problem Statement: Evaluation methods are
   important because of the imbalance in the class
   - 4 -
   ■ Solar Flare Prediction
   ■ Predicting the class of largest solar flare that will occur within
   24 hours from time t
   ■ Input
   1. Magnetograms
   ■ Magnetic images of the sun
   taken hourly
   2. Sunspot Physical Features
   ■ Physical features including
   time-series features extracted
   from solar images
   ■ Output
   ■ Solar flare classes (X, M, C, O)
   Class Frequency Damage
   X Low
   High
   Large
   Small
   M
   C
   O
   

   View Slide

5. Problem Statement: Solar Flare Prediction using
   Magnetograms and Sunspot Physical Features
   - 5 -
   Characteristics of Solar Flare Prediction Task
   1. It is important to predict X/M class solar
   flares that can cause significant damage
   2. X/M class solar flares occur rarely
   ■ Evaluate using standard metrics
   ■ Gandin–Murphy–Gerrity score (GMGS)
   ■ True skill statistics (TSS)
   ■ Brier skill score (BSS)
   1%
   8%
   32%
   59%
   X-class M-class
   C-class
   O-class
   Breakdown of solar flare classes
   that occurred from 2010 to 2017
   

   View Slide

6. Proposed Method: Flare Transformer
   - 6 -
   ■ Introduced Magnetogram Module
   which handles magnetograms
   ■ Sunspot Feature Module which
   handles physical features
   ■ Introduced transformer attention
   mechanism to both modules
   ■ Introduced GMGS loss and BSS
   loss to balance the major metrics
   Novelty
   

   View Slide

7. Proposed Method: Predicts flare classes from
   magnetograms and physical features
   - 7 -
   Input
   ■ Magnetograms and 90
   physical features from
   time to time
   Output
   ■ A 4 dimensional vector that
   denotes the predicted
   probabilities for each solar
   flare class
   

   View Slide

8. Proposed Method: Introduced MM and SFM to
   handle both magnetograms and physical features
   - 8 -
   Magnetogram Module /
   Sunspot Feature Module
   ■ Extract features from
   magnetograms/physical
   features
   ■ Model the temporal
   relationship by /
   transformer layers
   

   View Slide

9. Loss Function:
   Weighted sum of CE loss, GMGS loss and BSS loss
   - 9 -
   ■ Loss function: Trained to minimize
   Cross Entropy Loss BSS Loss GMGS Loss
   Cross Entropy Loss BSS Loss GMGS Loss
   BSS Loss GMGS Loss
   GMGS Loss
   Number of samples
   Number of classes
   Predicted probability
   Ground truth label
   Cross entropy between predicted labels
   and ground truth labels
   

   View Slide

10. Loss Function:
    Weighted sum of CE loss, GMGS loss and BSS loss
    - 10 -
    ■ Loss function: Trained to minimize
    Cross Entropy Loss BSS Loss GMGS Loss
    Cross Entropy Loss BSS Loss GMGS Loss
    BSS Loss GMGS Loss
    GMGS Loss
    Number of samples
    Number of classes
    Predicted probability
    Ground truth label
    BSS is a differentiable metric
    Introduce a loss function that maximizes BSS
    

    View Slide

11. Loss Function:
    Weighted sum of CE loss, GMGS loss and BSS loss
    - 11 -
    ■ Loss function: Trained to minimize
    Cross Entropy Loss BSS Loss GMGS Loss
    Cross Entropy Loss BSS Loss GMGS Loss
    BSS Loss GMGS Loss
    GMGS Loss
    Number of samples
    Number of classes
    Predicted probability
    Ground truth label
    Introduce weights that
    directly maximizes GMGS
    

    View Slide

12. Experiment Setting: Dataset containing
    magnetograms and physical features
    - 12 -
    ■ Total of 61315 samples from 2010-2017
    ■ Contains magnetograms and a set of 90
    physical features
    ■ Division based on time-series
    cross-validation [Tashman+, 00]
    Magnetogram
    Physical Features
    Training Set Test Set
    Period Num of Samples Period Num of Samples
    2010-2013 29247 2014 8127
    2010-2014 37374 2015 8155
    2010-2015 45529 2016 7795
    2010-2016 53324 2017 7991
    

    View Slide

13. Quantitative Results:
    Outperformed human experts in terms of GMGS and TSS
    - 13 -
    GMGS ↑ TSS≧M
    ↑ BSS≧M
    ↑
    DeFN [Nishizuka+, AsJ18] 0.38±0.14 0.41±0.15 -0.02±0.78
    DeFN-R [Nishizuka+, AsJ20] 0.30±0.06 0.28±0.16 0.04±0.98
    Proposed Method 0.50±0.06 0.53±0.11 0.08±0.97
    Human Experts
    [Kubo+, AsJ18][Murray, SW17]
    0.48 0.50 0.16
    ■ Our method outperformed the baseline methods (DeFN, DeFN-R) in
    terms of GMGS, BSS, and TSS
    +0.20
    +0.02 +0.03
    +0.25 +0.04
    ■ Flare Transformer gives better predictions than can be achieved by
    human experts in terms of GMGS and TSS
    

    View Slide

14. Ablation Studies: Scores improved by handling both
    magnetograms and physical features as input
    - 14 -
    ■ Ablation (a) : Removed physical features from input
    Scores improved by handling both magnetograms and
    physical features as input
    ■ Ablation (b) : Reduced a transformer layer in SFM
    ■ Ablation (c) : Added a transformer layer in MM
    GMGS ↑ TSS≧M
    ↑ BSS≧M
    ↑
    (a) w/o Ft-k+1:t 0.22±0.12 0.19±0.37 -1.77±0.23
    (b) NV = 1, Nf = 1 0.52±0.09 0.49±0.08 0.05±1.05
    (c) NV = 2, Nf = 2 0.56±0.07 0.55±0.12 0.01±0.97
    (d) NV = 1, Nf = 2 0.50±0.06 0.53±0.11 0.08±0.97
    Scores fluctuated
    slightly
    +0.28 +0.34 +1.85
    

    View Slide

15. ■ Investigate the impact on GMGS from
    confusion matrices by GMGSInfluence
    ■ The main bottleneck is the
    misprediction of X-class flares
    for M-class flares
    Error Analysis:
    Main bottleneck is the misprediction of X-class flares
    - 15 -
    (Observed Class➙Predicted Class)
    0
    0.02
    0.04
    0.06
    0.08
    0.1
    0.12
    0.14
    0.16
    X➙M M➙C C➙O M➙O X➙O
    Element from the confusion matrix
    Element from the GMGS score matrix
    

    View Slide

16. ■ We proposed the Flare Transformer, a
    solar flare prediction model which
    handles heterogenous data
    (magnetograms and physical features)
    ■ Flare Transformer gives better
    predictions than can be achieved by
    human experts in terms of GMGS and
    TSS (standard metrics in the field of
    solar flare prediction)
    Conclusion
    - 16 -
    

    View Slide