blasé: An interpretable transfer learning approach to cool star échelle spectroscopy

Comparison of échelle spectra to synthetic models has become a computational statistics challenge, with over ten thousand individual spectral lines affecting a typical cool star échelle spectrum. Telluric artifacts, imperfect line lists, inexact continuum placement, and inflexible models frustrate the scientific promise of these information-rich datasets. Here we debut a scalable machine-learning framework "blasé" that addresses these challenges. The semi-empirical approach can be viewed as "transfer learning"--first pre-training models on noise-free precomputed synthetic spectral models, then learning the corrections to line depths and widths from noisy whole-spectrum fitting. The auto-differentiable model employs back-propagation, the fundamental algorithm empowering modern Neural Networks. Here, however, the 40,000+ parameters symbolize physically interpretable line profile properties (amplitude, width, location, shape) plus RV and vsini, rather than difficult-to-interpret neural network "weights". This hybrid data-/model- driven framework allows joint modeling of stellar and telluric lines simultaneously, a potentially transformative step forwards for pesky telluric mitigation in the near-infrared. Blasé also acts as a deconvolution tool. It is suitable for Doppler Imaging scenarios with longitudinally symmetric surface features like bands or polar spots, which evade detection in differential techniques. Blasé can also create super-resolution semi-empirical templates useful for critically evaluating atomic and molecular line lists. Its sparse-matrix architecture and GPU-acceleration make blasé fast, with end-to-end training of 50+ échelle orders in under 1 minute. The open-source PyTorch-based code (blase.readthedocs.io) includes tutorials and API documentation. We show how the tool fits into the existing Python spectroscopy ecosystem, demonstrate a range of astrophysical applications, and discuss limitations and future extensions.

Transcript

1. blasé
   
   
   A transfer learning approach to échelle spectra
   Michael Gully-Santiago
   
   
   The University of Texas at Austin
   Machine Learning for Cool Stars Splinter Session
   
   
   Cool Stars 21
   
   
   July 5, 2022
   

   View Slide

2. blasé
   
   
   is
   
   
   a general purpose experimentation framework
   
   
   aimed primary at
   
   
   high bandwidth, high-resolution vis/near-IR spectra
   X-shooter CRIRES+ APOGEE IGRINS HPF UVES iSHELL NEID
   HARPS MAROON-X CARMENES VELOCE Keck NIRSPEC HRS
   etc…
   

   View Slide

3. blasé
   
   
   is
   
   
   a general purpose experimentation framework
   
   
   aimed primary at
   
   
   high bandwidth, high-resolution vis/near-IR spectra
   IGRINS HPF
   (this talk)
   Park et al. 2014 Mahadevan 2012
   

   View Slide

4. blasé
   
   
   

   View Slide

5. The problem with échelle spectra:
   The mere act of comparing an échelle spectrum to a model
   becomes a computational statistics challenge.
   • Voluminous data
   
   
   • Imperfect models
   
   
   • Confounding factors (tellurics)
   
   
   +
   
   
   • In
   f
   lexible models (~4 parameters in a typical grid)
   
   
   • Expensive models
   

   View Slide

6. Stellar spectral models are imperfect.
   Missing lines; under-/over- predicted depths, widths, shapes, locations.
   λ (Å)
   

   View Slide

7. The model and data are voluminous.
   A typical HPF/IGRINS spectrum of a K/M dwarf contains >10,000 stellar lines.
   

   View Slide

8. (In some places worse than others.)
   Telluric absorption confounds the stellar spectrum.
   

   View Slide

9. Four modern solutions to (aspects of) these problems:
   github.com/Star
   f
   ish-develop/Star
   f
   ish
   github.com/HajimeKawahara/exojax
   Czekala et al. 2015
   Starfish wobble
   Bedell et al. 2019
   github.com/megbedell/wobble
   exojax
   Kawahara et al. 2022
   FAL
   Cargile et al. (in prep)
   github.com/pacargile/FAL
   (See also ThePayne by Phill Cargile)
   

   View Slide

10. wobble
    Bedell et al. 2019
    github.com/megbedell/wobble
    github.com/HajimeKawahara/exojax
    exojax
    Kawahara et al. 2022
    

    View Slide

11. wobble
    Bedell et al. 2019
    github.com/megbedell/wobble
    github.com/HajimeKawahara/exojax
    exojax
    Kawahara et al. 2022
    more data-driven
    more model-driven
    more parameters (~105)
    fewer parameters (~15)
    more precise
    better out-of-band prediction accuracy
    

    View Slide

12. wobble
    Bedell et al. 2019
    github.com/megbedell/wobble
    github.com/HajimeKawahara/exojax
    exojax
    Kawahara et al. 2022
    more data-driven
    more model-driven “Hybrid”
    github.com/gully/blase
    blasé
    Gully-Santiago & Morley (in prep)
    

    View Slide

13. wobble
    Bedell et al. 2019
    github.com/megbedell/wobble
    github.com/HajimeKawahara/exojax
    exojax
    Kawahara et al. 2022
    github.com/gully/blase
    blasé
    Gully-Santiago & Morley (in prep)
    underlying machine learning frameworks:
    

    View Slide

14. underlying machine learning frameworks:
    machine learning frameworks provide
    
    
    GPU acceleration and automatic differentiation*
    
    
    For a tutorial of autodi
    ff
    , Jacobians, and Hessians applied to spectroscopy, see:
    
    
    github.com/gully/TgiF
    *c.f. Gunes-Baydin et al. 2015 for a review ArXiV: 1502.05767
    

    View Slide

15. Key idea
    A typical echelle spectrum has:
    Nlines
    }
    (
    𝛌
    c
    , A,
    𝛔
    ,
    𝛄
    )
    Line center
    
    
    Amplitude
    
    
    Gaussian width
    
    
    Lorentzian width
    Four parameters for all 10,000 spectral lines
    

    View Slide

16. Key idea
    A typical echelle spectrum has:
    Nlines
    }
    (
    𝛌
    c
    , A,
    𝛔
    ,
    𝛄
    )
    Line center
    
    
    Amplitude
    
    
    Gaussian width
    
    
    Lorentzian width
    Four parameters for all 10,000 spectral lines
    blasé
    f
    its all 40,000 parameters simultaneously with autodi
    ff
    .
    = 40,000 total parameters
    

    View Slide

17. Step 0
    Pick closest stellar model template to your target of interest:
    Te = 4700 K logg = 4.5 [Fe/H] = 0
    

    View Slide

18. Step 1
    Place a coarsely-tuned Voigt pro
    f
    ile at the location of all lines.
    

    View Slide

19. Step 1
    Optimize all 40,000 line parameters simultaneously.
    

    View Slide

20. Step 0.5 (optional)
    Pick closest telluric model template to your time of observation:
    Tobs = 17° C Humidity = 40%
    Gullikson et al. 2014: TelFit
    

    View Slide

21. Step 1.5 (optional)
    Place a coarsely-tuned Voigt pro
    f
    ile at the location of all lines.
    Gullikson et al. 2014: TelFit
    

    View Slide

22. Step 1.5 (optional)
    Optimize all 12,000 telluric line parameters simultaneously.
    Gullikson et al. 2014: TelFit
    

    View Slide

23. Clone the spectra
    Step 1
    Apply vsini & RV
    Step 2
    Step 0
    Choose
    
    
    Templates
    Multiply by telluric
    Step 3
    Instrument convolve
    
    
    & resample
    Step 4
    

    View Slide

24. Clone the spectra
    Step 1
    Apply vsini & RV
    Step 2
    Step 0
    Choose
    
    
    Templates
    Multiply by telluric
    Step 3
    Instrument convolve
    
    
    & resample
    Step 4
    Compare to data
    Step 5
    RV, vsini, and all stellar and telluric line
    properties are tuned simultaneously.
    

    View Slide

25. Before
    λ (Å)
    

    View Slide

26. λ (Å)
    After
    

    View Slide

27. blasé learns a super-resolution template.
    before
    λ (Å)
    

    View Slide

28. blasé learns a super-resolution template.
    after
    λ (Å)
    

    View Slide

29. 56 s
    End-to-end HPF whole-spectrum
    f
    itting
    on NVIDIA RTX 2070 GPU.
    GPUs, sparse matrices, and approximations yield huge speedups.
    blasé is fast.
    End-to-end HPF whole-spectrum
    f
    itting
    on M1 MacBook Air (CPU only).
    1h15m
    

    View Slide

30. Key idea: Sparsity
    We use sparse matrices* in PyTorch to achieve 100x speedups.
    *c.f. Saad 2003
    

    View Slide

31. Main challenge: Regularization
    We face a bias/variance tradeo
    ff
    .
    c.f. Bedell et al. 2019
    In
    fi
    nite regularization
    No regularization
    Laissez-faire:
    
    
    Surrealist over
    fi
    tting
    No learning:
    
    
    recover the input template
    

    View Slide

32. Main challenge: Regularization
    We face a bias/variance tradeo
    ff
    .
    c.f. Bedell et al. 2019
    In
    fi
    nite regularization
    No regularization
    Laissez-faire:
    
    
    Surrealist over
    fi
    tting
    No learning:
    
    
    recover the input template
    “Sweet spot”
    Regularization encodes our intuition:
    
    
    Line properties should match PHOENIX unless
    there’s enough data to say otherwise
    

    View Slide

33. What’s possible?
    blasé serves as a foundation for many new strategies.
    1. Doppler Imaging with imperfect templates (Luger et al. 2021)
    
    
    • Zonal bands (Cross
    f
    ield et al. 2014)
    
    
    • Polar spots (Roettenbacher et al. 2017)
    
    
    2. Starspot spectral decomposition (Gully-Santiago et al. 2017)
    
    
    3. Spectroscopic binary joint modeling (Czekala et al. 2017)
    
    
    4. Automatic equivalent width tabulation
    
    
    5. Discover “true” lineshapes with hidden neural network layers
    
    
    6. Spectral calibration across the grid dimensions (c.f. previous talk by Bello-Garcia)
    
    
    7. Brown Dwarfs with non-parametric pseudo-continuum (e.g. GPyTorch)
    
    
    8. Adapt the templates to magnetic
    f
    ields / Zeeman effect
    
    
    9. Provide quantitative feedback to modelers at native resolution
    

    View Slide

34. blasé is open source.
    We have tutorials, deep dives, FAQs, and API documentation.
    blase.readthedocs.io
    

    View Slide

35. blasé is open source.
    We have tutorials, deep dives, FAQs, and API documentation.
    blase.readthedocs.io
    

    View Slide

36. blasé is user-friendly.
    It serves as a gentle introduction to key concepts in ML for newcomers.
    blase.readthedocs.io
    

    View Slide

37. blasé is a work in progress, we invite your help:
    Engage with us on GitHub Issues and more.
    github.com/gully/blase
    

    View Slide

38. Typical spectral analysis work
    f
    low with blasé
    blasé
    2D echellogram
    reduction pipeline
    1D spectrum
    
    
    post-processing
    Initial spectral
    template selection
    Spectral
    f
    itting
    or wobble
    
    
    or starfish
    
    
    or …
    muler
    🆕 gollum
    🆕
    Gully-Santiago et al. 2022; JOSS
    github.com/OttoStruve/muler
    github.com/BrownDwarf/gollum
    

    View Slide

39. blasé
    • blasé offers an interpretable model for echelle spectra, including tellurics.
    
    
    • It leverages the key enabling technology autodiff to achieve excellent precision.
    
    
    • By using Sparse Matrices and GPUs we achieve excellent computational performance.
    
    
    • It has tons of conceivable extensions.
    
    
    • We have also made ancillary tools muler and gollum for lowering the barrier to entry.
    
    
    • Altogether these codes form an easy-to-use ecosystem for end-to-end spectral analysis.
    

    View Slide

40. The end.
    

    View Slide