Unlocking the Universe with Python and LSST

Unlocking the Universe with Python and LSST

Invited talk from the 2013 RuPy conference, Budapest, Hungary.

56c4053438af8e8b90d6f53cbb7573be?s=128

Jake VanderPlas

October 13, 2013
Tweet

Transcript

  1. 1.

    Unlocking  the  Universe  with   Python  and  LSST   Jake

     VanderPlas   RuPy,  Oct  11th  2013  
  2. 2.

    Then:  Edwin  Hubble  at   Mount  Wilson  Observatory   1990s

     -­‐  Now:  the  Hubble  Space   Telescope  in  Low  Earth  Orbit…  
  3. 3.
  4. 5.

    Large   SynopQc   Survey   Telescope   LSST in

    a sentence: An optical/near-IR multi-color survey of half the sky, based on ~1000 visits over a 10-year period
  5. 6.
  6. 7.

    8.4  meter  primary  mirror   3,000  Megapixel  CCD  camera  

    Field  of  View:  9.5  deg2   Largest  digital  camera  in  the   world!  
  7. 8.

    LSST:  a    10-­‐year,  full-­‐sky  digital  color  movie   -­‐ 

    Full  southern  sky  every  3-­‐4  nights   -­‐  30,000  GB/night  data  stream   -­‐  Real-­‐Qme  transient  alerts   -­‐  Final  Catalog:  100s  of  Petabytes  
  8. 9.

    •  Adler  Planetarium   •  Argonne  NaQonal  Laboratory   • 

    Brookhaven  NaQonal  Laboratory  (BNL)     •  California  InsQtute  of  Technology   •  Carnegie  Mellon  University   •  Chile   •  Cornell  University   •  Drexel  University   •  Fermi  NaQonal  Accelerator  Laboratory     •  George  Mason  University   •  Google,  Inc.   •  Harvard-­‐Smithsonian  Center  for  Astrophysics   •  InsQtut  de  Physique  Nucleaire  et  de  Physique   des  ParQcules  (IN2P3)   •  Johns  Hopkins  University   •  Kavli  InsQtute  for  ParQcle  Astrophysics  and   Cosmology  (KIPAC)  -­‐  Stanford  University   •  Las  Cumbres  Observatory  Global  Telescope   Network,  Inc.   •  Lawrence  Livermore  NaQonal  Laboratory  (LLNL)   •  Los  Alamos  NaQonal  Laboratory  (LANL)   •  NaQonal  OpQcal  Astronomy  Observatory   •  NaQonal  Radio  Astronomy  Observatory   •  Northwestern  University   •  Princeton  University   •  Purdue  University   •  Research  CorporaQon  for  Science  Advancement   •  Rutgers  University   •  SLAC  NaQonal  Accelerator  Laboratory   •  Space  Telescope  Science  InsQtute   •  Texas  A  &  M  University   •  The  InsQtute  of  Physics  of  the  Academy  of   Sciences  of  the  Czech  Republic   •  The  Pennsylvania  State  University   •  The  University  of  Arizona   •  University  of  California  at  Davis   •  University  of  California  at  Irvine   •  University  of  Illinois  at  Urbana-­‐Champaign   •  University  of  Michigan   •  University  of  Pennsylvania   •  University  of  Piesburgh   •  University  of  Washington   •  Vanderbilt  and  Fisk  UniversiQes   LSST  Ins(tu(onal  Members:  
  9. 10.

    •  Adler  Planetarium   •  Argonne  NaQonal  Laboratory   • 

    Brookhaven  NaQonal  Laboratory  (BNL)     •  California  InsQtute  of  Technology   •  Carnegie  Mellon  University   •  Chile   •  Cornell  University   •  Drexel  University   •  Fermi  NaQonal  Accelerator  Laboratory     •  George  Mason  University   •  Google,  Inc.   •  Harvard-­‐Smithsonian  Center  for  Astrophysics   •  InsQtut  de  Physique  Nucleaire  et  de  Physique   des  ParQcules  (IN2P3)   •  Johns  Hopkins  University   •  Kavli  InsQtute  for  ParQcle  Astrophysics  and   Cosmology  (KIPAC)  -­‐  Stanford  University   •  Las  Cumbres  Observatory  Global  Telescope   Network,  Inc.   •  Lawrence  Livermore  NaQonal  Laboratory  (LLNL)   •  Los  Alamos  NaQonal  Laboratory  (LANL)   •  NaQonal  OpQcal  Astronomy  Observatory   •  NaQonal  Radio  Astronomy  Observatory   •  Northwestern  University   •  Princeton  University   •  Purdue  University   •  Research  CorporaQon  for  Science  Advancement   •  Rutgers  University   •  SLAC  NaQonal  Accelerator  Laboratory   •  Space  Telescope  Science  InsQtute   •  Texas  A  &  M  University   •  The  InsQtute  of  Physics  of  the  Academy  of   Sciences  of  the  Czech  Republic   •  The  Pennsylvania  State  University   •  The  University  of  Arizona   •  University  of  California  at  Davis   •  University  of  California  at  Irvine   •  University  of  Illinois  at  Urbana-­‐Champaign   •  University  of  Michigan   •  University  of  Pennsylvania   •  University  of  Piesburgh   •  University  of  Washington   •  Vanderbilt  and  Fisk  UniversiQes   LSST  Ins(tu(onal  Members:  
  10. 11.

    •  Adler  Planetarium   •  Argonne  NaQonal  Laboratory   • 

    Brookhaven  NaQonal  Laboratory  (BNL)     •  California  InsQtute  of  Technology   •  Carnegie  Mellon  University   •  Chile   •  Cornell  University   •  Drexel  University   •  Fermi  NaQonal  Accelerator  Laboratory     •  George  Mason  University   •  Google,  Inc.   •  Harvard-­‐Smithsonian  Center  for  Astrophysics   •  InsQtut  de  Physique  Nucleaire  et  de  Physique   des  ParQcules  (IN2P3)   •  Johns  Hopkins  University   •  Kavli  InsQtute  for  ParQcle  Astrophysics  and   Cosmology  (KIPAC)  -­‐  Stanford  University   •  Las  Cumbres  Observatory  Global  Telescope   Network,  Inc.   •  Lawrence  Livermore  NaQonal  Laboratory  (LLNL)   •  Los  Alamos  NaQonal  Laboratory  (LANL)   •  NaQonal  OpQcal  Astronomy  Observatory   •  NaQonal  Radio  Astronomy  Observatory   •  Northwestern  University   •  Princeton  University   •  Purdue  University   •  Research  CorporaQon  for  Science  Advancement   •  Rutgers  University   •  SLAC  NaQonal  Accelerator  Laboratory   •  Space  Telescope  Science  InsQtute   •  Texas  A  &  M  University   •  The  InsQtute  of  Physics  of  the  Academy  of   Sciences  of  the  Czech  Republic   •  The  Pennsylvania  State  University   •  The  University  of  Arizona   •  University  of  California  at  Davis   •  University  of  California  at  Irvine   •  University  of  Illinois  at  Urbana-­‐Champaign   •  University  of  Michigan   •  University  of  Pennsylvania   •  University  of  Piesburgh   •  University  of  Washington   •  Vanderbilt  and  Fisk  UniversiQes   LSST  Ins(tu(onal  Members:  
  11. 14.

    0.2”   OpQcal  Model            

                                 +Tracking                                                            +DiffracQon                            +Detector  PerturbaQons                 +Lens  PerturbaQons                  +Mirror  PerturbaQons                    +Detector                                                        +Dome  Seeing                 +Low  AlQtude                                          +Mid  AlQtude                                              +High  AlQtude                                        +PixelizaQon        Atmosphere                                                Atmosphere                                                Atmosphere   Peterson  et  al  2013  
  12. 17.

    Key  Component:  Difference  Imaging   OpQcal  Burst  in  the  Deep

     Lens  Survey   hep://www.lsst.org/lsst/public/transient  
  13. 18.

    Key  Component:  Difference  Imaging   OpQcal  Burst  in  the  Deep

     Lens  Survey   hep://www.lsst.org/lsst/public/transient   2  million  events  per  night,   ~500,000  are  random  fluctuaQons   several  thousand  are  “interesQng”  
  14. 19.

    ~  100  Supernovae  used  to  discover  dark  energy    

    By  contrast,  LSST  will  observe  millions   Supernova  in  Pinwheel  galaxy  (BJ  Fulton,  The  Guardian)  
  15. 20.

    LSST  and  Cosmology:   Supernova:  measure  of  the   rate

     of  expansion   GravitaQonal  Lensing:   measure  of  geometry   Baryon  OscillaQons:  trace   of  primordial  fluctuaQons   Clustering:  measure  of  the   geometry  and  structure  growth  
  16. 21.

    LSST  and  The  Transient  Universe:   TransiQng  Planets   Supernovae

      Gamma  Ray  Bursts   Quasars/AGN   Stellar  flares  &   Variability   Asteroids  &   Comets  
  17. 28.

    Image  Differencing:  not  as  easy  as  it  sounds…   Rotate

     &  align  with   “science  image”   Detect  sources   Model  point-­‐spread   funcQon   De-­‐convolve  &  subtract   –   =  
  18. 29.
  19. 30.

    Base  ComputaQon  Layer:  C++  Object  Model.   User  Interface:  Python

     Object  Model.   SWIG  binding  layer   Pipeline  Language  Model  
  20. 31.

    Why  Python?   Open  source  (free!)  &  cross-­‐plaqorm   Easy

     wrapping  of  compiled  legacy  code   Dynamic,  InteracQve,  Easy  to  learn  and  use   (even  for  scienQsts!)  
  21. 32.

    Excellent  core  set  of  scienQfic  packages:   NumPy:  Array-­‐based  operaQons

      SciPy:  ScienQfic  Python   Matplotlib:  PublicaQon-­‐quality  plosng   IPython:  InteracQve  CompuQng  
  22. 34.
  23. 45.

    Our  Effort:  AstroML  –  Python  machine  learning  package   hep://www.astroML.org

      200+  Examples  of  real  analysis  on  real  data!   Already  several  papers  with  reproducible   analysis  through  AstroML  
  24. 46.

    •  LSST:  a  decade-­‐long  movie  of  the  enQre  southern  

    sky,  which  will  change  the  way  Astronomers  do  their   science.   •  Python:  quickly  becoming  the  tool  of  choice  for   much  data-­‐driven  research.   •  Reproducibility:  open-­‐source  tools  like  Python  &   IPython  notebook  will  be  vital  as  large  datasets   become  more  and  more  central  to  research.  
  25. 47.

    Email:  jakevdp@cs.washington.edu     Twieer:  @jakevdp     Github:  jakevdp

        Web:  hep://www.vanderplas.com     Blog:  hep://jakevdp.github.io