Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Unlocking the Universe with Python and LSST

Unlocking the Universe with Python and LSST

Invited talk from the 2013 RuPy conference, Budapest, Hungary.

Jake VanderPlas

October 13, 2013
Tweet

More Decks by Jake VanderPlas

Other Decks in Science

Transcript

  1. Unlocking  the  Universe  with   Python  and  LSST   Jake

     VanderPlas   RuPy,  Oct  11th  2013  
  2. Then:  Edwin  Hubble  at   Mount  Wilson  Observatory   1990s

     -­‐  Now:  the  Hubble  Space   Telescope  in  Low  Earth  Orbit…  
  3. Large   SynopQc   Survey   Telescope   LSST in

    a sentence: An optical/near-IR multi-color survey of half the sky, based on ~1000 visits over a 10-year period
  4. 8.4  meter  primary  mirror   3,000  Megapixel  CCD  camera  

    Field  of  View:  9.5  deg2   Largest  digital  camera  in  the   world!  
  5. LSST:  a    10-­‐year,  full-­‐sky  digital  color  movie   -­‐ 

    Full  southern  sky  every  3-­‐4  nights   -­‐  30,000  GB/night  data  stream   -­‐  Real-­‐Qme  transient  alerts   -­‐  Final  Catalog:  100s  of  Petabytes  
  6. •  Adler  Planetarium   •  Argonne  NaQonal  Laboratory   • 

    Brookhaven  NaQonal  Laboratory  (BNL)     •  California  InsQtute  of  Technology   •  Carnegie  Mellon  University   •  Chile   •  Cornell  University   •  Drexel  University   •  Fermi  NaQonal  Accelerator  Laboratory     •  George  Mason  University   •  Google,  Inc.   •  Harvard-­‐Smithsonian  Center  for  Astrophysics   •  InsQtut  de  Physique  Nucleaire  et  de  Physique   des  ParQcules  (IN2P3)   •  Johns  Hopkins  University   •  Kavli  InsQtute  for  ParQcle  Astrophysics  and   Cosmology  (KIPAC)  -­‐  Stanford  University   •  Las  Cumbres  Observatory  Global  Telescope   Network,  Inc.   •  Lawrence  Livermore  NaQonal  Laboratory  (LLNL)   •  Los  Alamos  NaQonal  Laboratory  (LANL)   •  NaQonal  OpQcal  Astronomy  Observatory   •  NaQonal  Radio  Astronomy  Observatory   •  Northwestern  University   •  Princeton  University   •  Purdue  University   •  Research  CorporaQon  for  Science  Advancement   •  Rutgers  University   •  SLAC  NaQonal  Accelerator  Laboratory   •  Space  Telescope  Science  InsQtute   •  Texas  A  &  M  University   •  The  InsQtute  of  Physics  of  the  Academy  of   Sciences  of  the  Czech  Republic   •  The  Pennsylvania  State  University   •  The  University  of  Arizona   •  University  of  California  at  Davis   •  University  of  California  at  Irvine   •  University  of  Illinois  at  Urbana-­‐Champaign   •  University  of  Michigan   •  University  of  Pennsylvania   •  University  of  Piesburgh   •  University  of  Washington   •  Vanderbilt  and  Fisk  UniversiQes   LSST  Ins(tu(onal  Members:  
  7. •  Adler  Planetarium   •  Argonne  NaQonal  Laboratory   • 

    Brookhaven  NaQonal  Laboratory  (BNL)     •  California  InsQtute  of  Technology   •  Carnegie  Mellon  University   •  Chile   •  Cornell  University   •  Drexel  University   •  Fermi  NaQonal  Accelerator  Laboratory     •  George  Mason  University   •  Google,  Inc.   •  Harvard-­‐Smithsonian  Center  for  Astrophysics   •  InsQtut  de  Physique  Nucleaire  et  de  Physique   des  ParQcules  (IN2P3)   •  Johns  Hopkins  University   •  Kavli  InsQtute  for  ParQcle  Astrophysics  and   Cosmology  (KIPAC)  -­‐  Stanford  University   •  Las  Cumbres  Observatory  Global  Telescope   Network,  Inc.   •  Lawrence  Livermore  NaQonal  Laboratory  (LLNL)   •  Los  Alamos  NaQonal  Laboratory  (LANL)   •  NaQonal  OpQcal  Astronomy  Observatory   •  NaQonal  Radio  Astronomy  Observatory   •  Northwestern  University   •  Princeton  University   •  Purdue  University   •  Research  CorporaQon  for  Science  Advancement   •  Rutgers  University   •  SLAC  NaQonal  Accelerator  Laboratory   •  Space  Telescope  Science  InsQtute   •  Texas  A  &  M  University   •  The  InsQtute  of  Physics  of  the  Academy  of   Sciences  of  the  Czech  Republic   •  The  Pennsylvania  State  University   •  The  University  of  Arizona   •  University  of  California  at  Davis   •  University  of  California  at  Irvine   •  University  of  Illinois  at  Urbana-­‐Champaign   •  University  of  Michigan   •  University  of  Pennsylvania   •  University  of  Piesburgh   •  University  of  Washington   •  Vanderbilt  and  Fisk  UniversiQes   LSST  Ins(tu(onal  Members:  
  8. •  Adler  Planetarium   •  Argonne  NaQonal  Laboratory   • 

    Brookhaven  NaQonal  Laboratory  (BNL)     •  California  InsQtute  of  Technology   •  Carnegie  Mellon  University   •  Chile   •  Cornell  University   •  Drexel  University   •  Fermi  NaQonal  Accelerator  Laboratory     •  George  Mason  University   •  Google,  Inc.   •  Harvard-­‐Smithsonian  Center  for  Astrophysics   •  InsQtut  de  Physique  Nucleaire  et  de  Physique   des  ParQcules  (IN2P3)   •  Johns  Hopkins  University   •  Kavli  InsQtute  for  ParQcle  Astrophysics  and   Cosmology  (KIPAC)  -­‐  Stanford  University   •  Las  Cumbres  Observatory  Global  Telescope   Network,  Inc.   •  Lawrence  Livermore  NaQonal  Laboratory  (LLNL)   •  Los  Alamos  NaQonal  Laboratory  (LANL)   •  NaQonal  OpQcal  Astronomy  Observatory   •  NaQonal  Radio  Astronomy  Observatory   •  Northwestern  University   •  Princeton  University   •  Purdue  University   •  Research  CorporaQon  for  Science  Advancement   •  Rutgers  University   •  SLAC  NaQonal  Accelerator  Laboratory   •  Space  Telescope  Science  InsQtute   •  Texas  A  &  M  University   •  The  InsQtute  of  Physics  of  the  Academy  of   Sciences  of  the  Czech  Republic   •  The  Pennsylvania  State  University   •  The  University  of  Arizona   •  University  of  California  at  Davis   •  University  of  California  at  Irvine   •  University  of  Illinois  at  Urbana-­‐Champaign   •  University  of  Michigan   •  University  of  Pennsylvania   •  University  of  Piesburgh   •  University  of  Washington   •  Vanderbilt  and  Fisk  UniversiQes   LSST  Ins(tu(onal  Members:  
  9. 0.2”   OpQcal  Model            

                                 +Tracking                                                            +DiffracQon                            +Detector  PerturbaQons                 +Lens  PerturbaQons                  +Mirror  PerturbaQons                    +Detector                                                        +Dome  Seeing                 +Low  AlQtude                                          +Mid  AlQtude                                              +High  AlQtude                                        +PixelizaQon        Atmosphere                                                Atmosphere                                                Atmosphere   Peterson  et  al  2013  
  10. Key  Component:  Difference  Imaging   OpQcal  Burst  in  the  Deep

     Lens  Survey   hep://www.lsst.org/lsst/public/transient  
  11. Key  Component:  Difference  Imaging   OpQcal  Burst  in  the  Deep

     Lens  Survey   hep://www.lsst.org/lsst/public/transient   2  million  events  per  night,   ~500,000  are  random  fluctuaQons   several  thousand  are  “interesQng”  
  12. ~  100  Supernovae  used  to  discover  dark  energy    

    By  contrast,  LSST  will  observe  millions   Supernova  in  Pinwheel  galaxy  (BJ  Fulton,  The  Guardian)  
  13. LSST  and  Cosmology:   Supernova:  measure  of  the   rate

     of  expansion   GravitaQonal  Lensing:   measure  of  geometry   Baryon  OscillaQons:  trace   of  primordial  fluctuaQons   Clustering:  measure  of  the   geometry  and  structure  growth  
  14. LSST  and  The  Transient  Universe:   TransiQng  Planets   Supernovae

      Gamma  Ray  Bursts   Quasars/AGN   Stellar  flares  &   Variability   Asteroids  &   Comets  
  15. Image  Differencing:  not  as  easy  as  it  sounds…   Rotate

     &  align  with   “science  image”   Detect  sources   Model  point-­‐spread   funcQon   De-­‐convolve  &  subtract   –   =  
  16. Base  ComputaQon  Layer:  C++  Object  Model.   User  Interface:  Python

     Object  Model.   SWIG  binding  layer   Pipeline  Language  Model  
  17. Why  Python?   Open  source  (free!)  &  cross-­‐plaqorm   Easy

     wrapping  of  compiled  legacy  code   Dynamic,  InteracQve,  Easy  to  learn  and  use   (even  for  scienQsts!)  
  18. Excellent  core  set  of  scienQfic  packages:   NumPy:  Array-­‐based  operaQons

      SciPy:  ScienQfic  Python   Matplotlib:  PublicaQon-­‐quality  plosng   IPython:  InteracQve  CompuQng  
  19. Our  Effort:  AstroML  –  Python  machine  learning  package   hep://www.astroML.org

      200+  Examples  of  real  analysis  on  real  data!   Already  several  papers  with  reproducible   analysis  through  AstroML  
  20. •  LSST:  a  decade-­‐long  movie  of  the  enQre  southern  

    sky,  which  will  change  the  way  Astronomers  do  their   science.   •  Python:  quickly  becoming  the  tool  of  choice  for   much  data-­‐driven  research.   •  Reproducibility:  open-­‐source  tools  like  Python  &   IPython  notebook  will  be  vital  as  large  datasets   become  more  and  more  central  to  research.  
  21. Email:  [email protected]     Twieer:  @jakevdp     Github:  jakevdp

        Web:  hep://www.vanderplas.com     Blog:  hep://jakevdp.github.io