Convnets + Galaxies

from Computing the Universe, January 2015, UC Berkeley. More info at http://stanford.edu/~rkeisler/gz/ .

Transcript

1. Convolutional Neural Networks & Galaxies Ryan Keisler, Stanford “Computing the Universe”, UC Berkeley, Jan. 2015
2. David W. Hogg and the Sloan Digital Sky Survey Collaborations.! All images are © 2013 David W. Hogg.
3. > (5 numbers) How to quantify beyond fluxes?
4. …using machines
5. …using people
6. Galaxy Zoo 2: detailed morphological classifications for ! 304,122 galaxies from the Sloan Digital Sky Survey! ! K. Willett et al 2013 • ~300e3 galaxies (mr > 17) • ~16e6 classifications • ~50 classifications / galaxy 100s of 1000s of volunteers!
7. Classification Decision Tree 37 different outcomes
8. 0 B B B B B B B B B B @ n1 . . . . . . . . . . . . n37 1 C C C C C C C C C C A 3 x 96 x 96 = 27648 numbers 37 numbers People (even “pro astronomers”) disagree, so there is no single classification per galaxy.
 
 The distribution of votes is the “classification”.
9. 0 B B B B B B B B B B @ n1 . . . . . . . . . . . . n37 1 C C C C C C C C C C A 3 x 96 x 96 = 27648 numbers 37 numbers Thanks for the labels, volunteers! ! Now time for some supervised machine learning. some algorithm
10. kaggle! a website for machine learning competitions
11. 0 B B B B B B B B B B @ n1 . . . . . . . . . . . . n37 1 C C C C C C C C C C A 3 x 96 x 96 = 27648 numbers 37 numbers What algorithm? some algorithm
12. 0 B B B B B B B B B B @ n1 . . . . . . . . . . . . n37 1 C C C C C C C C C C A 3 x 96 x 96 = 27648 numbers 37 numbers An early try: PCA + RF PCA Random Forest Regressor
14. PCA + RF: Quick to run, good performance.
15. 0 B B B B B B B B B B @ n1 . . . . . . . . . . . . n37 1 C C C C C C C C C C A 3 x 96 x 96 = 27648 numbers 37 numbers What algorithm? some algorithm
16. - “Analyzing complex image data? 
 Use a deep, convolutional neural network.” etc. lots of buzz over the past 2 years
17. What can they do? classification localization! ! …building block for 
 machine vision pipelines Russakovsky et al 2014 Karpathy et al 2014
18. What is a 
 deep convolutional neural net?
19. What is a 
 deep convolutional neural net? a (non-linear) function from to Rm Rn
20. What is a 
 deep convolutional neural net? a (non-linear) function from to Rm Rn
21. What is a 
 deep convolutional neural net? Self-learn a set of multi-channel convolutional kernels.
 (tiles, filter bank) Take advantage of approximate translational invariance. Fewer parameters to learn.
22. What is a 
 deep convolutional neural net? “Lots” of layers hierarchical / compositional
23. from “Visualizing and Understanding Convolutional Networks” Zeiler & Fergus 2013 A Hierarchy of Features
24. Why now?
25. • number of layers • width of layers • size of conv. kernels • pooling stride • choice of non-linear fn’s. • dropout • learning rates • … Lots of hyper-parameters ! ! ! ! ! ! ! ! Convnets might be overkill for a domain as “limited” as astronomical images. Computationally expensive to train (GPU-days or CPU-weeks) Cons
26. ~All the best classifiers, including the best, 
 used convolutional neural nets. results …300+ teams
27. neuron 0 neuron 1 neuron 2 neuron 3 neuron 4 neuron 5 neuron 6 neuron 7 neuron 8 neuron 9 …
28. Astro. Applications? • objective (non-human) morphology • finding strong gravitational lenses • image-based redshifts (as opposed to photo-z.) • spectrograms or other 2d data • extending any of above to LSST-like scale • ?
29. “Smooth Galaxies” movie! 1000 nearby spirals 15 frames (galaxies) per second Subsequent frames chosen to be close in 
 morphology space and image space. http://vimeo.com/86254924