emcee

Some intro slides for Python lunch at MPIA.

Transcript

1. emcee danfm.ca/emcee p(⇥) I have a function
2. emcee danfm.ca/emcee p(⇥) I have a function I can Evaluate it
3. emcee danfm.ca/emcee p(⇥) I have a function I can Evaluate it I can't Calculate the functional form
4. emcee danfm.ca/emcee p(⇥) I have a function I can Evaluate it I can't Calculate the functional form Markov chain Monte Carlo (MCMC)
5. emcee danfm.ca/emcee Metropolis-Hastings
6. emcee danfm.ca/emcee min ✓ 1 , p ( x 0) p ( x ) Q ( x ; x 0) Q ( x 0; x ) ◆ Metropolis-Hastings
7. emcee danfm.ca/emcee min ✓ 1 , p ( x 0) p ( x ) Q ( x ; x 0) Q ( x 0; x ) ◆ Metropolis-Hastings Proposal D (D-1) parameters
8. emcee danfm.ca/emcee Metropolis-Hastings x y
9. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
10. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
11. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
12. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world) min ✓ 1, p (x 0 ) p (x ) Q (x ;x 0 ) Q (x 0 ;x ) ◆ ?
13. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
14. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
15. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
16. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world) min ✓ 1 , p( x 0 ) p( x) Q( x; x 0 ) Q( x 0; x) ◆ ?
17. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
18. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
19. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
20. emcee danfm.ca/emcee Metropolis-Hastings x y (in an ideal world)
21. emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) x y
22. emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) x y
23. emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) x y
24. emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) x y
25. emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) x y
26. emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) x y SMALL ACCEPTANCE FRACTION the problem
27. emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) x y LARGE ACCEPTANCE FRACTION the problem
28. x y emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world)
29. x y emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world)
30. x y emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world)
31. x y emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) positive-definite symmetric Proposal D (D-1) parameters
32. x y emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) positive-definite symmetric Proposal D (D-1) parameters This is the Dimension of your parameter space!
33. emcee danfm.ca/emcee Metropolis-Hastings (in the REAL world) Scientific Awesomeness how hard is MCMC Metropolis Hastings how things Should be (~number of parameters)
34. emcee danfm.ca/emcee Why does all this matter?
35. emcee danfm.ca/emcee How do you calculate the optimal proposal?
36. emcee danfm.ca/emcee Temperature
37. emcee danfm.ca/emcee Temperature
38. emcee danfm.ca/emcee Temperature
39. emcee danfm.ca/emcee Temperature
40. emcee danfm.ca/emcee Temperature
41. emcee danfm.ca/emcee Temperature
42. emcee danfm.ca/emcee Temperature
43. emcee danfm.ca/emcee Temperature
44. emcee danfm.ca/emcee Temperature
45. emcee danfm.ca/emcee Temperature time
46. emcee danfm.ca/emcee Temperature that should be spent interpreting your results writing papers finding bugs in your code time
47. emcee danfm.ca/emcee Luckily I have a solution!
48. emcee danfm.ca/emcee Luckily I have a solution! HINT: it's up here...
49. emcee danfm.ca/emcee bit.ly/mcmc-gw10 "Ensemble samplers with affine invariance" Jonathan Goodman Jonathan Weare Mustaches courtesy: mustachify.me
50. emcee danfm.ca/emcee bit.ly/mcmc-gw10 "Ensemble samplers with affine invariance" Jonathan Goodman Jonathan Weare Mustaches courtesy: mustachify.me
51. emcee danfm.ca/emcee bit.ly/mcmc-gw10 "Ensemble samplers with affine invariance" Jonathan Goodman Jonathan Weare Mustaches courtesy: mustachify.me
52. emcee danfm.ca/emcee affine invariance
53. emcee danfm.ca/emcee affine invariance y A x + b Affine Transformation
54. emcee danfm.ca/emcee affine invariance The sampler performs Equally well on X and Y y A x + b Affine Transformation
55. emcee danfm.ca/emcee Easy to sample Hard to sample
56. emcee danfm.ca/emcee Easy to sample Hard to sample y A x + b Affine Transformation
57. emcee danfm.ca/emcee Easy to sample Hard to sample y A x + b Affine Transformation easy!
58. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
59. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
60. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
61. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance this is a walker
62. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance this is a walker
63. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
64. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
65. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
66. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
67. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
68. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance min ✓ 1,Z D 1 p (x 0 ) p (x ) ◆
69. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance min ✓ 1,Z D 1 p (x 0 ) p (x ) ◆
70. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
71. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
72. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
73. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance
74. emcee danfm.ca/emcee Ensemble Samplers (in the REAL world) x y with affine invariance Aside: this looks nice and parallel, eh? * * not quite as trivial as you might hope—but possible!
75. emcee danfm.ca/emcee +
76. emcee danfm.ca/emcee it's hammer time! emceethe MCMC Hammer introducing arxiv.org/abs/1202.3665
77. emcee danfm.ca/emcee pip install emcee get it:
78. emcee danfm.ca/emcee import numpy as np import emcee def lnprob(x): return -0.5 * np.sum(x ** 2) ndim, nwalkers = 10, 100 p0 = [np.random.rand(ndim) for i in range(nwalkers)] sampler = emcee.EnsembleSampler(nwalkers, ndim, lnprob) sampler.run_mcmc(p0, 1000) use it:
79. emcee danfm.ca/emcee DOES IT WORK? obviously it does.
80. emcee danfm.ca/emcee 1.0 0.5 0.0 0.5 1.0 1.0 0.5 0.0 0.5 1.0 exp ✓ (x1 x2) 2 2 ✏ (x1 + x2) 2 2 ◆
81. emcee danfm.ca/emcee github.com/dfm/acor Autocorrelation Function the (covariance)
82. emcee danfm.ca/emcee 1.0 0.5 0.0 0.5 1.0 1.0 0.5 0.0 0.5 1.0 exp ✓ (x1 x2) 2 2 ✏ (x1 + x2) 2 2 ◆ Metropolis-Hastings Emcee Autocorrelation Function the
83. emcee danfm.ca/emcee 1.0 0.5 0.0 0.5 1.0 1.0 0.5 0.0 0.5 1.0 exp ✓ (x1 x2) 2 2 ✏ (x1 + x2) 2 2 ◆ Metropolis-Hastings Emcee Autocorrelation Function the
84. emcee danfm.ca/emcee 1.0 0.5 0.0 0.5 1.0 1.0 0.5 0.0 0.5 1.0 exp ✓ (x1 x2) 2 2 ✏ (x1 + x2) 2 2 ◆ Metropolis-Hastings Emcee Autocorrelation Function the
85. emcee danfm.ca/emcee Metropolis-Hastings Boom!
86. emcee danfm.ca/emcee 4 2 0 2 4 6 0 5 10 15 20 25 30 exp ✓ 100 (x2 x 2 1) 2 + (1 x1) 2 20 ◆
87. 4 2 0 2 4 6 0 5 10 15 20 25 30 exp ✓ 100 (x2 x 2 1) 2 + (1 x1) 2 20 ◆ emcee danfm.ca/emcee Metropolis-Hastings Emcee Autocorrelation Function the
88. emcee isn't always The Right Choice™ emcee danfm.ca/emcee Mustache courtesy: mustachify.me Brendon Brewer Remember:
89. emcee danfm.ca/emcee Mustache courtesy: mustachify.me continuous parameters in a vector space emcee needs highly multimodal problems and it is not good at
90. emcee danfm.ca/emcee Mustache courtesy: mustachify.me continuous parameters in a vector space emcee needs highly multimodal problems and it is not good at what is?
91. emcee danfm.ca/emcee Mustache courtesy: mustachify.me continuous parameters in a vector space emcee needs highly multimodal problems and it is not good at what is? maybe Dnest github.com/eggplantbren/DNest3
92. emcee danfm.ca/emcee Mustache courtesy: mustachify.me continuous parameters in a vector space emcee needs highly multimodal problems and it is not good at what is? maybe Dnest github.com/eggplantbren/DNest3 for example
93. emcee danfm.ca/emcee it's still been pretty useful... Lang & Hogg (2011) Bovy et al. (2011) Dorman et al. (2012) Foreman-Mackey & Widrow (in prep) Mustaches courtesy: mustachify.me ... ... ...
94. emceethe MCMC Hammer arxiv.org/abs/1202.3665 danfm.ca/emcee github.com/dfm/emcee paper documentation issues/contributions Check it out: Dustin Lang (Princeton) David W. Hogg (NYU) Jonathan Goodman (NYU)