Markov Chain Simulations Basic Monte Carlo Methods Rohit Goswami [email protected] Presented to the Turku Data Science Group June 12, 2020

Bayes Again • We only care about expectation values i.e. effective integration (|) [()] = ∫ ()(|), where (|) = (|)() ∫ (|)() Aki’s slides for ch11, and Speagle [3] Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 2 13

Quadratures Definite Integrals mesh points ( ) = 0 < 1 ⋯ < = evaluation points ( ∈ [−1 , ]) lim→∞ [1≤≤ ( − −1 )] = 0 then = lim→∞ ∑ =1 ( )( − −1 ) practically = ∫ () ≈ ∑ =0 ( ) Epperson [1] and Speagle [3] Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 3 13

Grid Caveats Consistency lim →∞ ∑ =1 ( )(|)Δ ∑ =1 (|)Δ = (|) [()] Convergence lim →∞ ∑ =1 ( )(|)Δ ∑ =1 (|)Δ = Dimensionality For ≥ 2 grid points in each dimension and points ≈ ∏ =1 = Speagle [3] Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 4 13

Proposal Densities and Monte Carlo • By definition, at the limit of infinite points, the function is continuous Is actually the ideal w.r.t Metropolis-Hastings • Expectations can be estimated with samples from () Monte Carlo approach Grids to Integrals (|) [()] = [()(|)/()] [(|)/()] Epperson [1] and Speagle [3] Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 5 13

Markov Chains • Discrete random variables, characterized by one-step transition probabilities • Future depends on the the current state (theoretically) Current state contains information for future (stochastic) behaviour • At any given state and instant, the future and past are independent { ≤ |−1 = −1 , ⋯ 1 = 1 } = { ≤ |−1 = −1 } See J. Virtamo’s slides on Queuing theory (S-38.143) Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 6 13

Markov Chain Monte Carlo • Sample density () ≡ () at over is ∫ ∈ () ≈ ∫ ∈ () ≈ −1 ∑ =1 ifelse[ ∈ , = 1, = 0] • MCMC simulates values of such that () follows () eventually (|) [()] ≈ −1 ∑ =1 • Converges slowly (normally) • Estimates the expectation Positives • Is consistent (almost always) with ∞ samples Speagle [3] Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 7 13

Gibbs Sampling • Special form of the Metropolis Hastings • Slow if parameters are dependent in the posterior Algorithm • Sample from ( |−1 − , ) −1 − = ( 1 , … , −1 −1 +1 , … , −1 ) Gelman [2] and Aki’s ch11 slides Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 8 13

Metropolis Hastings • Propose → ′ +1 from (′ 1+ | ) • Compute (+1 | ) = min[1, (+1 |),( |′ +1 ) ( )(′ +1 | ) ] ifelse(+1 ≤ (′ +1 | ) → +1 = ′ +1 → +1 = Positives • Allows for asymmetric jumps • Converges to a unique stationary distribution Assumptions • Ideal proposal distribution is the posterior • Shape assumed to be a Gaussian or a distribution Scale • Smaller scales, high acceptance → slow chain • Larger scales, low acceptance → slow chain Gelman [2] and Speagle [3] and Aki’s ch11 slides Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 9 13

Visual Convergence Pitfalls • Seemingly stable, but not converged • Common distribution, but not stationary • Assess between sequence and within sequence information for convergence!! Gelman [2] Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 10 13

Practical Considerations Thinning • Reduce storage load by skipping simulation draws Warm-up • Remove early draws (a.k.a burn-in) • Adapts algorithm parameters in Stan Stopping Criteria ̂ = √ ̂ + ≈ 1 • Tends to one for N → ∞ Gelman [2] and Aki’s ch11 slides Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 11 13

References I James F Epperson. “An Introduction to Numerical Methods and Analysis”. In: (), p. 615. Andrew Gelman. Bayesian Data Analysis. Third edition. Chapman & Hall/CRC Texts in Statistical Science. Boca Raton: CRC Press, 2014. 661 pp. isbn: 978-1-4398-4095-5. Joshua S. Speagle. “A Conceptual Introduction to Markov Chain Monte Carlo Methods”. In: (Mar. 7, 2020). arXiv: 1909.12313 [astro-ph, physics:physics, stat]. url: http://arxiv.org/abs/1909.12313 (visited on 06/11/2020). Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 12 13

End Thank you Rohit Goswami (Presented to the Turku Data Science Group) Markov Chain Simulations June 12, 2020 13 13