Quasi-Monte Carlo Software Fred J. Hickernell Department of Applied Mathematics & Center for Interdisciplinary Scientific Computation Illinois Institute of Technology [email protected] mypages.iit.edu/~hickernell with Sou-Cheng Choi, Mike McCourt, Jagadeeswaran Rathinavel, Aleksei Sorokin and the rest of the GAIL an QMCPy teams partially supported by SigOpt and NSF-DMS-1522687 Thanks to the organizers, especially during these unusual times Slides at speakerdeck.com/fjhickernell/quasi-monte-carlo-software Google Colaboratory notebook at tinyurl.com/QMCPyTutorial Blog at qmcpy.wordpress.com/ MCQMC 2020, August 2020

Background QMCPy on Google Colaboratory Why collaborate? You have heard about QMC. How do you try it? QMC will give you 100 times the accuracy in the same amount of time as simple MC Just replace your IID random points with low discrepancy (LD) points 2/12

Background QMCPy on Google Colaboratory Why collaborate? You have heard about QMC. How do you try it? QMC will give you 100 times the accuracy in the same amount of time as simple MC Often Just replace your IID random points with low discrepancy (LD) points Sometimes 2/12

Background QMCPy on Google Colaboratory Why collaborate? You have heard about QMC. How do you try it? QMC will give you 100 times the accuracy in the same amount of time as simple MC Often Just replace your IID random points with low discrepancy (LD) points Sometimes Where can I get accurate, efficient, easy to use QMC software to try for my problem? How can I make my great QMC software available for others? 2/12

Background QMCPy on Google Colaboratory Why collaborate? You have heard about QMC. How do you try it? QMC will give you 100 times the accuracy in the same amount of time as simple MC Often Just replace your IID random points with low discrepancy (LD) points Sometimes Where can I get accurate, efficient, easy to use QMC software to try for my problem? How can I make my great QMC software available for others? Let’s try to help 2/12

Background QMCPy on Google Colaboratory Why collaborate? Quasi-Monte Carlo (QMC) uses low discrepancy (LD) sequences Ti are random Xi may be deterministic or random T1, T2 · · · IID ∼ F X1, X2 · · · LD ∼ F Ti do not know about one another {Xi}n i=1 represent F well Fn(t1, . . . , tn) = F(t1) · · · F(tn) F{Xi}n i=1 (x) ≈ F(x) 3/12

Background QMCPy on Google Colaboratory Why collaborate? Where is the software? (apologies to those I missed) LD Sequence Generators Multi-Level, Stopping Criteria, Applications MATLAB Statistics Toolbox— Sobol’ and Halton Mike Giles—Multi-Level (Quasi-)Monte Carlo in C++, MATLAB, Python, and R Marius Hofert & Christiane Lemieux—qrng R package, Sobol’ and Halton Guaranteed Automatic Integration Library (GAIL)—Stopping criteria in MATLAB BRODA, Sergei Kucherenko—Sobol’ in C, MATLAB, and Excel UQLab—Framework for Uncertainty Quantification in MATLAB Art Owen—randomized Halton in R OpenTURNS—An Open source initiative for the Treatment of Uncertainties, Risks ’N Statistics in Python Pieterjan Robbe—LD sequences in Julia PyTorch—Sobol’ in Python Pierre L’Ecuyer—Lattice Builder and Stochastic Simulation in C/C++ and Java Dirk Nuyens—Magic Point Shop and QMC4PDE in MATLAB, Python, and C++ John Burkhardt—variety in C++, Fortran, MATLAB, & Python QMCPy—Python package incorporating and connecting the work of different groups 4/12

Background QMCPy on Google Colaboratory Why collaborate? Where is the software? (apologies to those I missed) LD Sequence Generators Multi-Level, Stopping Criteria, Applications Mike Giles—Multi-Level (Quasi-)Monte Carlo Marius Hofert & Christiane Lemieux—qrng Sobol’ and Halton Guaranteed Automatic Integration Library (GAIL)—Stopping criteria Art Owen—randomized Halton PyTorch—Sobol’ Pierre L’Ecuyer—Lattice Builder Dirk Nuyens—Magic Point Shop QMCPy—Python package incorporating and connecting the work of different groups 4/12

Background QMCPy on Google Colaboratory Why collaborate? Why LD is better than IID Integration/Expectation Arising in finance, uncertainty quantification, Bayesian inference, ... µ := E[f(X)] = X f(x) (x) dx ≈ 1 n n i=1 f(Xi) =: ^ µn LD points give faster convergence than IID Design of Computer Experiments LD can be more space filling (even than Latin hypercube sampling) for use in constructing surrogate models and uncertainty quantification Global Optimization LD points are more space filling and find good starting points for local methods 5/12

Background QMCPy on Google Colaboratory Why collaborate? What software components do we need? µ := E[f(X)] expectation = X f(x) (x) dx integration ≈ 1 n n i=1 f(Xi) =: ^ µn LD Generator producing {X1, X2, . . . } that mimics the distribution with PDF , e.g., uniform 6/12

Background QMCPy on Google Colaboratory Why collaborate? What software components do we need? µ := T g(t) λ(t) dt = · · · = E[f(X)] expectation = X f(x) (x) dx integration ≈ 1 n n i=1 f(Xi) =: ^ µn LD Generator producing {X1, X2, . . . } that mimics the distribution with PDF , e.g., uniform True Measure that defines the original integral, e.g., Lebesgue 6/12

Background QMCPy on Google Colaboratory Why collaborate? What software components do we need? µ := T g(t) λ(t) dt = · · · = E[f(X)] expectation = X f(x) (x) dx integration ≈ 1 n n i=1 f(Xi) =: ^ µn LD Generator producing {X1, X2, . . . } that mimics the distribution with PDF , e.g., uniform True Measure that defines the original integral, e.g., Lebesgue Integrand g, which defines the original integral, plus the transformed version, f, to fit the LD generator 6/12

Background QMCPy on Google Colaboratory Why collaborate? What software components do we need? µ := T g(t) λ(t) dt = · · · = E[f(X)] expectation = X f(x) (x) dx integration ≈ 1 n n i=1 f(Xi) =: ^ µn LD Generator producing {X1, X2, . . . } that mimics the distribution with PDF , e.g., uniform True Measure that defines the original integral, e.g., Lebesgue Integrand g, which defines the original integral, plus the transformed version, f, to fit the LD generator Stopping Criterion that determines how large n should be to ensure that |µ − ^ µn| ε 6/12

Background QMCPy on Google Colaboratory Why collaborate? QMCPy in a Jupyter Notebooks on Google Colaboratory Prerequisites—No Python, Jupyter, or QMC knowledge assumed Goals Show you how QMC software works Interest you in using/contributing Directions Point your browser to https://tinyurl.com/QMCPyTutorial Open the file in Google Colaboratory (may need to push a button at the top of your browser) Make a copy of this file onto your own Google drive account File → Save a copy in Drive Pause for questions 7/12

Background QMCPy on Google Colaboratory Why collaborate? Acknowledgments for QMCPy Coded almost entirely by Aleksei Sorokin (BS/MS ’21) Relies on code from several groups (see above) Documentation and testing by Sou-Cheng Choi and Jagadeeswaran R. Funded and encouraged by Mike McCourt at SigOpt 8/12

Background QMCPy on Google Colaboratory Why collaborate? The Guaranteed Automatic Integration Library (GAIL) and QMCPy teams Sou-Cheng Choi (Chief Data Scientist, Kamakura) Yuhan Ding (IIT PhD ’15, Lecturer, IIT) Lan Jiang (IIT PhD ’16, Compass) Lluís Antoni Jiménez Rugama (IIT PhD ’17, UBS) Mike McCourt (IIT BS ’07, Cornell PhD ’12, Head of Research, SigOpt) Jagadeeswaran Rathinavel (IIT PhD ’19, Wi-Tronix) Aleksei Sorokin (IIT BS & MAS ’21 exp.) Xin Tong (IIT MS, UIC PhD ’20 exp.) Kan Zhang (IIT PhD ’20 exp.) Yizhi Zhang (IIT PhD ’18, Jamran Int’l) Xuan Zhou (IIT PhD ’15, JP Morgan) and others 9/12

Background QMCPy on Google Colaboratory Why collaborate? The argument for community software Our research groups are typically expert at only part of the whole picture: LD sequence generators Increasing efficiency, e.g., MLMC, MDM Stopping criteria Use cases A community library lets us all take advantage of the best 10/12

Background QMCPy on Google Colaboratory Why collaborate? The argument for community software Our research groups are typically expert at only part of the whole picture: LD sequence generators Increasing efficiency, e.g., MLMC, MDM Stopping criteria Use cases A community library lets us all take advantage of the best Provides a consistent interface for pieces from different places 10/12

Background QMCPy on Google Colaboratory Why collaborate? The argument for community software Our research groups are typically expert at only part of the whole picture: LD sequence generators Increasing efficiency, e.g., MLMC, MDM Stopping criteria Use cases A community library lets us all take advantage of the best Provides a consistent interface for pieces from different places Enables reproducible computational research 10/12

Background QMCPy on Google Colaboratory Why collaborate? The argument for community software Our research groups are typically expert at only part of the whole picture: LD sequence generators Increasing efficiency, e.g., MLMC, MDM Stopping criteria Use cases A community library lets us all take advantage of the best Provides a consistent interface for pieces from different places Enables reproducible computational research Tedious stuff only done once 10/12

Background QMCPy on Google Colaboratory Why collaborate? The argument for community software Our research groups are typically expert at only part of the whole picture: LD sequence generators Increasing efficiency, e.g., MLMC, MDM Stopping criteria Use cases A community library lets us all take advantage of the best Provides a consistent interface for pieces from different places Enables reproducible computational research Tedious stuff only done once Many eyes find and correct errors MATLAB’s Sobol’ generator’s scrambling corrected in MATLAB 2017a after Tony Jiménez Rugama noticed the problem PyTorch’s Sobol’ generator found to be wrong unless double precision is proactively specified; also missing the first point; reported at https://github.com/pytorch/pytorch/issues/32047 10/12

Background QMCPy on Google Colaboratory Why collaborate? How you can contribute Try out QMCPy and then Easy Submit your bugs and feature requests as issues to https://github.com/QMCSoftware/QMCSoftware/issues Moderately Difficult Ask your students or collaborators to try QMCPy themselves and submit their bugs and feature requests Email us your blog post to add to qmcpy.wordpress.com/ Heroic Add a feature or use case and make a pull request at https://github.com/QMCSoftware/QMCSoftware/pulls so that we can included it in our next release Questions or suggestions? Email us at [email protected] 11/12

Background QMCPy on Google Colaboratory Why collaborate? How you can contribute Try out QMCPy and then Easy Submit your bugs and feature requests as issues to https://github.com/QMCSoftware/QMCSoftware/issues Moderately Difficult Ask your students or collaborators to try QMCPy themselves and submit their bugs and feature requests Email us your blog post to add to qmcpy.wordpress.com/ Heroic Add a feature or use case and make a pull request at https://github.com/QMCSoftware/QMCSoftware/pulls so that we can included it in our next release Questions or suggestions? Email us at [email protected] If you are wedded to another language, think of designing your software so that others can add to it easily. 11/12

Thank you These slides are available at speakerdeck.com/fjhickernell/quasi-monte-carlo-software Google Colaboratory notebook at tinyurl.com/QMCPyTutorial Blog at qmcpy.wordpress.com/