28

# MCM 2019 Pilot Sample

Talk given at the MCM 2019 conference in Sydney as part of at a special session on current challenges in high dimensional algorithms.

July 08, 2019

## Transcript

1. ### An Optimal Adaptive Algorithm Based on a Pilot Sample Fred

J. Hickernell Department of Applied Mathematics Center for Interdisciplinary Scientiﬁc Computation Illinois Institute of Technology hickernell@iit.edu mypages.iit.edu/~hickernell with Yuhan Ding, Peter Kritzer, and Simon Mak partially supported by NSF-DMS-1522687 and NSF-DMS-1638521 (SAMSI) Twelfth International Conference on Monte Carlo Methods and Applications, July 8, 2019
2. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Context Linear

solution operator SOL : F → G, input space F contains functions deﬁned on Ω ⊆ Rd e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary Approximation APP(f, n) = n i=1 Li (f)gi , can sample linear functionals SOL(f) − APP(f, n) G SOL − APP(·, n) F→G f F Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε) G ε for all f ∈ C ⊂ F 2/14
3. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Context Linear

solution operator SOL : F → G, input space F contains functions deﬁned on Ω ⊆ Rd e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary Approximation APP(f, n) = n i=1 Li (f)gi , can sample linear functionals SOL(f) − APP(f, n) G SOL − APP(·, n) F→G f F Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε) G ε for all f ∈ C ⊂ F Solvability: what C? how to determine n∗(f, ε) ∈ N? n∗(f, ε) = min{n : SOL − APP(·, n) F→G ε/R} if f ∈ of radius R Alternatively, assume f ∈ and bound f F or equivalent What you do not see is not much worse than what you see? H., F. J., Jiménez Rugama, & Li, D. in Contemporary Computational Mathematics — a celebration of the 80th birthday of Ian Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.) 597–619 (Springer-Verlag, 2018), Kunsch, R. J., Novak, E. & Rudolf, D. Solvable Integration Problems and Optimal Sample Size Selection. Journal of Complexity. To appear (2018), Ding, Y., H., F. J. & Jiménez Rugama, An Adaptive Algorithm Employing Continuous Linear Functionals. in Monte Carlo and Quasi-Monte Carlo Methods: MCQMC, Rennes, France, July 2018 (eds L’Ecuyer, P. & Tuﬃn, B.) Under revision (Springer-Verlag, Berlin, 2019+). 2/14
4. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Context Linear

solution operator SOL : F → G, input space F contains functions deﬁned on Ω ⊆ Rd e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary Approximation APP(f, n) = n i=1 Li (f)gi , can sample linear functionals SOL(f) − APP(f, n) G SOL − APP(·, n) F→G f F Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε) G ε for all f ∈ C ⊂ F Solvability: what C? how to determine n∗(f, ε) ∈ N? ×or Optimality: is n∗(f, ε) essentially as small as possible? Tractability: does n∗(f, ε) depend nicely on d? 2/14
5. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References General Linear

Problems Deﬁned on Series Spaces F :=    f = ∞ i=1 f(ki )uki : f F := f(ki ) λki ∞ i=1 ρ    λk1 λk2 · · · > 0 λ aﬀects convergence rate & tractability G := g = ∞ i=1 ^ g(ki )vki : g G := ^ g τ , SOL(f) = ∞ i=1 f(ki )vki , τ ρ Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 3/14
6. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Legendre and

Chebyshev Bases for Function Approximation Legendre Chebyshev 4/14
7. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Adaptive Algorithm

for General Linear Problems on Cone of Inputs F :=    f = ∞ i=1 f(ki )uki : f F := f(ki ) λki ∞ i=1 ρ    λk1 λk2 · · · > 0 λ aﬀects convergence rate & tractability G := g = ∞ i=1 ^ g(ki )vki : g G := ^ g τ , SOL(f) = ∞ i=1 f(ki )vki , τ ρ Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input APP(f, n) = n i=1 f(ki )vki optimal for ﬁxed n SOL(f) − APP(f, n) G ERR f(ki ) n i=1 , n ERR f(ki ) n i=1 , n :=  Aρ f(ki ) λki n1 i=1 ρ ρ − f(ki ) λki n i=1 ρ ρ   1/ρ upper bound on f− n i=1 f(ki)uki F λki ∞ i=n+1 ρ SOL − APP(·,n) F→G 1 ρ + 1 ρ = 1 τ data-driven Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 5/14
8. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Adaptive Algorithm

for General Linear Problems on Cone of Inputs Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input APP(f, n) = n i=1 f(ki )vki optimal for ﬁxed n SOL(f) − APP(f, n) G ERR f(ki ) n i=1 , n ERR f(ki ) n i=1 , n :=  Aρ f(ki ) λki n1 i=1 ρ ρ − f(ki ) λki n i=1 ρ ρ   1/ρ upper bound on f− n i=1 f(ki)uki F λki ∞ i=n+1 ρ SOL − APP(·,n) F→G 1 ρ + 1 ρ = 1 τ data-driven ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki ) n i=1 , n ε} Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 5/14
9. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Adaptive Algorithm

for General Linear Problems on Cone of Inputs Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input APP(f, n) = n i=1 f(ki )vki optimal for ﬁxed n SOL(f) − APP(f, n) G ERR f(ki ) n i=1 , n ERR f(ki ) n i=1 , n :=  Aρ f(ki ) λki n1 i=1 ρ ρ − f(ki ) λki n i=1 ρ ρ   1/ρ upper bound on f− n i=1 f(ki)uki F λki ∞ i=n+1 ρ SOL − APP(·,n) F→G 1 ρ + 1 ρ = 1 τ data-driven ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki ) n i=1 , n ε} COST(ALG, Cλ,n1 ,A , ε, R) = max n∗(f, ε) : f ∈ Cλ,n1 ,A ∩ BR = ∩ = min n n1 : λki ∞ i=n+1 ρ ε/[(Aρ − 1)1/ρR] Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 5/14
10. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Optimality of

Algorithm Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki ) n i=1 , n ε} COST(ALG, Cλ,n1 ,A , ε, R) = max n∗(f, ε) : f ∈ Cλ,n1 ,A ∩ BR = ∩ = min n n1 : λki ∞ i=n+1 ρ ε/[(Aρ − 1)1/ρR] COMP(A(Cλ,n1 ,A , ε, R)) = min{COST(ALG , Cλ,n1 ,A , ε, R) : ALG ∈ A(Cλ,n1 ,A )} COST(ALG, Cλ,n1 ,A , ωε, R), ω = 2A(Aρ − 1)1/ρ A − 1 > 1 via fooling functions ALG is essentially optimal Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 6/14
11. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Tractability of

Solving General Linear Problems on Cone of Inputs Cd,λd ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input For τ = ρ (ρ = ∞), the problem SOLd : Cd,λd ,n1 ,A → Gd is Strongly polynomial tractable iﬀ there exist i0 ∈ N and η > 0 such that sup d∈N ∞ i=i0 λη d,ki < ∞ Polynomial tractable iﬀ there exist η1 , η2 0 and η3 , K > 0 such that sup d∈N d−η1 ∞ i= Kdη2 λη3 d,ki < ∞ Weakly tractable iﬀ sup d∈N exp(−cd) ∞ i=1 exp −c 1 λd,ki < ∞ for all c > 0 Analogous results exist for τ < ρ (ρ < ∞) Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 7/14
12. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References An Alternate

Cone, Similar to Our Cone f F := f(ki ) λki ∞ i=1 ρ , λki ↓ 0, f F := f(ki )ζki λki ∞ i=1 ρ , ζk1 = 1, ζki ↓ Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input Cλ,ζ,A := f ∈ F : f F A f F stronger norm is bounded above by weaker norm Cλ,ζ,A ⊆ Cλ,n1 ,A for A A 1 + ζρ kn1+1 1 − (ζkn1+1 A )ρ 1/ρ , ζkn1+1 A < 1 Cλ,n1 ,A ⊆ Cλ,ζ,A for A A ζkn1 8/14
13. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References An Alternative

Cone, Similar to Our Cone For any f ∈ Cλ,ζ,A it follows that f ρ F − f(ki )ζki λki n1 i=1 ρ ρ = f(ki )ζki λki ∞ i=n1 +1 ρ ρ ζρ kn1+1 f(ki ) λki ∞ i=n1 +1 ρ ρ = ζρ kn1+1   f ρ F − f(ki ) λki n1 i=1 ρ ρ   ζρ kn1+1  A ρ f ρ F − f(ki )ζki λki n1 i=1 ρ ρ   9/14
14. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References An Alternative

Cone, Similar to Our Cone For any f ∈ Cλ,ζ,A it follows that f ρ F − f(ki )ζki λki n1 i=1 ρ ρ = f(ki )ζki λki ∞ i=n1 +1 ρ ρ ζρ kn1+1 f(ki ) λki ∞ i=n1 +1 ρ ρ = ζρ kn1+1   f ρ F − f(ki ) λki n1 i=1 ρ ρ   ζρ kn1+1  A ρ f ρ F − f(ki )ζki λki n1 i=1 ρ ρ   f ρ F 1 + ζρ kn1+1 1 − (ζkn1+1 A )ρ f(ki )ζki λki n1 i=1 ρ ρ f F A f F A 1 + ζρ kn1+1 1 − (ζkn1+1 A )ρ 1/ρ f(ki ) λki n1 i=1 ρ 9/14
15. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References An Alternate

Cone, Similar to Our Cone For any f ∈ Cλ,n1 ,A it follows that f F A f(ki ) λki n1 i=1 ρ A ζkn1 f(ki )ζki λki n1 i=1 ρ A ζkn1 f F 10/14
16. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Conclusion Summary

Adaptive algorithms can be constructed for non-convex, symmetric cones of inputs One possible cone assumes that a pilot sample tells you enough about the norm of the input Information cost and complexity depend on the norm of the input, but the adaptive algorithm does not know this norm There are tractability results for problems deﬁned on cones Our algorithm can be extended to infer the λk in terms of coordinate weights Further Work Need to develop and analyze algorithms based on function values, not series coeﬃcients Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 11/14
17. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References Cheng and

Sandu Function Function values for data Chebyshev polynomial basis, λk inferred , Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets. 2017. https://www.sfu.ca/~ssurjano/index.html (2017). 12/14

19. ### Introduction Algorithm Optimality Tractability Alternate Cone Summary References References I

H., F. J., Jiménez Rugama, & Li, D. in Contemporary Computational Mathematics — a celebration of the 80th birthday of Ian Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.) 597–619 (Springer-Verlag, 2018). Kunsch, R. J., Novak, E. & Rudolf, D. Solvable Integration Problems and Optimal Sample Size Selection. Journal of Complexity. To appear (2018). Ding, Y., H., F. J. & Jiménez Rugama, An Adaptive Algorithm Employing Continuous Linear Functionals. in Monte Carlo and Quasi-Monte Carlo Methods: MCQMC, Rennes, France, July 2018 (eds L’Ecuyer, P. & Tuﬃn, B.) Under revision (Springer-Verlag, Berlin, 2019+). Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets. 2017. https://www.sfu.ca/~ssurjano/index.html (2017). 14/14