Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MCM 2019 Pilot Sample

MCM 2019 Pilot Sample

Talk given at the MCM 2019 conference in Sydney as part of at a special session on current challenges in high dimensional algorithms.

9d6eae084bd3d9a3c86e9a182224f014?s=128

Fred J. Hickernell

July 08, 2019
Tweet

Transcript

  1. An Optimal Adaptive Algorithm Based on a Pilot Sample Fred

    J. Hickernell Department of Applied Mathematics Center for Interdisciplinary Scientific Computation Illinois Institute of Technology hickernell@iit.edu mypages.iit.edu/~hickernell with Yuhan Ding, Peter Kritzer, and Simon Mak partially supported by NSF-DMS-1522687 and NSF-DMS-1638521 (SAMSI) Twelfth International Conference on Monte Carlo Methods and Applications, July 8, 2019
  2. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Context Linear

    solution operator SOL : F → G, input space F contains functions defined on Ω ⊆ Rd e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary Approximation APP(f, n) = n i=1 Li (f)gi , can sample linear functionals SOL(f) − APP(f, n) G SOL − APP(·, n) F→G f F Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε) G ε for all f ∈ C ⊂ F 2/14
  3. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Context Linear

    solution operator SOL : F → G, input space F contains functions defined on Ω ⊆ Rd e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary Approximation APP(f, n) = n i=1 Li (f)gi , can sample linear functionals SOL(f) − APP(f, n) G SOL − APP(·, n) F→G f F Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε) G ε for all f ∈ C ⊂ F Solvability: what C? how to determine n∗(f, ε) ∈ N? n∗(f, ε) = min{n : SOL − APP(·, n) F→G ε/R} if f ∈ of radius R Alternatively, assume f ∈ and bound f F or equivalent What you do not see is not much worse than what you see? H., F. J., Jiménez Rugama, & Li, D. in Contemporary Computational Mathematics — a celebration of the 80th birthday of Ian Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.) 597–619 (Springer-Verlag, 2018), Kunsch, R. J., Novak, E. & Rudolf, D. Solvable Integration Problems and Optimal Sample Size Selection. Journal of Complexity. To appear (2018), Ding, Y., H., F. J. & Jiménez Rugama, An Adaptive Algorithm Employing Continuous Linear Functionals. in Monte Carlo and Quasi-Monte Carlo Methods: MCQMC, Rennes, France, July 2018 (eds L’Ecuyer, P. & Tuffin, B.) Under revision (Springer-Verlag, Berlin, 2019+). 2/14
  4. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Context Linear

    solution operator SOL : F → G, input space F contains functions defined on Ω ⊆ Rd e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary Approximation APP(f, n) = n i=1 Li (f)gi , can sample linear functionals SOL(f) − APP(f, n) G SOL − APP(·, n) F→G f F Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε) G ε for all f ∈ C ⊂ F Solvability: what C? how to determine n∗(f, ε) ∈ N? ×or Optimality: is n∗(f, ε) essentially as small as possible? Tractability: does n∗(f, ε) depend nicely on d? 2/14
  5. Introduction Algorithm Optimality Tractability Alternate Cone Summary References General Linear

    Problems Defined on Series Spaces F :=    f = ∞ i=1 f(ki )uki : f F := f(ki ) λki ∞ i=1 ρ    λk1 λk2 · · · > 0 λ affects convergence rate & tractability G := g = ∞ i=1 ^ g(ki )vki : g G := ^ g τ , SOL(f) = ∞ i=1 f(ki )vki , τ ρ Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 3/14
  6. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Legendre and

    Chebyshev Bases for Function Approximation Legendre Chebyshev 4/14
  7. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Adaptive Algorithm

    for General Linear Problems on Cone of Inputs F :=    f = ∞ i=1 f(ki )uki : f F := f(ki ) λki ∞ i=1 ρ    λk1 λk2 · · · > 0 λ affects convergence rate & tractability G := g = ∞ i=1 ^ g(ki )vki : g G := ^ g τ , SOL(f) = ∞ i=1 f(ki )vki , τ ρ Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input APP(f, n) = n i=1 f(ki )vki optimal for fixed n SOL(f) − APP(f, n) G ERR f(ki ) n i=1 , n ERR f(ki ) n i=1 , n :=  Aρ f(ki ) λki n1 i=1 ρ ρ − f(ki ) λki n i=1 ρ ρ   1/ρ upper bound on f− n i=1 f(ki)uki F λki ∞ i=n+1 ρ SOL − APP(·,n) F→G 1 ρ + 1 ρ = 1 τ data-driven Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 5/14
  8. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Adaptive Algorithm

    for General Linear Problems on Cone of Inputs Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input APP(f, n) = n i=1 f(ki )vki optimal for fixed n SOL(f) − APP(f, n) G ERR f(ki ) n i=1 , n ERR f(ki ) n i=1 , n :=  Aρ f(ki ) λki n1 i=1 ρ ρ − f(ki ) λki n i=1 ρ ρ   1/ρ upper bound on f− n i=1 f(ki)uki F λki ∞ i=n+1 ρ SOL − APP(·,n) F→G 1 ρ + 1 ρ = 1 τ data-driven ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki ) n i=1 , n ε} Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 5/14
  9. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Adaptive Algorithm

    for General Linear Problems on Cone of Inputs Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input APP(f, n) = n i=1 f(ki )vki optimal for fixed n SOL(f) − APP(f, n) G ERR f(ki ) n i=1 , n ERR f(ki ) n i=1 , n :=  Aρ f(ki ) λki n1 i=1 ρ ρ − f(ki ) λki n i=1 ρ ρ   1/ρ upper bound on f− n i=1 f(ki)uki F λki ∞ i=n+1 ρ SOL − APP(·,n) F→G 1 ρ + 1 ρ = 1 τ data-driven ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki ) n i=1 , n ε} COST(ALG, Cλ,n1 ,A , ε, R) = max n∗(f, ε) : f ∈ Cλ,n1 ,A ∩ BR = ∩ = min n n1 : λki ∞ i=n+1 ρ ε/[(Aρ − 1)1/ρR] Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 5/14
  10. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Optimality of

    Algorithm Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki ) n i=1 , n ε} COST(ALG, Cλ,n1 ,A , ε, R) = max n∗(f, ε) : f ∈ Cλ,n1 ,A ∩ BR = ∩ = min n n1 : λki ∞ i=n+1 ρ ε/[(Aρ − 1)1/ρR] COMP(A(Cλ,n1 ,A , ε, R)) = min{COST(ALG , Cλ,n1 ,A , ε, R) : ALG ∈ A(Cλ,n1 ,A )} COST(ALG, Cλ,n1 ,A , ωε, R), ω = 2A(Aρ − 1)1/ρ A − 1 > 1 via fooling functions ALG is essentially optimal Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 6/14
  11. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Tractability of

    Solving General Linear Problems on Cone of Inputs Cd,λd ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input For τ = ρ (ρ = ∞), the problem SOLd : Cd,λd ,n1 ,A → Gd is Strongly polynomial tractable iff there exist i0 ∈ N and η > 0 such that sup d∈N ∞ i=i0 λη d,ki < ∞ Polynomial tractable iff there exist η1 , η2 0 and η3 , K > 0 such that sup d∈N d−η1 ∞ i= Kdη2 λη3 d,ki < ∞ Weakly tractable iff sup d∈N exp(−cd) ∞ i=1 exp −c 1 λd,ki < ∞ for all c > 0 Analogous results exist for τ < ρ (ρ < ∞) Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 7/14
  12. Introduction Algorithm Optimality Tractability Alternate Cone Summary References An Alternate

    Cone, Similar to Our Cone f F := f(ki ) λki ∞ i=1 ρ , λki ↓ 0, f F := f(ki )ζki λki ∞ i=1 ρ , ζk1 = 1, ζki ↓ Cλ,n1 ,A := f ∈ F : f F A f(ki ) λki n1 i=1 ρ pilot sample bounds the norm of the input Cλ,ζ,A := f ∈ F : f F A f F stronger norm is bounded above by weaker norm Cλ,ζ,A ⊆ Cλ,n1 ,A for A A 1 + ζρ kn1+1 1 − (ζkn1+1 A )ρ 1/ρ , ζkn1+1 A < 1 Cλ,n1 ,A ⊆ Cλ,ζ,A for A A ζkn1 8/14
  13. Introduction Algorithm Optimality Tractability Alternate Cone Summary References An Alternative

    Cone, Similar to Our Cone For any f ∈ Cλ,ζ,A it follows that f ρ F − f(ki )ζki λki n1 i=1 ρ ρ = f(ki )ζki λki ∞ i=n1 +1 ρ ρ ζρ kn1+1 f(ki ) λki ∞ i=n1 +1 ρ ρ = ζρ kn1+1   f ρ F − f(ki ) λki n1 i=1 ρ ρ   ζρ kn1+1  A ρ f ρ F − f(ki )ζki λki n1 i=1 ρ ρ   9/14
  14. Introduction Algorithm Optimality Tractability Alternate Cone Summary References An Alternative

    Cone, Similar to Our Cone For any f ∈ Cλ,ζ,A it follows that f ρ F − f(ki )ζki λki n1 i=1 ρ ρ = f(ki )ζki λki ∞ i=n1 +1 ρ ρ ζρ kn1+1 f(ki ) λki ∞ i=n1 +1 ρ ρ = ζρ kn1+1   f ρ F − f(ki ) λki n1 i=1 ρ ρ   ζρ kn1+1  A ρ f ρ F − f(ki )ζki λki n1 i=1 ρ ρ   f ρ F 1 + ζρ kn1+1 1 − (ζkn1+1 A )ρ f(ki )ζki λki n1 i=1 ρ ρ f F A f F A 1 + ζρ kn1+1 1 − (ζkn1+1 A )ρ 1/ρ f(ki ) λki n1 i=1 ρ 9/14
  15. Introduction Algorithm Optimality Tractability Alternate Cone Summary References An Alternate

    Cone, Similar to Our Cone For any f ∈ Cλ,n1 ,A it follows that f F A f(ki ) λki n1 i=1 ρ A ζkn1 f(ki )ζki λki n1 i=1 ρ A ζkn1 f F 10/14
  16. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Conclusion Summary

    Adaptive algorithms can be constructed for non-convex, symmetric cones of inputs One possible cone assumes that a pilot sample tells you enough about the norm of the input Information cost and complexity depend on the norm of the input, but the adaptive algorithm does not know this norm There are tractability results for problems defined on cones Our algorithm can be extended to infer the λk in terms of coordinate weights Further Work Need to develop and analyze algorithms based on function values, not series coefficients Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). 11/14
  17. Introduction Algorithm Optimality Tractability Alternate Cone Summary References Cheng and

    Sandu Function Function values for data Chebyshev polynomial basis, λk inferred , Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets. 2017. https://www.sfu.ca/~ssurjano/index.html (2017). 12/14
  18. Thank you These slides are available at speakerdeck.com/fjhickernell/mcm-2019-pilot-sample

  19. Introduction Algorithm Optimality Tractability Alternate Cone Summary References References I

    H., F. J., Jiménez Rugama, & Li, D. in Contemporary Computational Mathematics — a celebration of the 80th birthday of Ian Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.) 597–619 (Springer-Verlag, 2018). Kunsch, R. J., Novak, E. & Rudolf, D. Solvable Integration Problems and Optimal Sample Size Selection. Journal of Complexity. To appear (2018). Ding, Y., H., F. J. & Jiménez Rugama, An Adaptive Algorithm Employing Continuous Linear Functionals. in Monte Carlo and Quasi-Monte Carlo Methods: MCQMC, Rennes, France, July 2018 (eds L’Ecuyer, P. & Tuffin, B.) Under revision (Springer-Verlag, Berlin, 2019+). Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for publication (DeGruyter, 2019+). Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets. 2017. https://www.sfu.ca/~ssurjano/index.html (2017). 14/14