Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MCM 2019 Pilot Sample

MCM 2019 Pilot Sample

Talk given at the MCM 2019 conference in Sydney as part of at a special session on current challenges in high dimensional algorithms.

Fred J. Hickernell

July 08, 2019
Tweet

More Decks by Fred J. Hickernell

Other Decks in Programming

Transcript

  1. An Optimal Adaptive Algorithm Based on a Pilot Sample
    Fred J. Hickernell
    Department of Applied Mathematics
    Center for Interdisciplinary Scientific Computation
    Illinois Institute of Technology
    [email protected] mypages.iit.edu/~hickernell
    with Yuhan Ding, Peter Kritzer, and Simon Mak
    partially supported by NSF-DMS-1522687 and NSF-DMS-1638521 (SAMSI)
    Twelfth International Conference on Monte Carlo Methods and Applications, July 8, 2019

    View full-size slide

  2. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Context
    Linear solution operator SOL : F → G, input space F contains functions defined on Ω ⊆ Rd
    e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary
    Approximation APP(f, n) =
    n
    i=1
    Li
    (f)gi
    , can sample linear functionals
    SOL(f) − APP(f, n)
    G
    SOL − APP(·, n) F→G
    f
    F
    Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε)
    G
    ε for all f ∈ C ⊂ F
    2/14

    View full-size slide

  3. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Context
    Linear solution operator SOL : F → G, input space F contains functions defined on Ω ⊆ Rd
    e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary
    Approximation APP(f, n) =
    n
    i=1
    Li
    (f)gi
    , can sample linear functionals
    SOL(f) − APP(f, n)
    G
    SOL − APP(·, n) F→G
    f
    F
    Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε)
    G
    ε for all f ∈ C ⊂ F
    Solvability: what C? how to determine n∗(f, ε) ∈ N?
    n∗(f, ε) = min{n : SOL − APP(·, n)
    F→G
    ε/R} if f ∈ of radius R
    Alternatively, assume f ∈ and bound f
    F
    or equivalent
    What you do not see is not much worse than what you see?
    H., F. J., Jiménez Rugama, & Li, D. in Contemporary Computational Mathematics — a celebration of the 80th birthday of Ian
    Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.) 597–619 (Springer-Verlag, 2018), Kunsch, R. J., Novak, E. & Rudolf, D.
    Solvable Integration Problems and Optimal Sample Size Selection. Journal of Complexity. To appear (2018), Ding, Y., H., F. J. &
    Jiménez Rugama, An Adaptive Algorithm Employing Continuous Linear Functionals. in Monte Carlo and Quasi-Monte Carlo
    Methods: MCQMC, Rennes, France, July 2018 (eds L’Ecuyer, P. & Tuffin, B.) Under revision (Springer-Verlag, Berlin, 2019+). 2/14

    View full-size slide

  4. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Context
    Linear solution operator SOL : F → G, input space F contains functions defined on Ω ⊆ Rd
    e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary
    Approximation APP(f, n) =
    n
    i=1
    Li
    (f)gi
    , can sample linear functionals
    SOL(f) − APP(f, n)
    G
    SOL − APP(·, n) F→G
    f
    F
    Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε)
    G
    ε for all f ∈ C ⊂ F
    Solvability: what C? how to determine n∗(f, ε) ∈ N?
    ×or
    Optimality: is n∗(f, ε) essentially as small as possible?
    Tractability: does n∗(f, ε) depend nicely on d?
    2/14

    View full-size slide

  5. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    General Linear Problems Defined on Series Spaces
    F :=



    f =

    i=1
    f(ki
    )uki
    : f
    F
    :=
    f(ki
    )
    λki

    i=1 ρ



    λk1
    λk2
    · · · > 0
    λ affects convergence rate &
    tractability
    G := g =

    i=1
    ^
    g(ki
    )vki
    : g
    G
    := ^
    g
    τ
    , SOL(f) =

    i=1
    f(ki
    )vki
    , τ ρ
    Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 3/14

    View full-size slide

  6. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Legendre and Chebyshev Bases for Function Approximation
    Legendre
    Chebyshev
    4/14

    View full-size slide

  7. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Adaptive Algorithm for General Linear Problems on Cone of Inputs
    F :=



    f =

    i=1
    f(ki
    )uki
    : f
    F
    :=
    f(ki
    )
    λki

    i=1 ρ



    λk1
    λk2
    · · · > 0
    λ affects convergence rate &
    tractability
    G := g =

    i=1
    ^
    g(ki
    )vki
    : g
    G
    := ^
    g
    τ
    , SOL(f) =

    i=1
    f(ki
    )vki
    , τ ρ
    Cλ,n1
    ,A
    := f ∈ F : f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 ρ
    pilot sample
    bounds the norm of the input
    APP(f, n) =
    n
    i=1
    f(ki
    )vki
    optimal for fixed n SOL(f) − APP(f, n)
    G
    ERR f(ki
    ) n
    i=1
    , n
    ERR f(ki
    ) n
    i=1
    , n :=

    Aρ
    f(ki
    )
    λki
    n1
    i=1
    ρ
    ρ

    f(ki
    )
    λki
    n
    i=1
    ρ
    ρ


    1/ρ
    upper bound on f− n
    i=1
    f(ki)uki F
    λki

    i=n+1 ρ
    SOL − APP(·,n) F→G
    1
    ρ
    +
    1
    ρ
    =
    1
    τ
    data-driven
    Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 5/14

    View full-size slide

  8. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Adaptive Algorithm for General Linear Problems on Cone of Inputs
    Cλ,n1
    ,A
    := f ∈ F : f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 ρ
    pilot sample
    bounds the norm of the input
    APP(f, n) =
    n
    i=1
    f(ki
    )vki
    optimal for fixed n SOL(f) − APP(f, n)
    G
    ERR f(ki
    ) n
    i=1
    , n
    ERR f(ki
    ) n
    i=1
    , n :=

    Aρ
    f(ki
    )
    λki
    n1
    i=1
    ρ
    ρ

    f(ki
    )
    λki
    n
    i=1
    ρ
    ρ


    1/ρ
    upper bound on f− n
    i=1
    f(ki)uki F
    λki

    i=n+1 ρ
    SOL − APP(·,n) F→G
    1
    ρ
    +
    1
    ρ
    =
    1
    τ
    data-driven
    ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki
    ) n
    i=1
    , n ε}
    Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 5/14

    View full-size slide

  9. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Adaptive Algorithm for General Linear Problems on Cone of Inputs
    Cλ,n1
    ,A
    := f ∈ F : f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 ρ
    pilot sample
    bounds the norm of the input
    APP(f, n) =
    n
    i=1
    f(ki
    )vki
    optimal for fixed n SOL(f) − APP(f, n)
    G
    ERR f(ki
    ) n
    i=1
    , n
    ERR f(ki
    ) n
    i=1
    , n :=

    Aρ
    f(ki
    )
    λki
    n1
    i=1
    ρ
    ρ

    f(ki
    )
    λki
    n
    i=1
    ρ
    ρ


    1/ρ
    upper bound on f− n
    i=1
    f(ki)uki F
    λki

    i=n+1 ρ
    SOL − APP(·,n) F→G
    1
    ρ
    +
    1
    ρ
    =
    1
    τ
    data-driven
    ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki
    ) n
    i=1
    , n ε}
    COST(ALG, Cλ,n1
    ,A
    , ε, R) = max n∗(f, ε) : f ∈ Cλ,n1
    ,A ∩ BR
    = ∩
    = min n n1
    : λki

    i=n+1 ρ
    ε/[(Aρ − 1)1/ρR]
    Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 5/14

    View full-size slide

  10. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Optimality of Algorithm
    Cλ,n1
    ,A
    := f ∈ F : f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 ρ
    pilot sample
    bounds the norm of the input
    ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki
    ) n
    i=1
    , n ε}
    COST(ALG, Cλ,n1
    ,A
    , ε, R) = max n∗(f, ε) : f ∈ Cλ,n1
    ,A ∩ BR
    = ∩
    = min n n1
    : λki

    i=n+1 ρ
    ε/[(Aρ − 1)1/ρR]
    COMP(A(Cλ,n1
    ,A
    , ε, R)) = min{COST(ALG , Cλ,n1
    ,A
    , ε, R) : ALG ∈ A(Cλ,n1
    ,A
    )}
    COST(ALG, Cλ,n1
    ,A
    , ωε, R), ω =
    2A(Aρ − 1)1/ρ
    A − 1
    > 1
    via fooling functions
    ALG is essentially optimal
    Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 6/14

    View full-size slide

  11. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Tractability of Solving General Linear Problems on Cone of Inputs
    Cd,λd
    ,n1
    ,A
    := f ∈ F : f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 ρ
    pilot sample
    bounds the norm of the input
    For τ = ρ (ρ = ∞), the problem SOLd
    : Cd,λd
    ,n1
    ,A
    → Gd
    is
    Strongly polynomial tractable iff there exist i0
    ∈ N and η > 0 such that sup
    d∈N

    i=i0
    λη
    d,ki
    < ∞
    Polynomial tractable iff there exist η1
    , η2
    0 and η3
    , K > 0 such that sup
    d∈N
    d−η1

    i= Kdη2
    λη3
    d,ki
    < ∞
    Weakly tractable iff sup
    d∈N
    exp(−cd)

    i=1
    exp −c
    1
    λd,ki
    < ∞ for all c > 0
    Analogous results exist for τ < ρ (ρ < ∞)
    Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 7/14

    View full-size slide

  12. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    An Alternate Cone, Similar to Our Cone
    f
    F
    :=
    f(ki
    )
    λki

    i=1 ρ
    , λki
    ↓ 0, f
    F
    :=
    f(ki
    )ζki
    λki

    i=1 ρ
    , ζk1
    = 1, ζki

    Cλ,n1
    ,A
    := f ∈ F : f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 ρ
    pilot sample bounds the norm of the input
    Cλ,ζ,A
    := f ∈ F : f
    F
    A f
    F
    stronger norm is bounded above by weaker norm
    Cλ,ζ,A
    ⊆ Cλ,n1
    ,A
    for A A
    1 + ζρ
    kn1+1
    1 − (ζkn1+1
    A )ρ
    1/ρ
    , ζkn1+1
    A < 1
    Cλ,n1
    ,A ⊆ Cλ,ζ,A
    for A A ζkn1
    8/14

    View full-size slide

  13. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    An Alternative Cone, Similar to Our Cone
    For any f ∈ Cλ,ζ,A
    it follows that
    f ρ
    F

    f(ki
    )ζki
    λki
    n1
    i=1
    ρ
    ρ
    =
    f(ki
    )ζki
    λki

    i=n1
    +1
    ρ
    ρ
    ζρ
    kn1+1
    f(ki
    )
    λki

    i=n1
    +1
    ρ
    ρ
    = ζρ
    kn1+1

     f ρ
    F

    f(ki
    )
    λki
    n1
    i=1
    ρ
    ρ


    ζρ
    kn1+1

    A ρ f ρ
    F

    f(ki
    )ζki
    λki
    n1
    i=1
    ρ
    ρ


    9/14

    View full-size slide

  14. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    An Alternative Cone, Similar to Our Cone
    For any f ∈ Cλ,ζ,A
    it follows that
    f ρ
    F

    f(ki
    )ζki
    λki
    n1
    i=1
    ρ
    ρ
    =
    f(ki
    )ζki
    λki

    i=n1
    +1
    ρ
    ρ
    ζρ
    kn1+1
    f(ki
    )
    λki

    i=n1
    +1
    ρ
    ρ
    = ζρ
    kn1+1

     f ρ
    F

    f(ki
    )
    λki
    n1
    i=1
    ρ
    ρ


    ζρ
    kn1+1

    A ρ f ρ
    F

    f(ki
    )ζki
    λki
    n1
    i=1
    ρ
    ρ


    f ρ
    F
    1 + ζρ
    kn1+1
    1 − (ζkn1+1
    A )ρ
    f(ki
    )ζki
    λki
    n1
    i=1
    ρ
    ρ
    f
    F
    A f
    F
    A
    1 + ζρ
    kn1+1
    1 − (ζkn1+1
    A )ρ
    1/ρ
    f(ki
    )
    λki
    n1
    i=1 ρ
    9/14

    View full-size slide

  15. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    An Alternate Cone, Similar to Our Cone
    For any f ∈ Cλ,n1
    ,A
    it follows that
    f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 ρ
    A
    ζkn1
    f(ki
    )ζki
    λki
    n1
    i=1 ρ
    A
    ζkn1
    f
    F
    10/14

    View full-size slide

  16. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Conclusion
    Summary
    Adaptive algorithms can be constructed for non-convex, symmetric cones of inputs
    One possible cone assumes that a pilot sample tells you enough about the norm of the input
    Information cost and complexity depend on the norm of the input, but the adaptive algorithm does
    not know this norm
    There are tractability results for problems defined on cones
    Our algorithm can be extended to infer the λk
    in terms of coordinate weights
    Further Work
    Need to develop and analyze algorithms based on function values, not series coefficients
    Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 11/14

    View full-size slide

  17. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    Cheng and Sandu Function
    Function values for data Chebyshev polynomial basis, λk
    inferred ,
    Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets. 2017.
    https://www.sfu.ca/~ssurjano/index.html (2017). 12/14

    View full-size slide

  18. Thank you
    These slides are available at
    speakerdeck.com/fjhickernell/mcm-2019-pilot-sample

    View full-size slide

  19. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
    References I
    H., F. J., Jiménez Rugama, & Li, D. in Contemporary Computational Mathematics — a celebration
    of the 80th birthday of Ian Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.) 597–619
    (Springer-Verlag, 2018).
    Kunsch, R. J., Novak, E. & Rudolf, D. Solvable Integration Problems and Optimal Sample Size
    Selection. Journal of Complexity. To appear (2018).
    Ding, Y., H., F. J. & Jiménez Rugama, An Adaptive Algorithm Employing Continuous Linear
    Functionals. in Monte Carlo and Quasi-Monte Carlo Methods: MCQMC, Rennes, France, July 2018
    (eds L’Ecuyer, P. & Tuffin, B.) Under revision (Springer-Verlag, Berlin, 2019+).
    Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. &
    Kritzer, P.) Submitted for publication (DeGruyter, 2019+).
    Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets. 2017.
    https://www.sfu.ca/~ssurjano/index.html (2017).
    14/14

    View full-size slide