Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CompMath Stat 2019 Aug

CompMath Stat 2019 Aug

Presentation given in my research seminar. Slightly revised version of LANL talk

Fred J. Hickernell

August 30, 2019
Tweet

More Decks by Fred J. Hickernell

Other Decks in Research

Transcript

  1. The Challenges of Approximating Functions of Many Variables
    Fred J. Hickernell
    Department of Applied Mathematics
    Center for Interdisciplinary Scientific Computation
    Illinois Institute of Technology
    [email protected] mypages.iit.edu/~hickernell speakerdeck.com/fjhickernell
    Joint work with Yuhan Ding, Peter Kritzer, and Simon Mak
    This work partially supported by NSF-DMS-1522687 and NSF-DMS-1638521 (SAMSI)
    Computational Mathematics and Statistics Seminar, August 30, 2019

    View full-size slide

  2. Introduction Solvability Smoothness Tractability Cones Design Example References
    Highlights
    Goal: Construct ALG such that given a black box providing information about f : Ω ⊂ Rd → R
    f − ALG(f, ε)
    G
    ε ∀ε > 0, f ∈ H ⊆ F (Banach space)
    Impossible for infinite dimensional Banach space H = F, so what is H?
    Smoothness assumed by F speeds up ALG, not surprising
    Smoothness alone cannot save from the curse of dimensionality, but a low effective-dimension
    structure can
    Choosing H to be a cone , rather than a ball , paves the way for adaptive algorithms
    Interesting design (where to sample) problems remain
    2/18

    View full-size slide

  3. Introduction Solvability Smoothness Tractability Cones Design Example References
    Problem
    Input
    Black box providing noiseless information about f : Ω ⊆ Rd → R
    e.g., function values or series coefficients, costing $(f) each
    Error tolerance ε
    Output ALG(f, ε) (as a surrogate, for solving PDEs, for uncertainty quantification) that is
    Cheap to evaluate and manipulate
    Accurate f − ALG(f, ε)
    G
    ε ∀ε > 0
    Efficient to construct
    3/18

    View full-size slide

  4. Introduction Solvability Smoothness Tractability Cones Design Example References
    Problem
    Input
    Black box providing noiseless information about f : Ω ⊆ Rd → R
    e.g., function values or series coefficients, costing $(f) each
    Error tolerance ε
    Output ALG(f, ε) that is
    Cheap to evaluate and manipulate
    Accurate f − ALG(f, ε)
    G
    ε ∀ε > 0
    Efficient to construct
    Approximation with fixed computation budget: APP(f, n) =
    n
    i=1
    Li
    (f)gi,n
    L1
    (f), L2
    (f), . . . is input function information, e.g., function values or series coefficients
    gn
    = (g1,n
    , . . . , gn,n
    ) ∈ Gn cardinal functions
    COST(f, n) = O(n$(f) + COST(gn
    ))
    Algorithm ALG(f, ε) = APP(f, n∗(f, ε)) satisfying f − APP(f, n∗(f, ε))
    G
    ε ∀ε > 0
    COST(f, ε) = COST(f, n∗(f, ε)) + cost to determine n∗(f, ε)
    3/18

    View full-size slide

  5. Introduction Solvability Smoothness Tractability Cones Design Example References
    Problem
    Input
    Black box providing noiseless information about f : Ω ⊆ Rd → R costing $(f) each
    f ∈ F, definition of · F
    enshrines smoothness assumptions
    Error tolerance ε
    Output ALG(f, ε) that is
    Cheap to evaluate and manipulate
    Accurate f − ALG(f, ε)
    G
    ε ∀ε > 0, f ∈ H ⊂ F, provably
    Efficient to construct
    Approximation with fixed computation budget: APP(f, n) =
    n
    i=1
    Li
    (f)gi,n
    L1
    (f), L2
    (f), . . . is input function information
    gn
    = (g1,n
    , . . . , gn,n
    ) ∈ Gn cardinal functions
    COST(f, n) = O(n$(f) + COST(gn
    ))
    Algorithm ALG(f, ε) = APP(f, n∗(f, ε)) satisfying f − APP(f, n∗(f, ε))
    G
    ε ∀ε > 0, f ∈ H ⊂ F
    COST(f, ε) = COST(f, n∗(f, ε)) + cost to determine n∗(f, ε)
    3/18

    View full-size slide

  6. Introduction Solvability Smoothness Tractability Cones Design Example References
    Impossible for All f in Infinite Dimensional F
    f − ALG(f, ε)
    G
    ε ∀f ∈ H ⊂ F
    Proof by contradiction
    Suppose H = F
    Fix ε > 0
    Let L1
    , . . . , Ln
    be the linear information used to construct ALG(0, ε)
    Choose nonzero fooling function f ∈ F, such that L1
    (f) = · · · = Ln
    (f) = 0
    ALG(±cf, ε) = ALG(0, ε) for all c > 0
    For all c > 0
    ε max cf − ALG(cf, ε)
    G
    , −cf − ALG(−cf, ε)
    G
    1
    2
    cf − ALG(cf, ε)
    G
    + −cf − ALG(−cf, ε)
    G
    1
    2
    cf − ALG(0, ε)
    G
    + cf + ALG(0, ε)
    G
    c f
    G
    =⇒⇐=
    4/18

    View full-size slide

  7. Introduction Solvability Smoothness Tractability Cones Design Example References
    Smoothness Makes Algorithm Less Expensive
    For d = 1, let {u0
    , u1
    , . . .} be an orthogonal (polynomial) basis for F and G
    F := f =

    k=0
    f(k)uk
    : f
    F
    :=
    f(k)
    λk

    k=0 2
    < ∞ , λ0
    λ1 · · · > 0
    G := g =

    k=0
    ^
    g(k)uk
    : g
    G
    := ^
    g(k) ∞
    k=0 2
    < ∞ , APP(f, n) =
    n−1
    k=0
    f(k)uk
    5/18

    View full-size slide

  8. Introduction Solvability Smoothness Tractability Cones Design Example References
    Bases for Function Approximation
    Legendre
    Chebyshev
    Sine and Cosine
    6/18

    View full-size slide

  9. Introduction Solvability Smoothness Tractability Cones Design Example References
    Smoothness Makes Algorithm Less Expensive
    For d = 1, let {u0
    , u1
    , . . .} be an orthogonal (polynomial) basis for F and G
    F := f =

    k=0
    f(k)uk
    : f
    F
    :=
    f(k)
    λk

    k=0 2
    < ∞ , λ0
    λ1 · · · > 0
    G := g =

    k=0
    ^
    g(k)uk
    : g
    G
    := ^
    g(k) ∞
    k=0 2
    < ∞ , APP(f, n) =
    n−1
    k=0
    f(k)uk
    f − APP(f, n)
    G
    = f(k) ∞
    k=n 2
    =
    λk
    f(k)
    λk

    k=n 2 tight
    f
    F
    λn
    ?
    ε
    require λn ↓ 0
    smoothness of F
    > smoothness of G
    7/18

    View full-size slide

  10. Introduction Solvability Smoothness Tractability Cones Design Example References
    Smoothness Makes Algorithm Less Expensive
    For d = 1, let {u0
    , u1
    , . . .} be an orthogonal (polynomial) basis for F and G
    F := f =

    k=0
    f(k)uk
    : f
    F
    :=
    f(k)
    λk

    k=0 2
    < ∞ , λ0
    λ1 · · · > 0
    G := g =

    k=0
    ^
    g(k)uk
    : g
    G
    := ^
    g(k) ∞
    k=0 2
    < ∞ , APP(f, n) =
    n−1
    k=0
    f(k)uk
    f − APP(f, n)
    G
    = f(k) ∞
    k=n 2
    =
    λk
    f(k)
    λk

    k=n 2 tight
    f
    F
    λn
    ?
    ε
    require λn ↓ 0
    smoothness of F
    > smoothness of G
    By choosing H = BR
    := {f ∈ F : f
    F
    R} , we can define our algorithm
    ALG(f, ε) = APP(f, n∗) & n∗ = min{n : λn
    ε/R} =⇒ f − ALG(f, ε)
    G
    ε ∀f ∈ BR
    λn
    = O(n−1/p) =⇒ COST(BR
    , ε) = O(Rpε−p)
    7/18

    View full-size slide

  11. Introduction Solvability Smoothness Tractability Cones Design Example References
    Smoothness Makes Algorithm Less Expensive
    For d = 1, let {u0
    , u1
    , . . .} be an orthogonal (polynomial) basis for F and G
    F := f =

    k=0
    f(k)uk
    : f
    F
    :=
    f(k)
    λk

    k=0 2
    < ∞ , λ0
    λ1 · · · > 0
    G := g =

    k=0
    ^
    g(k)uk
    : g
    G
    := ^
    g(k) ∞
    k=0 2
    < ∞ , APP(f, n) =
    n−1
    k=0
    f(k)uk
    f − APP(f, n)
    G
    = f(k) ∞
    k=n 2
    =
    λk
    f(k)
    λk

    k=n 2 tight
    f
    F
    λn
    ?
    ε
    require λn ↓ 0
    smoothness of F
    > smoothness of G
    By choosing H = BR
    := {f ∈ F : f
    F
    R} , we can define our algorithm
    ALG(f, ε) = APP(f, n∗) & n∗ = min{n : λn
    ε/R} =⇒ f − ALG(f, ε)
    G
    ε ∀f ∈ BR
    λn
    = O(n−1/p) =⇒ COST(BR
    , ε) = O(Rpε−p)
    ALG has optimal cost among all successful algorithms using Fourier coefficients
    (look at the cost of approximating the zero function)
    7/18

    View full-size slide

  12. Introduction Solvability Smoothness Tractability Cones Design Example References
    Smoothness Makes Algorithm Less Expensive
    For d = 1, let {u0
    , u1
    , . . .} be an orthogonal (polynomial) basis for F and G
    F := f =

    k=0
    f(k)uk
    : f
    F
    :=
    f(k)
    λk

    k=0 2
    < ∞ , λ0
    λ1 · · · > 0
    G := g =

    k=0
    ^
    g(k)uk
    : g
    G
    := ^
    g(k) ∞
    k=0 2
    < ∞ , APP(f, n) =
    n−1
    k=0
    f(k)uk
    f − APP(f, n)
    G
    = f(k) ∞
    k=n 2
    =
    λk
    f(k)
    λk

    k=n 2 tight
    f
    F
    λn
    ?
    ε
    require λn ↓ 0
    smoothness of F
    > smoothness of G
    By choosing H = BR
    := {f ∈ F : f
    F
    R} , we can define our algorithm
    ALG(f, ε) = APP(f, n∗) & n∗ = min{n : λn
    ε/R} =⇒ f − ALG(f, ε)
    G
    ε ∀f ∈ BR
    λn
    = O(n−1/p) =⇒ COST(BR
    , ε) = O(Rpε−p)
    ALG has optimal cost among all successful algorithms using Fourier coefficients
    (look at the cost of approximating the zero function)
    Similar results for algorithms based on function values, but need to choose the design carefully
    7/18

    View full-size slide

  13. Introduction Solvability Smoothness Tractability Cones Design Example References
    Smoothness Cannot Save You from the Curse of Dimensionality1
    For arbitrary d, let {u0
    = 1, u1
    } be used to construct a product basis F and G (multlinear functions)
    F :=



    f(x) =
    k∈{0,1}d
    f(k)uk : f
    F
    :=
    f(k)
    λk
    k∈{0,1}d
    2
    < ∞



    , uk(x) :=
    d
    =1
    uk
    (x )
    G :=



    g =
    k∈{0,1}d
    ^
    g(k)uk : g
    G
    := ^
    g(k)
    k∈{0,1}d
    2
    < ∞



    , λk :=
    d
    =1
    k =0
    s = s k 0
    APP(f, n) =
    n
    i=1
    f(ki
    )uki
    , λk1
    = 1 s = λk2
    · · · sd
    1Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information. EMS Tracts in Mathematics
    6 (European Mathematical Society, Zürich, 2008). 8/18

    View full-size slide

  14. Introduction Solvability Smoothness Tractability Cones Design Example References
    Bases for Function Approximation
    Legendre
    Chebyshev
    Sine and Cosine
    9/18

    View full-size slide

  15. Introduction Solvability Smoothness Tractability Cones Design Example References
    Smoothness Cannot Save You from the Curse of Dimensionality1
    For arbitrary d, let {u0
    = 1, u1
    } be used to construct a product basis F and G (multlinear functions)
    F :=



    f(x) =
    k∈{0,1}d
    f(k)uk : f
    F
    :=
    f(k)
    λk
    k∈{0,1}d
    2
    < ∞



    , uk(x) :=
    d
    =1
    uk
    (x )
    G :=



    g =
    k∈{0,1}d
    ^
    g(k)uk : g
    G
    := ^
    g(k)
    k∈{0,1}d
    2
    < ∞



    , λk :=
    d
    =1
    k =0
    s = s k 0
    APP(f, n) =
    n
    i=1
    f(ki
    )uki
    , λk1
    = 1 s = λk2
    · · · sd
    ALG(f, ε) = APP(f, n∗) & n∗ = min{n : λkn+1
    ε/R} =⇒ f − ALG(f, ε)
    G
    ε ∀f ∈ BR
    λkn
    = O n−1/pespd/p =⇒ COST(BR
    , ε) = O Rpε−pespd ∀p exponential growth in d
    1Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information. EMS Tracts in Mathematics
    6 (European Mathematical Society, Zürich, 2008). 10/18

    View full-size slide

  16. Introduction Solvability Smoothness Tractability Cones Design Example References
    Proof that λkn+1
    = O n−1/pespd/p ∀p > 0
    λp
    kn+1
    1
    n
    λp
    k1
    + · · · + λp
    kn
    λki
    are ordered
    λkn+1
    1
    n1/p
    λp
    k1
    + · · · + λp
    kn
    1/p
    pth root
    1
    n1/p
    λp
    k1
    + · · · + λp
    k
    2d
    1/p
    add the rest in
    1
    n1/p
    1 + sp d/p
    binomial theorem
    espd/p
    n1/p
    1 + x ex for x 0
    There is a similar proof that provides a lower bound on λkn+1
    11/18

    View full-size slide

  17. Introduction Solvability Smoothness Tractability Cones Design Example References
    Coordinate Weights Can Save You1
    For arbitrary d, let {u0
    = 1, u1
    } be used to construct a product basis F and G (multlinear functions)
    F :=



    f(x) =
    k∈{0,1}d
    f(k)uk : f
    F
    :=
    f(k)
    λk
    k∈{0,1}d
    2
    < ∞



    , uk(x) :=
    d
    =1
    uk
    (x )
    G :=



    g =
    k∈{0,1}d
    ^
    g(k)uk : g
    G
    := ^
    g(k)
    k∈{0,1}d
    2
    < ∞



    , λk :=
    d
    =1
    k =0
    w s
    APP(f, n) =
    n
    i=1
    f(ki
    )uki
    , λk1
    = 1 w1
    s = λk2
    · · · , 1 = w1
    w2 · · ·
    ALG(f, ε) = APP(f, n∗) & n∗ = min{n : λkn+1
    ε/R} =⇒ f − ALG(f, ε)
    G
    ε ∀f ∈ BR
    λkn
    = O n−1/p exp p−1sp d
    =1
    wp =⇒ COST(BR
    , ε) = O Rpε−p exp sp d
    =1
    wp ∀p
    cost is independent of d if coordinate weights decay quickly
    1Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information. EMS Tracts in Mathematics
    6 (European Mathematical Society, Zürich, 2008). 12/18

    View full-size slide

  18. Introduction Solvability Smoothness Tractability Cones Design Example References
    Coordinate Weights Can Save You, Even with Higher Order Polynomials1
    For arbitrary d, let {u0
    = 1, u1
    , u2
    , . . .} be used to construct a product basis F and G
    F :=



    f(x) =
    k∈Nd
    0
    f(k)uk : f
    F
    :=
    f(k)
    λk
    k∈Nd
    0 2
    < ∞



    , uk(x) :=
    d
    =1
    uk
    (x )
    G :=



    g =
    k∈Nd
    0
    ^
    g(k)uk : g
    G
    := ^
    g(k)
    k∈Nd
    0 2
    < ∞



    , λk :=
    d
    =1
    k =0
    w sk
    APP(f, n) =
    n
    i=1
    f(ki
    )uki
    , λk1
    = 1 λk2
    · · · , 1 = w1
    w2 · · ·
    ALG(f, ε) = APP(f, n∗) & n∗ = min{n : λkn+1
    ε/R} =⇒ f − ALG(f, ε)
    G
    ε ∀f ∈ BR
    λkn
    = O n−1/p exp p−1 ∞
    k=1
    sp
    k
    d
    =1
    wp =⇒ COST(BR
    , ε) = O Rpε−p exp ∞
    k=1
    sp
    k
    d
    =1
    wp ∀p
    cost is independent of d if coordinate and smoothness weights decay quickly
    1Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information. EMS Tracts in Mathematics
    6 (European Mathematical Society, Zürich, 2008). 12/18

    View full-size slide

  19. Introduction Solvability Smoothness Tractability Cones Design Example References
    Look to Cones for Adaptive Algorithms
    Goal: Construct ALG such that given a black box providing information about f : Ω ⊂ Rd → R
    f − ALG(f, ε)
    G
    ε ∀ε > 0, f ∈ H ⊆ F (Banach space)
    So far, H = BR
    Hard to know a priori how large R should be for your problem
    Computational cost depends on R and ε, but not on f data
    Choosing H = makes adaptive algorithms possible2
    2H., F. J., Jiménez Rugama, Ll. A. & Li, D. in Contemporary Computational Mathematics — a celebration of the 80th birthday
    of Ian Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.) 597–619 (Springer-Verlag, 2018). doi:10.1007/978-3-319-72456-0,
    Kunsch, R. J., Novak, E. & Rudolf, D. Solvable Integration Problems and Optimal Sample Size Selection. Journal of Complexity.
    To appear (2018), Ding, Y., H., F. J. & Jiménez Rugama, Ll. A. An Adaptive Algorithm Employing Continuous Linear Functionals.
    in Monte Carlo and Quasi-Monte Carlo Methods: MCQMC, Lille, France, July 2018 (eds L’Ecuyer, P. & Tuffin, B.) Under revision
    (Springer-Verlag, Berlin, 2019+), Jagadeeswaran, R. & H., F. J. Fast Automatic Bayesian Cubature Using Lattice Sampling. Stat.
    Comput. Under revision (2019+). 13/18

    View full-size slide

  20. Introduction Solvability Smoothness Tractability Cones Design Example References
    Adaptive Algorithm for Cone of Inputs Based on Pilot Sample3
    F := f =

    i=1
    f(ki
    )uki
    : f
    F
    :=
    f(ki
    )
    λki

    i=1 2
    λk1
    λk2
    · · · > 0
    λ affects convergence rate &
    tractability
    G := g =

    i=1
    ^
    g(ki
    )uki
    : g
    G
    := ^
    g
    2
    , APP(f, n) =
    n
    i=1
    f(ki
    )uki
    3Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 14/18

    View full-size slide

  21. Introduction Solvability Smoothness Tractability Cones Design Example References
    Adaptive Algorithm for Cone of Inputs Based on Pilot Sample3
    F := f =

    i=1
    f(ki
    )uki
    : f
    F
    :=
    f(ki
    )
    λki

    i=1 2
    λk1
    λk2
    · · · > 0
    λ affects convergence rate &
    tractability
    G := g =

    i=1
    ^
    g(ki
    )uki
    : g
    G
    := ^
    g
    2
    , APP(f, n) =
    n
    i=1
    f(ki
    )uki
    Cd,λ,n1,A
    := f ∈ F : f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 2
    pilot sample bounds the norm of the input
    A is inflation factor, n1
    is initial sample size
    f − APP(f, n)
    G

    A2
    f(ki
    )
    λki
    n1
    i=1
    2
    2

    f(ki
    )
    λki
    n
    i=1
    2
    2


    1/2
    upper bound on f− n
    i=1
    f(ki)uki F
    λkn+1
    =: ERR f(ki
    ) n
    i=1
    , n
    data-driven
    3Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 14/18

    View full-size slide

  22. Introduction Solvability Smoothness Tractability Cones Design Example References
    Adaptive Algorithm for Cone of Inputs Based on Pilot Sample3
    F := f =

    i=1
    f(ki
    )uki
    : f
    F
    :=
    f(ki
    )
    λki

    i=1 2
    λk1
    λk2
    · · · > 0
    λ affects convergence rate &
    tractability
    G := g =

    i=1
    ^
    g(ki
    )uki
    : g
    G
    := ^
    g
    2
    , APP(f, n) =
    n
    i=1
    f(ki
    )uki
    Cd,λ,n1,A
    := f ∈ F : f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 2
    pilot sample bounds the norm of the input
    A is inflation factor, n1
    is initial sample size
    f − APP(f, n)
    G

    A2
    f(ki
    )
    λki
    n1
    i=1
    2
    2

    f(ki
    )
    λki
    n
    i=1
    2
    2


    1/2
    upper bound on f− n
    i=1
    f(ki)uki F
    λkn+1
    =: ERR f(ki
    ) n
    i=1
    , n
    data-driven
    ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki
    ) n
    i=1
    , n ε}
    3Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+). 14/18

    View full-size slide

  23. Introduction Solvability Smoothness Tractability Cones Design Example References
    Adaptive Algorithm for Cone of Inputs Based on Pilot Sample
    F := f =

    i=1
    f(ki
    )uki
    : f
    F
    :=
    f(ki
    )
    λki

    i=1 2
    λk1
    λk2
    · · · > 0
    λ affects convergence rate &
    tractability
    G := g =

    i=1
    ^
    g(ki
    )uki
    : g
    G
    := ^
    g
    2
    , APP(f, n) =
    n
    i=1
    f(ki
    )uki
    Cd,λ,n1,A
    := f ∈ F : f
    F
    A
    f(ki
    )
    λki
    n1
    i=1 2
    pilot sample bounds the norm of the input
    A is inflation factor, n1
    is initial sample size
    f − APP(f, n)
    G

    A2
    f(ki
    )
    λki
    n1
    i=1
    2
    2

    f(ki
    )
    λki
    n
    i=1
    2
    2


    1/2
    upper bound on f− n
    i=1
    f(ki)uki F
    λkn+1
    =: ERR f(ki
    ) n
    i=1
    , n
    data-driven
    ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki
    ) n
    i=1
    , n ε}
    COST(ALG, Cd,λ,n1,A
    , ε, R) = max n∗(f, ε) : f ∈ Cλ,n1,A ∩ BR
    = ∩
    = min n n1
    : λkn+1
    ε/[(A2 − 1)1/2R]
    ALG is essentially optimal; computational cost is d independent if λki
    decay quickly 14/18

    View full-size slide

  24. Introduction Solvability Smoothness Tractability Cones Design Example References
    Challenges When Using Function Values as Information
    Goal: Construct ALG such that given a black box providing information about f : Ω ⊂ Rd → R
    f − ALG(f, ε)
    G
    ε ∀ε > 0, f ∈ H ⊆ F (Banach space)
    So far, the function information is series coefficients
    COST(f, ε) = O n∗(f, ε) $(f) , the best one can hope for
    Cost of constructing the approximation and determining the stopping sample size is essentially the same
    as getting the data
    But using series coefficients is not so realistic
    Developing theory for multivariate function approximation using function values is challenging
    One must bound the aliasing effects of using interpolation or other means to approximate the coefficients
    Interpolation, reproducing kernel Hilbert space methods, and kriging typically require O(n3) operations
    to compute approximation, perhaps more if one is tuning the parameters of the kernels; but there are
    efforts to speed this up3
    Space filling designs such as integration lattices4, digital nets5, and sparse grids6 are promising
    3Schäfer, F., Sullivan, T. J. & Owhadi, H. Compression, inversion, and approximate PCA of dense kernel matrices at
    near-linear computational complexity. arXiv:1706.02205. 2019+.
    4Dick, J., Kuo, F. & Sloan, I. H. High dimensional integration — the Quasi-Monte Carlo way. Acta Numer. 22, 133–288 (2013).
    5Dick, J. & Pillichshammer, F. Digital Nets and Sequences: Discrepancy Theory and Quasi-Monte Carlo Integration.
    (Cambridge University Press, Cambridge, 2010).
    6Bungartz, H.-J. & Griebel, M. Sparse grids. Acta Numer. 1–123 (2004). 15/18

    View full-size slide

  25. Introduction Solvability Smoothness Tractability Cones Design Example References
    Cheng and Sandu Function7
    Chebyshev polynomials, Coordinate weights w inferred, Smoothness weights sk
    inferred
    function values used
    7Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
    publication (DeGruyter, 2019+), Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets.
    2017. https://www.sfu.ca/~ssurjano/index.html (2017). 16/18

    View full-size slide

  26. Introduction Solvability Smoothness Tractability Cones Design Example References
    Highlights
    Goal: Construct ALG such that given a black box providing information about f : Ω ⊂ Rd → R
    f − ALG(f, ε)
    G
    ε ∀ε > 0, f ∈ H ⊆ F (Banach space)
    Impossible for infinite dimensional Banach space H = F, so what is H?
    Smoothness assumed by F speeds up ALG, not surprising
    Smoothness alone cannot save from the curse of dimensionality, but a low effective-dimension
    structure can
    Choosing H to be a cone , rather than a ball , paves the way for adaptive algorithms
    Interesting design (where to sample) problems remain
    17/18

    View full-size slide

  27. Thank you
    These slides are available at
    speakerdeck.com/fjhickernell/compmath-stat-2019-aug

    View full-size slide

  28. Introduction Solvability Smoothness Tractability Cones Design Example References
    Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information.
    EMS Tracts in Mathematics 6 (European Mathematical Society, Zürich, 2008).
    H., F. J., Jiménez Rugama, Ll. A. & Li, D. in Contemporary Computational Mathematics — a
    celebration of the 80th birthday of Ian Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.)
    597–619 (Springer-Verlag, 2018). doi:10.1007/978-3-319-72456-0.
    Kunsch, R. J., Novak, E. & Rudolf, D. Solvable Integration Problems and Optimal Sample Size
    Selection. Journal of Complexity. To appear (2018).
    Ding, Y., H., F. J. & Jiménez Rugama, Ll. A. An Adaptive Algorithm Employing Continuous Linear
    Functionals. in Monte Carlo and Quasi-Monte Carlo Methods: MCQMC, Lille, France, July 2018
    (eds L’Ecuyer, P. & Tuffin, B.) Under revision (Springer-Verlag, Berlin, 2019+).
    Jagadeeswaran, R. & H., F. J. Fast Automatic Bayesian Cubature Using Lattice Sampling. Stat.
    Comput. Under revision (2019+).
    Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. &
    Kritzer, P.) Submitted for publication (DeGruyter, 2019+).
    18/18

    View full-size slide

  29. Introduction Solvability Smoothness Tractability Cones Design Example References
    Schäfer, F., Sullivan, T. J. & Owhadi, H. Compression, inversion, and approximate PCA of dense
    kernel matrices at near-linear computational complexity. arXiv:1706.02205. 2019+.
    Dick, J., Kuo, F. & Sloan, I. H. High dimensional integration — the Quasi-Monte Carlo way. Acta
    Numer. 22, 133–288 (2013).
    Dick, J. & Pillichshammer, F. Digital Nets and Sequences: Discrepancy Theory and Quasi-Monte
    Carlo Integration. (Cambridge University Press, Cambridge, 2010).
    Bungartz, H.-J. & Griebel, M. Sparse grids. Acta Numer. 1–123 (2004).
    Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets. 2017.
    https://www.sfu.ca/~ssurjano/index.html (2017).
    18/18

    View full-size slide