Fred J. Hickernell
July 08, 2019
35

# MCM 2019 Pilot Sample

Talk given at the MCM 2019 conference in Sydney as part of at a special session on current challenges in high dimensional algorithms.

July 08, 2019

## Transcript

1. An Optimal Adaptive Algorithm Based on a Pilot Sample
Fred J. Hickernell
Department of Applied Mathematics
Center for Interdisciplinary Scientiﬁc Computation
Illinois Institute of Technology
[email protected] mypages.iit.edu/~hickernell
with Yuhan Ding, Peter Kritzer, and Simon Mak
partially supported by NSF-DMS-1522687 and NSF-DMS-1638521 (SAMSI)
Twelfth International Conference on Monte Carlo Methods and Applications, July 8, 2019

2. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Context
Linear solution operator SOL : F → G, input space F contains functions deﬁned on Ω ⊆ Rd
e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary
Approximation APP(f, n) =
n
i=1
Li
(f)gi
, can sample linear functionals
SOL(f) − APP(f, n)
G
SOL − APP(·, n) F→G
f
F
Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε)
G
ε for all f ∈ C ⊂ F
2/14

3. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Context
Linear solution operator SOL : F → G, input space F contains functions deﬁned on Ω ⊆ Rd
e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary
Approximation APP(f, n) =
n
i=1
Li
(f)gi
, can sample linear functionals
SOL(f) − APP(f, n)
G
SOL − APP(·, n) F→G
f
F
Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε)
G
ε for all f ∈ C ⊂ F
Solvability: what C? how to determine n∗(f, ε) ∈ N?
n∗(f, ε) = min{n : SOL − APP(·, n)
F→G
ε/R} if f ∈ of radius R
Alternatively, assume f ∈ and bound f
F
or equivalent
What you do not see is not much worse than what you see?
H., F. J., Jiménez Rugama, & Li, D. in Contemporary Computational Mathematics — a celebration of the 80th birthday of Ian
Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.) 597–619 (Springer-Verlag, 2018), Kunsch, R. J., Novak, E. & Rudolf, D.
Solvable Integration Problems and Optimal Sample Size Selection. Journal of Complexity. To appear (2018), Ding, Y., H., F. J. &
Jiménez Rugama, An Adaptive Algorithm Employing Continuous Linear Functionals. in Monte Carlo and Quasi-Monte Carlo
Methods: MCQMC, Rennes, France, July 2018 (eds L’Ecuyer, P. & Tuﬃn, B.) Under revision (Springer-Verlag, Berlin, 2019+). 2/14

4. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Context
Linear solution operator SOL : F → G, input space F contains functions deﬁned on Ω ⊆ Rd
e.g., SOL(f) = f, −∇2 SOL(f) = f, SOL(f) = 0 on boundary
Approximation APP(f, n) =
n
i=1
Li
(f)gi
, can sample linear functionals
SOL(f) − APP(f, n)
G
SOL − APP(·, n) F→G
f
F
Algorithm ALG(f, ε) = APP f, n∗(f, ε) satisfying SOL(f) − ALG(f, ε)
G
ε for all f ∈ C ⊂ F
Solvability: what C? how to determine n∗(f, ε) ∈ N?
×or
Optimality: is n∗(f, ε) essentially as small as possible?
Tractability: does n∗(f, ε) depend nicely on d?
2/14

5. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
General Linear Problems Deﬁned on Series Spaces
F :=

f =

i=1
f(ki
)uki
: f
F
:=
f(ki
)
λki

i=1 ρ

λk1
λk2
· · · > 0
λ aﬀects convergence rate &
tractability
G := g =

i=1
^
g(ki
)vki
: g
G
:= ^
g
τ
, SOL(f) =

i=1
f(ki
)vki
, τ ρ
Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
publication (DeGruyter, 2019+). 3/14

6. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Legendre and Chebyshev Bases for Function Approximation
Legendre
Chebyshev
4/14

7. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Adaptive Algorithm for General Linear Problems on Cone of Inputs
F :=

f =

i=1
f(ki
)uki
: f
F
:=
f(ki
)
λki

i=1 ρ

λk1
λk2
· · · > 0
λ aﬀects convergence rate &
tractability
G := g =

i=1
^
g(ki
)vki
: g
G
:= ^
g
τ
, SOL(f) =

i=1
f(ki
)vki
, τ ρ
Cλ,n1
,A
:= f ∈ F : f
F
A
f(ki
)
λki
n1
i=1 ρ
pilot sample
bounds the norm of the input
APP(f, n) =
n
i=1
f(ki
)vki
optimal for ﬁxed n SOL(f) − APP(f, n)
G
ERR f(ki
) n
i=1
, n
ERR f(ki
) n
i=1
, n :=

Aρ
f(ki
)
λki
n1
i=1
ρ
ρ

f(ki
)
λki
n
i=1
ρ
ρ

1/ρ
upper bound on f− n
i=1
f(ki)uki F
λki

i=n+1 ρ
SOL − APP(·,n) F→G
1
ρ
+
1
ρ
=
1
τ
data-driven
Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
publication (DeGruyter, 2019+). 5/14

8. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Adaptive Algorithm for General Linear Problems on Cone of Inputs
Cλ,n1
,A
:= f ∈ F : f
F
A
f(ki
)
λki
n1
i=1 ρ
pilot sample
bounds the norm of the input
APP(f, n) =
n
i=1
f(ki
)vki
optimal for ﬁxed n SOL(f) − APP(f, n)
G
ERR f(ki
) n
i=1
, n
ERR f(ki
) n
i=1
, n :=

Aρ
f(ki
)
λki
n1
i=1
ρ
ρ

f(ki
)
λki
n
i=1
ρ
ρ

1/ρ
upper bound on f− n
i=1
f(ki)uki F
λki

i=n+1 ρ
SOL − APP(·,n) F→G
1
ρ
+
1
ρ
=
1
τ
data-driven
ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki
) n
i=1
, n ε}
Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
publication (DeGruyter, 2019+). 5/14

9. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Adaptive Algorithm for General Linear Problems on Cone of Inputs
Cλ,n1
,A
:= f ∈ F : f
F
A
f(ki
)
λki
n1
i=1 ρ
pilot sample
bounds the norm of the input
APP(f, n) =
n
i=1
f(ki
)vki
optimal for ﬁxed n SOL(f) − APP(f, n)
G
ERR f(ki
) n
i=1
, n
ERR f(ki
) n
i=1
, n :=

Aρ
f(ki
)
λki
n1
i=1
ρ
ρ

f(ki
)
λki
n
i=1
ρ
ρ

1/ρ
upper bound on f− n
i=1
f(ki)uki F
λki

i=n+1 ρ
SOL − APP(·,n) F→G
1
ρ
+
1
ρ
=
1
τ
data-driven
ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki
) n
i=1
, n ε}
COST(ALG, Cλ,n1
,A
, ε, R) = max n∗(f, ε) : f ∈ Cλ,n1
,A ∩ BR
= ∩
= min n n1
: λki

i=n+1 ρ
ε/[(Aρ − 1)1/ρR]
Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
publication (DeGruyter, 2019+). 5/14

10. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Optimality of Algorithm
Cλ,n1
,A
:= f ∈ F : f
F
A
f(ki
)
λki
n1
i=1 ρ
pilot sample
bounds the norm of the input
ALG(f, ε) = APP(f, n∗(f, ε)) for n∗(f, ε) = min{n ∈ N : ERR f(ki
) n
i=1
, n ε}
COST(ALG, Cλ,n1
,A
, ε, R) = max n∗(f, ε) : f ∈ Cλ,n1
,A ∩ BR
= ∩
= min n n1
: λki

i=n+1 ρ
ε/[(Aρ − 1)1/ρR]
COMP(A(Cλ,n1
,A
, ε, R)) = min{COST(ALG , Cλ,n1
,A
, ε, R) : ALG ∈ A(Cλ,n1
,A
)}
COST(ALG, Cλ,n1
,A
, ωε, R), ω =
2A(Aρ − 1)1/ρ
A − 1
> 1
via fooling functions
ALG is essentially optimal
Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
publication (DeGruyter, 2019+). 6/14

11. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Tractability of Solving General Linear Problems on Cone of Inputs
Cd,λd
,n1
,A
:= f ∈ F : f
F
A
f(ki
)
λki
n1
i=1 ρ
pilot sample
bounds the norm of the input
For τ = ρ (ρ = ∞), the problem SOLd
: Cd,λd
,n1
,A
→ Gd
is
Strongly polynomial tractable iﬀ there exist i0
∈ N and η > 0 such that sup
d∈N

i=i0
λη
d,ki
< ∞
Polynomial tractable iﬀ there exist η1
, η2
0 and η3
, K > 0 such that sup
d∈N
d−η1

i= Kdη2
λη3
d,ki
< ∞
Weakly tractable iﬀ sup
d∈N
exp(−cd)

i=1
exp −c
1
λd,ki
< ∞ for all c > 0
Analogous results exist for τ < ρ (ρ < ∞)
Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
publication (DeGruyter, 2019+). 7/14

12. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
An Alternate Cone, Similar to Our Cone
f
F
:=
f(ki
)
λki

i=1 ρ
, λki
↓ 0, f
F
:=
f(ki
)ζki
λki

i=1 ρ
, ζk1
= 1, ζki

Cλ,n1
,A
:= f ∈ F : f
F
A
f(ki
)
λki
n1
i=1 ρ
pilot sample bounds the norm of the input
Cλ,ζ,A
:= f ∈ F : f
F
A f
F
stronger norm is bounded above by weaker norm
Cλ,ζ,A
⊆ Cλ,n1
,A
for A A
1 + ζρ
kn1+1
1 − (ζkn1+1
A )ρ
1/ρ
, ζkn1+1
A < 1
Cλ,n1
,A ⊆ Cλ,ζ,A
for A A ζkn1
8/14

13. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
An Alternative Cone, Similar to Our Cone
For any f ∈ Cλ,ζ,A
it follows that
f ρ
F

f(ki
)ζki
λki
n1
i=1
ρ
ρ
=
f(ki
)ζki
λki

i=n1
+1
ρ
ρ
ζρ
kn1+1
f(ki
)
λki

i=n1
+1
ρ
ρ
= ζρ
kn1+1

 f ρ
F

f(ki
)
λki
n1
i=1
ρ
ρ

ζρ
kn1+1

A ρ f ρ
F

f(ki
)ζki
λki
n1
i=1
ρ
ρ

9/14

14. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
An Alternative Cone, Similar to Our Cone
For any f ∈ Cλ,ζ,A
it follows that
f ρ
F

f(ki
)ζki
λki
n1
i=1
ρ
ρ
=
f(ki
)ζki
λki

i=n1
+1
ρ
ρ
ζρ
kn1+1
f(ki
)
λki

i=n1
+1
ρ
ρ
= ζρ
kn1+1

 f ρ
F

f(ki
)
λki
n1
i=1
ρ
ρ

ζρ
kn1+1

A ρ f ρ
F

f(ki
)ζki
λki
n1
i=1
ρ
ρ

f ρ
F
1 + ζρ
kn1+1
1 − (ζkn1+1
A )ρ
f(ki
)ζki
λki
n1
i=1
ρ
ρ
f
F
A f
F
A
1 + ζρ
kn1+1
1 − (ζkn1+1
A )ρ
1/ρ
f(ki
)
λki
n1
i=1 ρ
9/14

15. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
An Alternate Cone, Similar to Our Cone
For any f ∈ Cλ,n1
,A
it follows that
f
F
A
f(ki
)
λki
n1
i=1 ρ
A
ζkn1
f(ki
)ζki
λki
n1
i=1 ρ
A
ζkn1
f
F
10/14

16. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Conclusion
Summary
Adaptive algorithms can be constructed for non-convex, symmetric cones of inputs
One possible cone assumes that a pilot sample tells you enough about the norm of the input
Information cost and complexity depend on the norm of the input, but the adaptive algorithm does
not know this norm
There are tractability results for problems deﬁned on cones
Our algorithm can be extended to infer the λk
in terms of coordinate weights
Further Work
Need to develop and analyze algorithms based on function values, not series coeﬃcients
Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. & Kritzer, P.) Submitted for
publication (DeGruyter, 2019+). 11/14

17. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
Cheng and Sandu Function
Function values for data Chebyshev polynomial basis, λk
inferred ,
Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets. 2017.
https://www.sfu.ca/~ssurjano/index.html (2017). 12/14

18. Thank you
These slides are available at
speakerdeck.com/fjhickernell/mcm-2019-pilot-sample

19. Introduction Algorithm Optimality Tractability Alternate Cone Summary References
References I
H., F. J., Jiménez Rugama, & Li, D. in Contemporary Computational Mathematics — a celebration
of the 80th birthday of Ian Sloan (eds Dick, J., Kuo, F. Y. & Woźniakowski, H.) 597–619
(Springer-Verlag, 2018).
Kunsch, R. J., Novak, E. & Rudolf, D. Solvable Integration Problems and Optimal Sample Size
Selection. Journal of Complexity. To appear (2018).
Ding, Y., H., F. J. & Jiménez Rugama, An Adaptive Algorithm Employing Continuous Linear
Functionals. in Monte Carlo and Quasi-Monte Carlo Methods: MCQMC, Rennes, France, July 2018
(eds L’Ecuyer, P. & Tuﬃn, B.) Under revision (Springer-Verlag, Berlin, 2019+).
Ding, Y., H., F. J., Kritzer, P. & Mak, S. in RICAM Fall Semester Proceedings (eds H., F. J. &
Kritzer, P.) Submitted for publication (DeGruyter, 2019+).
Bingham, D. The Virtual Library of Simulation Experiments: Test Functions and Data Sets. 2017.
https://www.sfu.ca/~ssurjano/index.html (2017).
14/14