160

# SAMSI-QMC WG 5-3 Research Problem Proposal & Preliminary Results

A concrete research problem proposed in the SAMSI Working Group Meeting, with appended results January 17, 2018

## Transcript

1. Research Problem Proposal:
Function Approximation when Function Values Are Expensive
Fred J. Hickernell
Department of Applied Mathematics, Illinois Institute of Technology
[email protected] mypages.iit.edu/~hickernell
Supported by NSF-DMS-1522687
Working Group V.3, January 17 & 31, 2018

2. Background Known Results Results Under Construction References
Prologue
May 7–9 we have our SAMSI-QMC Transitions Workshop where we should report on our
progress.
Now is the time for proposing and addressing concrete problems.
These slides outline a speciﬁc problem that I will work on. I welcome collaborators.
If you have another speciﬁc problem to propose, that you are willing to lead please do so.
If you just wish to observe and comment, you are welcome also.
2/14

3. Background Known Results Results Under Construction References
Notation and Assumptions
Let F be a vector space of functions f : [0, 1]d → R that have L2[0, 1]d orthogonal series
expansions:
f(x) =
j∈Nd
0
f(j)φj(x), φj(x) = φj1
(x1) · · · φjd
(xd) What is the right basis?
Suppose that we may observe the Fourier coeﬃcients f(j) at a cost of \$1 each. (Eventually we
want to consider the case of observing function values.) For any vector of non-negative
constants, γ = (γj)j∈Nd
0
, deﬁne the inner product on F,
f, g γ
=
j∈Nd
0
f(j)^
g(j)γ−2
j
, 0/0 = 0, γj = 0, f γ
< ∞ =⇒ f(j) = 0
Order the elements of γ such that γj1
γj2
· · · . The best approximation to f given n
Fourier coeﬃcients chosen optimally is
f(x) =
n
i=1
f(ji)φji
, f − f
2
=

i=n+1
f(ji)
2
γ−2
ji
· γ2
ji
γjn+1
f γ
3/14

4. Background Known Results Results Under Construction References
Over-Arching Problem
f(x) =
j∈Nd
0
f(j)φj(x) =

i=1
f(ji)φji
(x) dependence of f on d is hidden
f(x) =
n
i=1
f(ji)φji
(x), f − f
2
=

i=n+1
f(ji)
2
γ−2
ji
· γ2
ji
γjn+1
f γ
Over-arching problem: Under what conditions on γ can we make γjn+1
ε, and so
f − f
2
ε f γ
for n = O(d)?
Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information. EMS Tracts in
Mathematics 6 (European Mathematical Society, Zürich, 2008), Kühn, T. et al. Approximation numbers of Sobolev
embeddings—Sharp constants and tractability. J. Complexity 30, 95–116 (2014). 4/14

5. Background Known Results Results Under Construction References
Over-Arching Problem
f(x) =
j∈Nd
0
f(j)φj(x) =

i=1
f(ji)φji
(x) dependence of f on d is hidden
f(x) =
n
i=1
f(ji)φji
(x), f − f
2
=

i=n+1
f(ji)
2
γ−2
ji
· γ2
ji
γjn+1
f γ
Trick: γjn+1
1
n
γ1/p
j1
+ · · · + γ1/p
jn
p 1
np
j∈Nd
0
γ1/p
j
p
∀p > 0
n
1
ε1/p
j∈Nd
0
γ1/p
j
=⇒ γjn+1
ε =⇒ f − f
2
ε f γ
so want
j∈Nd
0
γ1/p
j
= O(d)
5/14

6. Background Known Results Results Under Construction References
C = {f ∈ F : f
γ
< ∞} Where γ Is Known
j∈Nd
0
γ1/p
j
= O(d) =⇒ f − f
2
ε f γ
for n = O(d) ∀f ∈ C,
p = order of convergence
how the sum varies with d tells you whether you can aﬀord the approximation
Fail Case. γj = γj1
· · · γjd
, γ0 = 1
j∈Nd
0
γ1/p
j
=

j=0
γ1/p
j

d
exponential growth in d
6/14

7. Background Known Results Results Under Construction References
C = {f ∈ F : f
γ
< ∞} Where γ Is Known
j∈Nd
0
γ1/p
j
= O(d) =⇒ f − f
2
ε f γ
for n = O(d) ∀f ∈ C
Success Case. γj = γj1,1 · · · γjd,d
, γj,k =

1, j = 0
1
(kj)p(1+δ)
, j > 0,
δ > 0
j∈Nd
0
γ1/p
j
=
d
k=1

j=0
γ1/p
jk,k
=
d
k=1

1 +

j=1
1
k1+δj1+δ

 = exp

d
k=1
log

1 +
1
k1+δ

j=1
1
j1+δ

exp

d
k=1
1
k1+δ

j=1
1
j1+δ

 exp

k=1
1
k1+δ

j=1
1
j1+δ

 < ∞
Power of k or j can be larger than the other, but it does not help
7/14

8. Background Known Results Results Under Construction References
C = {f ∈ F : f
γ
< ∞} Where γ Is Known
j∈Nd
0
γ1/p
j
= O(d) =⇒ f − f
2
ε f γ
for n = O(d) ∀f ∈ C
Success Case. γ0 = 1, γ(0,...,0,j,0,...0)
= j−p(1+δ), and γj = 0 otherwise (additive functions)
j∈Nd
0
γ1/p
j
=
d
k=1

1 +

j=1
1
j1+δ

 = d

1 +

j=1
1
j1+δ

 linear in d
8/14

9. Background Known Results Results Under Construction References
Open Questions
The known results assume that some terms in the expansion are small and that you know
which terms are small a priori: cone of successful functions is
C = f ∈ F : f γ
< ∞, sup
d
1
d
j∈Nd
0
γ1/p
j
where γ is ﬁxed in advance
Is it possible to have a cone of successful functions for which
γ is not ﬁxed in advance,
γ is inferred by the observed f(ji), and
n = O(d) obtains f − f
2
ε?
9/14

10. Background Known Results Results Under Construction References
Some Assumptions
Experimental design assumes the following
Eﬀect sparsity: Only a small number of eﬀects are important
Eﬀect hierarchy: Lower-order eﬀects are more important than higher-order eﬀects
Eﬀect heredity: Interaction is active only if both parent eﬀects are also active
Eﬀect smoothness: Coarse horizontal scales are more important than ﬁne horizontal scales
These assumptions may suggest γ of product form:
γj = γj1,1 · · · γjd,d, γj,k =

1, j = 0,
γ1,k
jp
, j > 0,
γ1,k
unknown, p known
To infer the γ1,k
, set
γ1,1 = f(1, 0, . . . , 0)/f(0) , γ1,2 = f(0, 1, 0, . . . , 0)/f(0) , . . . , γ1,d = f(0, . . . , 0, 1)/f(0)
Wu, C. F. J. & Hamada, M. Experiments: Planning, Analysis, and Parameter Design Optimization. (John Wiley
& Sons, Inc., New York, 2000). 10/14

11. Background Known Results Results Under Construction References
Possible Algorithm
If the Fourier coeﬃcients are sampled in the order j1, j2, . . ., then
f(x) =
j∈Nd
0
f(j)φj(x) =

i=1
f(ji)φji
(x), f(x) =
n
i=1
f(ji)φji
f − f
2
=

i=n+1
f(ji)
2
γ−2
ji
· γ2
ji
sup
j∈Nd
0
f(j)
γj

i=n+1
γ2
ji
= sup
j∈Nd
0
f(j)
γj
inferred from
what’s observed

j∈Nd
0
γ2
j

n
i=1
γ2
ji
→0 as n→∞
γj = γj1,1 · · · γjd,d, γj,k =

1, j = 0,
γ1,k
jp
, j > 0,

j∈Nd
0
γ2
j
=
d
k=1
[1 + ζ(2p)γ1,k], γ1,k
unknown
Set j1 = 0, j1 = (1, 0 . . . , 0), . . . , jd+1
= (0, . . . , 0, 1), sample f(j1), . . . , f(jd+1
), and set
γ1,1 = f(1, 0, . . . , 0)/f(0) , γ1,2 = f(0, 1, 0, . . . , 0)/f(0) , . . . , γ1,d = f(0, . . . , 0, 1)/f(0)
Increment i, choose ji with the next largest γj, and sample f(ji), until error bound is small.
11/14

12. Background Known Results Results Under Construction References
What Needs Attention
Our cone of nice functions looks somewhat like
C = f ∈ F : sup
j∈Nd
0
f(j)
γj
< ∞, for some γ satisfying
γj = γj1,1 · · · γjd,d, γj,k =

1, j = 0,
γ1,k
jp
, j > 0,
d
k=1
[1 + ζ(2p)γ1,k] = O(d2)
Bookkeeping on next largest γj
Have trouble if f(0) is too small
If f(j)/γj is observed to be too large, may need to increase γ1,k
for some k or decrease p
May want to infer p
Might be able to do L∞ function approximation
Need to move from observing Fourier coeﬃcients to observing function values
Try some examples
12/14

13. Background Known Results Results Under Construction References
Closing Thoughts
May 7–9 we have our SAMSI-QMC Transitions Workshop where we should report on our
progress.
Now is the time for proposing and addressing concrete problems.
These slides outline a speciﬁc problem that I will work on. I welcome collaborators.
If you have another speciﬁc problem to propose, that you are willing to lead please do so.
If you just wish to observe and comment, you are welcome also.
13/14

14. Thank you
These slides are at https://speakerdeck.com/fjhickernell/samsi-qmc-wg-5-3-
research-problem-proposal

15. Background Known Results Results Under Construction References
Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear
Information. EMS Tracts in Mathematics 6 (European Mathematical Society, Zürich, 2008).
Kühn, T., Sickel, W. & Ullrich, T. Approximation numbers of Sobolev embeddings—Sharp
constants and tractability. J. Complexity 30, 95–116 (2014).
Wu, C. F. J. & Hamada, M. Experiments: Planning, Analysis, and Parameter Design
Optimization. (John Wiley & Sons, Inc., New York, 2000).
14/14