150

# SAMSI-QMC WG 5-3 Research Problem Proposal & Preliminary Results

A concrete research problem proposed in the SAMSI Working Group Meeting, with appended results January 17, 2018

## Transcript

1. ### Research Problem Proposal: Function Approximation when Function Values Are Expensive

Fred J. Hickernell Department of Applied Mathematics, Illinois Institute of Technology [email protected] mypages.iit.edu/~hickernell Supported by NSF-DMS-1522687 Working Group V.3, January 17 & 31, 2018
2. ### Background Known Results Results Under Construction References Prologue May 7–9

we have our SAMSI-QMC Transitions Workshop where we should report on our progress. Now is the time for proposing and addressing concrete problems. These slides outline a speciﬁc problem that I will work on. I welcome collaborators. Please indicate your interest. If you have another speciﬁc problem to propose, that you are willing to lead please do so. If you just wish to observe and comment, you are welcome also. 2/14
3. ### Background Known Results Results Under Construction References Notation and Assumptions

Let F be a vector space of functions f : [0, 1]d → R that have L2[0, 1]d orthogonal series expansions: f(x) = j∈Nd 0 f(j)φj(x), φj(x) = φj1 (x1) · · · φjd (xd) What is the right basis? Suppose that we may observe the Fourier coeﬃcients f(j) at a cost of \$1 each. (Eventually we want to consider the case of observing function values.) For any vector of non-negative constants, γ = (γj)j∈Nd 0 , deﬁne the inner product on F, f, g γ = j∈Nd 0 f(j)^ g(j)γ−2 j , 0/0 = 0, γj = 0, f γ < ∞ =⇒ f(j) = 0 Order the elements of γ such that γj1 γj2 · · · . The best approximation to f given n Fourier coeﬃcients chosen optimally is f(x) = n i=1 f(ji)φji , f − f 2 = ∞ i=n+1 f(ji) 2 γ−2 ji · γ2 ji γjn+1 f γ 3/14
4. ### Background Known Results Results Under Construction References Over-Arching Problem f(x)

= j∈Nd 0 f(j)φj(x) = ∞ i=1 f(ji)φji (x) dependence of f on d is hidden f(x) = n i=1 f(ji)φji (x), f − f 2 = ∞ i=n+1 f(ji) 2 γ−2 ji · γ2 ji γjn+1 f γ Over-arching problem: Under what conditions on γ can we make γjn+1 ε, and so f − f 2 ε f γ for n = O(d)? Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information. EMS Tracts in Mathematics 6 (European Mathematical Society, Zürich, 2008), Kühn, T. et al. Approximation numbers of Sobolev embeddings—Sharp constants and tractability. J. Complexity 30, 95–116 (2014). 4/14
5. ### Background Known Results Results Under Construction References Over-Arching Problem f(x)

= j∈Nd 0 f(j)φj(x) = ∞ i=1 f(ji)φji (x) dependence of f on d is hidden f(x) = n i=1 f(ji)φji (x), f − f 2 = ∞ i=n+1 f(ji) 2 γ−2 ji · γ2 ji γjn+1 f γ Trick: γjn+1 1 n γ1/p j1 + · · · + γ1/p jn p 1 np j∈Nd 0 γ1/p j p ∀p > 0 n 1 ε1/p j∈Nd 0 γ1/p j =⇒ γjn+1 ε =⇒ f − f 2 ε f γ so want j∈Nd 0 γ1/p j = O(d) 5/14
6. ### Background Known Results Results Under Construction References C = {f

∈ F : f γ < ∞} Where γ Is Known j∈Nd 0 γ1/p j = O(d) =⇒ f − f 2 ε f γ for n = O(d) ∀f ∈ C, p = order of convergence how the sum varies with d tells you whether you can aﬀord the approximation Fail Case. γj = γj1 · · · γjd , γ0 = 1 j∈Nd 0 γ1/p j =   ∞ j=0 γ1/p j   d exponential growth in d 6/14
7. ### Background Known Results Results Under Construction References C = {f

∈ F : f γ < ∞} Where γ Is Known j∈Nd 0 γ1/p j = O(d) =⇒ f − f 2 ε f γ for n = O(d) ∀f ∈ C Success Case. γj = γj1,1 · · · γjd,d , γj,k =    1, j = 0 1 (kj)p(1+δ) , j > 0, δ > 0 j∈Nd 0 γ1/p j = d k=1 ∞ j=0 γ1/p jk,k = d k=1  1 + ∞ j=1 1 k1+δj1+δ   = exp   d k=1 log  1 + 1 k1+δ ∞ j=1 1 j1+δ     exp   d k=1 1 k1+δ ∞ j=1 1 j1+δ   exp   ∞ k=1 1 k1+δ ∞ j=1 1 j1+δ   < ∞ Power of k or j can be larger than the other, but it does not help 7/14
8. ### Background Known Results Results Under Construction References C = {f

∈ F : f γ < ∞} Where γ Is Known j∈Nd 0 γ1/p j = O(d) =⇒ f − f 2 ε f γ for n = O(d) ∀f ∈ C Success Case. γ0 = 1, γ(0,...,0,j,0,...0) = j−p(1+δ), and γj = 0 otherwise (additive functions) j∈Nd 0 γ1/p j = d k=1  1 + ∞ j=1 1 j1+δ   = d  1 + ∞ j=1 1 j1+δ   linear in d 8/14
9. ### Background Known Results Results Under Construction References Open Questions The

known results assume that some terms in the expansion are small and that you know which terms are small a priori: cone of successful functions is C = f ∈ F : f γ < ∞, sup d 1 d j∈Nd 0 γ1/p j where γ is ﬁxed in advance Is it possible to have a cone of successful functions for which γ is not ﬁxed in advance, γ is inferred by the observed f(ji), and n = O(d) obtains f − f 2 ε? 9/14
10. ### Background Known Results Results Under Construction References Some Assumptions Experimental

design assumes the following Eﬀect sparsity: Only a small number of eﬀects are important Eﬀect hierarchy: Lower-order eﬀects are more important than higher-order eﬀects Eﬀect heredity: Interaction is active only if both parent eﬀects are also active Eﬀect smoothness: Coarse horizontal scales are more important than ﬁne horizontal scales These assumptions may suggest γ of product form: γj = γj1,1 · · · γjd,d, γj,k =    1, j = 0, γ1,k jp , j > 0, γ1,k unknown, p known To infer the γ1,k , set γ1,1 = f(1, 0, . . . , 0)/f(0) , γ1,2 = f(0, 1, 0, . . . , 0)/f(0) , . . . , γ1,d = f(0, . . . , 0, 1)/f(0) Wu, C. F. J. & Hamada, M. Experiments: Planning, Analysis, and Parameter Design Optimization. (John Wiley & Sons, Inc., New York, 2000). 10/14
11. ### Background Known Results Results Under Construction References Possible Algorithm If

the Fourier coeﬃcients are sampled in the order j1, j2, . . ., then f(x) = j∈Nd 0 f(j)φj(x) = ∞ i=1 f(ji)φji (x), f(x) = n i=1 f(ji)φji f − f 2 = ∞ i=n+1 f(ji) 2 γ−2 ji · γ2 ji sup j∈Nd 0 f(j) γj ∞ i=n+1 γ2 ji = sup j∈Nd 0 f(j) γj inferred from what’s observed ∞ j∈Nd 0 γ2 j − n i=1 γ2 ji →0 as n→∞ γj = γj1,1 · · · γjd,d, γj,k =    1, j = 0, γ1,k jp , j > 0, ∞ j∈Nd 0 γ2 j = d k=1 [1 + ζ(2p)γ1,k], γ1,k unknown Set j1 = 0, j1 = (1, 0 . . . , 0), . . . , jd+1 = (0, . . . , 0, 1), sample f(j1), . . . , f(jd+1 ), and set γ1,1 = f(1, 0, . . . , 0)/f(0) , γ1,2 = f(0, 1, 0, . . . , 0)/f(0) , . . . , γ1,d = f(0, . . . , 0, 1)/f(0) Increment i, choose ji with the next largest γj, and sample f(ji), until error bound is small. 11/14
12. ### Background Known Results Results Under Construction References What Needs Attention

Our cone of nice functions looks somewhat like C = f ∈ F : sup j∈Nd 0 f(j) γj < ∞, for some γ satisfying γj = γj1,1 · · · γjd,d, γj,k =    1, j = 0, γ1,k jp , j > 0, d k=1 [1 + ζ(2p)γ1,k] = O(d2) Bookkeeping on next largest γj Have trouble if f(0) is too small If f(j)/γj is observed to be too large, may need to increase γ1,k for some k or decrease p May want to infer p Might be able to do L∞ function approximation Need to move from observing Fourier coeﬃcients to observing function values Try some examples 12/14
13. ### Background Known Results Results Under Construction References Closing Thoughts May

7–9 we have our SAMSI-QMC Transitions Workshop where we should report on our progress. Now is the time for proposing and addressing concrete problems. These slides outline a speciﬁc problem that I will work on. I welcome collaborators. Please indicate your interest. If you have another speciﬁc problem to propose, that you are willing to lead please do so. If you just wish to observe and comment, you are welcome also. 13/14

15. ### Background Known Results Results Under Construction References Novak, E. &

Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information. EMS Tracts in Mathematics 6 (European Mathematical Society, Zürich, 2008). Kühn, T., Sickel, W. & Ullrich, T. Approximation numbers of Sobolev embeddings—Sharp constants and tractability. J. Complexity 30, 95–116 (2014). Wu, C. F. J. & Hamada, M. Experiments: Planning, Analysis, and Parameter Design Optimization. (John Wiley & Sons, Inc., New York, 2000). 14/14