SAMSI-QMC WG 5-3 Research Problem Proposal & Preliminary Results

A concrete research problem proposed in the SAMSI Working Group Meeting, with appended results

Transcript

1. Research Problem Proposal: Function Approximation when Function Values Are Expensive Fred J. Hickernell Department of Applied Mathematics, Illinois Institute of Technology hickernell@iit.edu mypages.iit.edu/~hickernell Supported by NSF-DMS-1522687 Working Group V.3, January 17 & 31, 2018
2. Background Known Results Results Under Construction References Prologue May 7–9 we have our SAMSI-QMC Transitions Workshop where we should report on our progress. Now is the time for proposing and addressing concrete problems. These slides outline a specific problem that I will work on. I welcome collaborators. Please indicate your interest. If you have another specific problem to propose, that you are willing to lead please do so. If you just wish to observe and comment, you are welcome also. 2/14
3. Background Known Results Results Under Construction References Notation and Assumptions Let F be a vector space of functions f : [0, 1]d → R that have L2[0, 1]d orthogonal series expansions: f(x) = j∈Nd 0 f(j)φj(x), φj(x) = φj1 (x1) · · · φjd (xd) What is the right basis? Suppose that we may observe the Fourier coefficients f(j) at a cost of $1 each. (Eventually we want to consider the case of observing function values.) For any vector of non-negative constants, γ = (γj)j∈Nd 0 , define the inner product on F, f, g γ = j∈Nd 0 f(j)^ g(j)γ−2 j , 0/0 = 0, γj = 0, f γ < ∞ =⇒ f(j) = 0 Order the elements of γ such that γj1 γj2 · · · . The best approximation to f given n Fourier coefficients chosen optimally is f(x) = n i=1 f(ji)φji , f − f 2 = ∞ i=n+1 f(ji) 2 γ−2 ji · γ2 ji γjn+1 f γ 3/14
4. Background Known Results Results Under Construction References Over-Arching Problem f(x) = j∈Nd 0 f(j)φj(x) = ∞ i=1 f(ji)φji (x) dependence of f on d is hidden f(x) = n i=1 f(ji)φji (x), f − f 2 = ∞ i=n+1 f(ji) 2 γ−2 ji · γ2 ji γjn+1 f γ Over-arching problem: Under what conditions on γ can we make γjn+1 ε, and so f − f 2 ε f γ for n = O(d)? Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information. EMS Tracts in Mathematics 6 (European Mathematical Society, Zürich, 2008), Kühn, T. et al. Approximation numbers of Sobolev embeddings—Sharp constants and tractability. J. Complexity 30, 95–116 (2014). 4/14
5. Background Known Results Results Under Construction References Over-Arching Problem f(x) = j∈Nd 0 f(j)φj(x) = ∞ i=1 f(ji)φji (x) dependence of f on d is hidden f(x) = n i=1 f(ji)φji (x), f − f 2 = ∞ i=n+1 f(ji) 2 γ−2 ji · γ2 ji γjn+1 f γ Trick: γjn+1 1 n γ1/p j1 + · · · + γ1/p jn p 1 np j∈Nd 0 γ1/p j p ∀p > 0 n 1 ε1/p j∈Nd 0 γ1/p j =⇒ γjn+1 ε =⇒ f − f 2 ε f γ so want j∈Nd 0 γ1/p j = O(d) 5/14
6. Background Known Results Results Under Construction References C = {f ∈ F : f γ < ∞} Where γ Is Known j∈Nd 0 γ1/p j = O(d) =⇒ f − f 2 ε f γ for n = O(d) ∀f ∈ C, p = order of convergence how the sum varies with d tells you whether you can afford the approximation Fail Case. γj = γj1 · · · γjd , γ0 = 1 j∈Nd 0 γ1/p j =   ∞ j=0 γ1/p j   d exponential growth in d 6/14
7. Background Known Results Results Under Construction References C = {f ∈ F : f γ < ∞} Where γ Is Known j∈Nd 0 γ1/p j = O(d) =⇒ f − f 2 ε f γ for n = O(d) ∀f ∈ C Success Case. γj = γj1,1 · · · γjd,d , γj,k =    1, j = 0 1 (kj)p(1+δ) , j > 0, δ > 0 j∈Nd 0 γ1/p j = d k=1 ∞ j=0 γ1/p jk,k = d k=1  1 + ∞ j=1 1 k1+δj1+δ   = exp   d k=1 log  1 + 1 k1+δ ∞ j=1 1 j1+δ     exp   d k=1 1 k1+δ ∞ j=1 1 j1+δ   exp   ∞ k=1 1 k1+δ ∞ j=1 1 j1+δ   < ∞ Power of k or j can be larger than the other, but it does not help 7/14
8. Background Known Results Results Under Construction References C = {f ∈ F : f γ < ∞} Where γ Is Known j∈Nd 0 γ1/p j = O(d) =⇒ f − f 2 ε f γ for n = O(d) ∀f ∈ C Success Case. γ0 = 1, γ(0,...,0,j,0,...0) = j−p(1+δ), and γj = 0 otherwise (additive functions) j∈Nd 0 γ1/p j = d k=1  1 + ∞ j=1 1 j1+δ   = d  1 + ∞ j=1 1 j1+δ   linear in d 8/14
9. Background Known Results Results Under Construction References Open Questions The known results assume that some terms in the expansion are small and that you know which terms are small a priori: cone of successful functions is C = f ∈ F : f γ < ∞, sup d 1 d j∈Nd 0 γ1/p j where γ is fixed in advance Is it possible to have a cone of successful functions for which γ is not fixed in advance, γ is inferred by the observed f(ji), and n = O(d) obtains f − f 2 ε? 9/14
10. Background Known Results Results Under Construction References Some Assumptions Experimental design assumes the following Effect sparsity: Only a small number of effects are important Effect hierarchy: Lower-order effects are more important than higher-order effects Effect heredity: Interaction is active only if both parent effects are also active Effect smoothness: Coarse horizontal scales are more important than fine horizontal scales These assumptions may suggest γ of product form: γj = γj1,1 · · · γjd,d, γj,k =    1, j = 0, γ1,k jp , j > 0, γ1,k unknown, p known To infer the γ1,k , set γ1,1 = f(1, 0, . . . , 0)/f(0) , γ1,2 = f(0, 1, 0, . . . , 0)/f(0) , . . . , γ1,d = f(0, . . . , 0, 1)/f(0) Wu, C. F. J. & Hamada, M. Experiments: Planning, Analysis, and Parameter Design Optimization. (John Wiley & Sons, Inc., New York, 2000). 10/14
11. Background Known Results Results Under Construction References Possible Algorithm If the Fourier coefficients are sampled in the order j1, j2, . . ., then f(x) = j∈Nd 0 f(j)φj(x) = ∞ i=1 f(ji)φji (x), f(x) = n i=1 f(ji)φji f − f 2 = ∞ i=n+1 f(ji) 2 γ−2 ji · γ2 ji sup j∈Nd 0 f(j) γj ∞ i=n+1 γ2 ji = sup j∈Nd 0 f(j) γj inferred from what’s observed ∞ j∈Nd 0 γ2 j − n i=1 γ2 ji →0 as n→∞ γj = γj1,1 · · · γjd,d, γj,k =    1, j = 0, γ1,k jp , j > 0, ∞ j∈Nd 0 γ2 j = d k=1 [1 + ζ(2p)γ1,k], γ1,k unknown Set j1 = 0, j1 = (1, 0 . . . , 0), . . . , jd+1 = (0, . . . , 0, 1), sample f(j1), . . . , f(jd+1 ), and set γ1,1 = f(1, 0, . . . , 0)/f(0) , γ1,2 = f(0, 1, 0, . . . , 0)/f(0) , . . . , γ1,d = f(0, . . . , 0, 1)/f(0) Increment i, choose ji with the next largest γj, and sample f(ji), until error bound is small. 11/14
12. Background Known Results Results Under Construction References What Needs Attention Our cone of nice functions looks somewhat like C = f ∈ F : sup j∈Nd 0 f(j) γj < ∞, for some γ satisfying γj = γj1,1 · · · γjd,d, γj,k =    1, j = 0, γ1,k jp , j > 0, d k=1 [1 + ζ(2p)γ1,k] = O(d2) Bookkeeping on next largest γj Have trouble if f(0) is too small If f(j)/γj is observed to be too large, may need to increase γ1,k for some k or decrease p May want to infer p Might be able to do L∞ function approximation Need to move from observing Fourier coefficients to observing function values Try some examples 12/14
13. Background Known Results Results Under Construction References Closing Thoughts May 7–9 we have our SAMSI-QMC Transitions Workshop where we should report on our progress. Now is the time for proposing and addressing concrete problems. These slides outline a specific problem that I will work on. I welcome collaborators. Please indicate your interest. If you have another specific problem to propose, that you are willing to lead please do so. If you just wish to observe and comment, you are welcome also. 13/14
14. Thank you These slides are at https://speakerdeck.com/fjhickernell/samsi-qmc-wg-5-3- research-problem-proposal
15. Background Known Results Results Under Construction References Novak, E. & Woźniakowski, H. Tractability of Multivariate Problems Volume I: Linear Information. EMS Tracts in Mathematics 6 (European Mathematical Society, Zürich, 2008). Kühn, T., Sickel, W. & Ullrich, T. Approximation numbers of Sobolev embeddings—Sharp constants and tractability. J. Complexity 30, 95–116 (2014). Wu, C. F. J. & Hamada, M. Experiments: Planning, Analysis, and Parameter Design Optimization. (John Wiley & Sons, Inc., New York, 2000). 14/14