Paul Constantine
November 06, 2018
62

# Off-axis anisotropy in multivariate functions

Slides for my talk at the RICAM Multivariate Algorithms and Information-Based Complexity Workshop, Nov 6 (my birthday!), 2018

#### Paul Constantine

November 06, 2018

## Transcript

1. ### Off-axis anisotropy in multivariate functions PAUL CONSTANTINE Assistant Professor Department

of Computer Science University of Colorado, Boulder activesubspaces.org! @DrPaulynomial! SLIDES AVAILABLE UPON REQUEST DISCLAIMER: These slides are meant to complement the oral presentation. Use out of context at your own risk. Thanks to: Jeff Hokanson (CU Boulder) Andrew Glaws (CU Boulder, NREL)
2. ### Hypersonic scramjet models Constantine, Emory, Larsson, and Iaccarino (2015) Aerospace

design Lukaczyk, Palacios, Alonso, and Constantine (2014) Integrated hydrologic models Jefferson, Gilbert, Constantine, and Maxwell (2015) Solar cell models Constantine, Zaharatos, and Campanelli (2015) Magnetohydrodynamics models Glaws, Constantine, Shadid, and Wildey (2017) Ebola transmission models Diaz, Constantine, Kalmbach, Jones, and Pankavich (2018) Lithium ion battery model Constantine and Doostan (2017) Automobile design Othmer, Lukaczyk, Constantine, and Alonso (2016) f( x )
3. ### f( x ) PROPERTIES: Computer model of a physical system

Several independent inputs Deterministic Continuous inputs / outputs “Smoothness” Z f( x ) d x APPROXIMATION OPTIMIZATION INTEGRATION ˜ f( x ) ⇡ f( x ) minimize x f( x ) TO DO:
4. ### f ( x ) ⇡ r X k=1 fk,1( x1)

· · · fk,m( xm) f( x ) ⇡ p X k=1 ak k( x ), k a k0 ⌧ p f ( x ) ⇡ f1( x1) + · · · + fm( xm) Structure-exploiting methods STRUCTURE METHODS Separation of variables [Beylkin & Mohlenkamp (2005)], Tensor-train [Oseledets (2011)], Adaptive cross approximation [Bebendorff (2011)], Proper generalized decomposition [Chinesta et al. (2011)], … Compressed sensing [Donoho (2006), Candès & Wakin (2008)], … Sparse grids [Bungartz & Griebel (2004)], HDMR [Sobol (2003)], ANOVA [Hoeffding (1948)], QMC [Niederreiter (1992)], …
5. ### “Even more understanding is lost if we consider each thing

we can do to data only in terms of some set of very restrictive assumptions under which that thing is best possible--- assumptions we know we CANNOT check in practice.”
6. ### The best way to ﬁght the curse is to reduce

the dimension. But what is dimension reduction? •  dimensional analysis [Barrenblatt (1996)] •  correlation-based reduction [Jolliffe (2002)] •  sensitivity analysis [Saltelli et al. (2008)]

8. ### Design a jet nozzle under uncertainty (DARPA SEQUOIA project) 10-parameter

engine performance model (See animation at https://youtu.be/Fek2HstkFVc)
9. ### f( x ) ⇡ g(UT x ) Ridge approximations UT

: Rm ! Rn g : Rn ! R where Constantine, Eftekhari, Hokanson, and Ward (2017) f(x1 , x2 ) x2 x1
10. ### Ridge approximations A subset of related literature Approximation theory: Mayer

et al. (2015), Pinkus (2015), Diaconis and Shahshahani (1984), Donoho and Johnstone (1989) Compressed sensing: Fornasier et al. (2012), Cohen et al. (2012), Tyagi and Cevher (2014) Statistical regression: Friedman and Stuetzle (1981), Ichimura (1993), Hristache et al. (2001), Xia et al. (2002) Uncertainty quantiﬁcation & computational science: Tipireddy and Ghanem (2014); Lei et al. (2015); Stoyanov and Webster (2015); Tripathy, Bilionis, and Gonzalez (2016); Li, Lin, and Li (2016); … f( x ) ⇡ g(UT x )

I care?)
12. ### Hypersonic scramjet models Constantine, Emory, Larsson, and Iaccarino (2015) Evidence

of 1d ridge structures across science and engineering models
13. ### Integrated jet nozzle models Alonso, Eldred, Constantine, Duraisamy, Farhat, Iaccarino,

and Jakeman (2017) Evidence of 1d ridge structures across science and engineering models
14. ### Integrated hydrologic models Jefferson, Gilbert, Constantine, and Maxwell (2015) Evidence

of 1d ridge structures across science and engineering models
15. ### −2 −1 0 1 2 −0.1 0 0.1 0.2 0.3

0.4 0.5 0.6 0.7 0.8 0.9 Active Variable 1 Lift Lukaczyk, Constantine, Palacios, and Alonso (2014) −2 −1 0 1 2 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 Active Variable 1 Drag Aerospace vehicle geometries Evidence of 1d ridge structures across science and engineering models
16. ### In-host HIV dynamical models T-cell count Loudon and Pankavich (2016)

Evidence of 1d ridge structures across science and engineering models
17. ### Solar cell circuit models −2 −1 0 1 2 0

0.05 0.1 0.15 0.2 0.25 Active Variable 1 P max (watts) Constantine, Zaharatos, and Campanelli (2015) Evidence of 1d ridge structures across science and engineering models
18. ### Atmospheric reentry vehicle model Cortesi, Constantine, Magin, and Congedo (hal,

2017) −1 0 1 ˆ wT q x 0.4 0.6 0.8 1.0 1.2 Stagnation heat ﬂux qst ×107 −1 0 1 ˆ wT p x 20000 40000 60000 80000 100000 Stagnation pressure pst Evidence of 1d ridge structures across science and engineering models
19. ### Magnetohydrodynamics generator model -1 0 1 wT 1 x 0

5 10 15 f(x) Average velocity Glaws, Constantine, Shadid, and Wildey (2017) -1 0 1 wT 1 x 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 f(x) Induce magnetic field Evidence of 1d ridge structures across science and engineering models
20. ### Lithium ion battery model 2 0 2 wT x 3.65

3.70 Voltage [V] Constantine and Doostan (2017) 2 0 2 wT x 2.0 2.2 Capacity [mAh·cm 2] Evidence of 1d ridge structures across science and engineering models
21. ### Automobile geometries Othmer, Lukaczyk, Constantine, and Alonso (2016) Evidence of

1d ridge structures across science and engineering models
22. ### -4 -2 0 2 4 Quantity of interest #10-3 0

1 2 3 4 5 -4 -2 0 2 4 Quantity of interest #10-4 2.4 2.6 2.8 3 3.2 3.4 3.6 3.8 Long length scale Short length scale Constantine, Dow, and Wang (2014) r · (aru) = 1, s 2 D u = 0, s 2 1 n · aru = 0, s 2 2 Input field Solution Short corr Long corr Evidence of 1d ridge structures across science and engineering models

24. ### f( x ) ⇡ g(UT x ) What is U?

What is the approximation error? What is g? Constantine, Eftekhari, Hokanson, and Ward (2017) Ridge approximations

26. ### C = Z rf( x ) rf( x )T ⇢(

x ) d x = W ⇤W T Deﬁne the active subspace The average outer product of the gradient and its eigendecomposition, Partition the eigendecomposition, Rotate and separate the coordinates, ⇤ =  ⇤1 ⇤2 , W = ⇥ W 1 W 2 ⇤ , W 1 2 Rm⇥n x = W W T x = W 1W T 1 x + W 2W T 2 x = W 1y + W 2z active variables inactive variables f = f( x ), x 2 Rm, rf( x ) 2 Rm, ⇢ : Rm ! R + Constantine, Dow, and Wang (2014) Some relevant literature Statistical regression: Samarov (1993), Hristache et al. (2001) Machine learning: Mukerjee, Wu, and Xiao (2010); Fukumizu and Leng (2014) Detection and estimation theory: van Trees (2001) The function, its gradient vector, and a given weight function:
27. ### C = Z rf( x ) rf( x )T ⇢(

x ) d x = W ⇤W T Deﬁne the active subspace The function, its gradient vector, and a given weight function: The average outer product of the gradient and its eigendecomposition: f = f( x ), x 2 Rm, rf( x ) 2 Rm, ⇢ : Rm ! R + Constantine, Dow, and Wang (2014) i = Z w T i rf( x ) 2 ⇢( x ) d x , i = 1, . . . , m average, squared, directional derivative along eigenvector eigenvalue Eigenvalues measure ridge structure with eigenvectors:
28. ### conditional average active subspace Poincaré constant eigenvalues associated with inactive

subspace f( x ) µ(W T 1 x ) L2(⇢)  C ( n+1 + · · · + m)1 2 Constantine, Dow, and Wang (2014) Eigenvalues control the approximation error
29. ### (1) Draw samples: (2) Compute: and fj = f( xj)

(3) Approximate with Monte Carlo, and compute eigendecomposition Equivalent to SVD of samples of the gradient Called an active subspace method in T. Russi’s 2010 Ph.D. thesis, Uncertainty Quantification with Experimental Data in Complex System Models C ⇡ 1 N N X j=1 rfj rfT j = ˆ W ˆ ⇤ ˆ W T 1 p N ⇥ rf1 · · · rfN ⇤ = ˆ W p ˆ ⇤ ˆ V T rfj = rf( xj) Constantine, Dow, and Wang (2014), Constantine and Gleich (2015, arXiv) xj ⇠ ⇢( x ) Estimate the active subspace with Monte Carlo
30. ### 1 p N ⇥ rf1 · · · rfN ⇤

⇡ ˆ W 1 q ˆ ⇤1 ˆ V T 1 Low-rank approximation of the collection of gradients: Low-dimensional linear approximation of the gradient: f( x ) ⇡ g ⇣ ˆ W T 1 x ⌘ Approximate a function of many variables by a function of a few linear combinations of the variables: ✔ ✖ ✖ Remember the problem to solve span ( ˆ W 1) ⇡ { rf( x ) : x 2 supp ⇢( x ) }
31. ### N = ⌦ ✓ L2 1 2 k "2 log(

m ) ◆ = ) | k ˆk |  k " How many gradient samples? number of samples eigenvalue error (w.h.p.) subspace error (w.h.p.) Constantine and Gleich (2015) via Gittens and Tropp (2011), Stewart (1973) N = ⌦ ✓ L2 1"2 log( m ) ◆ = ) dist( W 1, ˆ W 1)  4 1" n n+1 bound on gradient dimension number of samples bound on gradient dimension
32. ### In practice, bootstrap Constantine and Gleich (2015, arXiv) Index 1

2 3 4 5 6 Eigenvalues 10-8 10-6 10-4 10-2 100 102 104 True Est BI Index 1 2 3 4 5 6 Eigenvalues 10-8 10-6 10-4 10-2 100 102 104 Subspace Dimension 1 2 3 4 5 6 Subspace Error 10-6 10-4 10-2 100 True Est BI Subspace Dimension 1 2 3 4 5 6 Subspace Error 10-6 10-4 10-2 100 Eigenvalue estimates and subspace error estimates with bootstrap intervals from quadratic function of 10 variables
33. ### Subspace error Eigenvalues for active variables Eigenvalues for inactive variables

Recall the subspace error: Effect of estimated eigenvectors? Constantine, Dow, and Wang (2014) " = dist( W 1, ˆ W 1) / O (eigval. error) n n+1 f( x ) µ( ˆ W T 1 x ) L2(⇢)  C ⇣ " ( 1 + · · · + n)1 2 + ( n+1 + · · · + m)1 2 ⌘

35. ### An example where it doesn’t work active subspace U =

[0; 1] inactive subspace U = [1; 0] f ( x1, x2) = 5 x1 + sin(10 ⇡x2) Constantine, Eftekhari, Hokanson, and Ward (2017) C = “ Z rf rfT ” =  25 0 0 526 <latexit sha1_base64="VTalW+3KthfPSuX7r/T/HHiOg+4=">AAAWgnicjZjvU9s2GMfdrtu67BfdbrzZG2+5/tiO6yVs3Xa77a6FUgqllBYSCJilsqPEAv+qLIeEnP/Gvd4fsrfbJNsJjvWFkTuIou/n0fPIsqRHsiOPxaLR+OvGzfduvf/Bh7c/qn38yaeffb5w54t2HCbcoS0n9EJ+YJOYeiygLcGERw8iTolve3TfPl1V+v6Q8piFwZ4YR/TYJ4OA9ZlDhKzqLjDLXzXv/W5ago7E5O3b1GKBMK2A2B4x+9bStPTHnixnzP37qWXVlIlNByyY2D4RnI3S2vIj857ZMC1L/rtnPlr+qWbRoDfTuwv1xsNG9jH1QrMo1I3is9O989UNqxc6iU8D4Xgkjo+ajUgcTwgXzPFoWrOSmEbEOSUDeiSLAfFpfDzJnklq3pU1PbMfcvkne5TVli0mxI/jsW9LUgboxlVNVSLtKBH9X44nLIgSQQMnd9RPPFOEpnrAZo9x6ghvLAvE4UzGajou4cQRchhqNSvi4ZD1qBP6PpGPx5Ie0qPm8cSyQ6+nIgq9Sb2ZpndrFqcBPZuBQ+rkoIrJ7mdQzaoQxMOt1Wp3zdWci2VRqWYSRZSbjnx79KjomKaZp8mG8qIF/USqNIgTTlU0OfkkhegKQFcwugrQVYw+BehTjK4BdA2jzwD6DKPrAF3H6HOAPsfoBkA3MLoJ0E2MvgDoC4xuAXQLoy8B+hKj2wDdxugrgL7C6A5AdzD6GqCvMfoGoG8wugvQXYzuAXQPoy2AtjDaBmgbo/sA3cfoAUAPMNoBaAejhwA9vOQllBtYjwDeKhQ8GmzgQ6NcgDbnlIfApKFoHQ8DGqfZOjuhqLlh1b0iseehDVAbow5AHYz2ANrDKAUoxWgfoH2MDgA6wKgLUBejDKAMoycAPcHoKUBPMeoB1MOoD1AfowFAA4xW31KFhhiNABph9B1A32GUA5RjNAZojFEBUIHRBKAJRocAHWL0DKBnGB0BdITRMUDHGD0H6PklAwumgUz4JlaUzQWZ18Uy44zEXGZXThDjJ3lS50ozla5V1JWSuqKpayV1TVM3SuqGpm6W1E1N3S6p25q6W1J3NfWgpB5oaqekdvJH5BIRfzOPuVni6aqtQpbSqrg+E9c1cbZTueXdqQLtuVTMmPyH5qQ1c9LSxfZMbOvi/kzc10RvLjwPhyfK4QkYXjZNMl2WNHE8E8e6eD4TzzWxU2gdTTkslENNiTuz/l4MbPnl98Iz/PKr+eu4LGOzA9CAU3o6D9nEi1z1MCRQHJ+svCp3EfrULA58zvQ4NdcAi9KjZXn+8mhfyAceDDxq1ptL1lJ92eJs4AqLZ5WVbq2NotLhzpYTLGvhqN7MrY4r/Go4zP2EcqYTEXJ1+p2o2qldyaVuzDm0ltXXMG8Tnsc6Z61qr455h4d27rbo5M7U4BJH9sXZeO6qQToo3yxUrHq9Ue6lLw/dcm1U9wXEk4OQXvwYdSf15bRqSbjME0e5xyToyfM6FZPMbq6nBYasWXAtaxZo1rKO/a+tgvzE143J6BrGEkLGsdqDs6udOLFPqCPUVUYVEvJZUjDqeX02jA+m4/5dNlXU1YtNxjRmJDBZUFxCzbcacRaqvIK7YbfScMS1QDN6fCm+pC0+mcH55QbaguSxU0wrQXvPnFDlGetE/u5mqXfVfZg9WuQ8jLXmnBx3rscyRap9v7I6DorFcaCtm1nykK2bKluotOfmDZaACjGgkczqBl15xOGyyLwwqPaWZYj8vgJyCkq5qoBaT/Lm8s5c0aQ7dTwN/Sr3bsk/5LX9vWh4q6psZe1sXRXY1kVcW1dFNRI8UYexoVyV8omYVagdp7tQb1avSPVCe/lhU5Zf/1h//GtxfXrb+Nr41nhgNI2fjcfGc2PHaBmO8afxt/GP8e/ircXvF5uLP+TozRuFzZfG3Gfxt/8AMi7iZg==</latexit> <latexit sha1_base64="VTalW+3KthfPSuX7r/T/HHiOg+4=">AAAWgnicjZjvU9s2GMfdrtu67BfdbrzZG2+5/tiO6yVs3Xa77a6FUgqllBYSCJilsqPEAv+qLIeEnP/Gvd4fsrfbJNsJjvWFkTuIou/n0fPIsqRHsiOPxaLR+OvGzfduvf/Bh7c/qn38yaeffb5w54t2HCbcoS0n9EJ+YJOYeiygLcGERw8iTolve3TfPl1V+v6Q8piFwZ4YR/TYJ4OA9ZlDhKzqLjDLXzXv/W5ago7E5O3b1GKBMK2A2B4x+9bStPTHnixnzP37qWXVlIlNByyY2D4RnI3S2vIj857ZMC1L/rtnPlr+qWbRoDfTuwv1xsNG9jH1QrMo1I3is9O989UNqxc6iU8D4Xgkjo+ajUgcTwgXzPFoWrOSmEbEOSUDeiSLAfFpfDzJnklq3pU1PbMfcvkne5TVli0mxI/jsW9LUgboxlVNVSLtKBH9X44nLIgSQQMnd9RPPFOEpnrAZo9x6ghvLAvE4UzGajou4cQRchhqNSvi4ZD1qBP6PpGPx5Ie0qPm8cSyQ6+nIgq9Sb2ZpndrFqcBPZuBQ+rkoIrJ7mdQzaoQxMOt1Wp3zdWci2VRqWYSRZSbjnx79KjomKaZp8mG8qIF/USqNIgTTlU0OfkkhegKQFcwugrQVYw+BehTjK4BdA2jzwD6DKPrAF3H6HOAPsfoBkA3MLoJ0E2MvgDoC4xuAXQLoy8B+hKj2wDdxugrgL7C6A5AdzD6GqCvMfoGoG8wugvQXYzuAXQPoy2AtjDaBmgbo/sA3cfoAUAPMNoBaAejhwA9vOQllBtYjwDeKhQ8GmzgQ6NcgDbnlIfApKFoHQ8DGqfZOjuhqLlh1b0iseehDVAbow5AHYz2ANrDKAUoxWgfoH2MDgA6wKgLUBejDKAMoycAPcHoKUBPMeoB1MOoD1AfowFAA4xW31KFhhiNABph9B1A32GUA5RjNAZojFEBUIHRBKAJRocAHWL0DKBnGB0BdITRMUDHGD0H6PklAwumgUz4JlaUzQWZ18Uy44zEXGZXThDjJ3lS50ozla5V1JWSuqKpayV1TVM3SuqGpm6W1E1N3S6p25q6W1J3NfWgpB5oaqekdvJH5BIRfzOPuVni6aqtQpbSqrg+E9c1cbZTueXdqQLtuVTMmPyH5qQ1c9LSxfZMbOvi/kzc10RvLjwPhyfK4QkYXjZNMl2WNHE8E8e6eD4TzzWxU2gdTTkslENNiTuz/l4MbPnl98Iz/PKr+eu4LGOzA9CAU3o6D9nEi1z1MCRQHJ+svCp3EfrULA58zvQ4NdcAi9KjZXn+8mhfyAceDDxq1ptL1lJ92eJs4AqLZ5WVbq2NotLhzpYTLGvhqN7MrY4r/Go4zP2EcqYTEXJ1+p2o2qldyaVuzDm0ltXXMG8Tnsc6Z61qr455h4d27rbo5M7U4BJH9sXZeO6qQToo3yxUrHq9Ue6lLw/dcm1U9wXEk4OQXvwYdSf15bRqSbjME0e5xyToyfM6FZPMbq6nBYasWXAtaxZo1rKO/a+tgvzE143J6BrGEkLGsdqDs6udOLFPqCPUVUYVEvJZUjDqeX02jA+m4/5dNlXU1YtNxjRmJDBZUFxCzbcacRaqvIK7YbfScMS1QDN6fCm+pC0+mcH55QbaguSxU0wrQXvPnFDlGetE/u5mqXfVfZg9WuQ8jLXmnBx3rscyRap9v7I6DorFcaCtm1nykK2bKluotOfmDZaACjGgkczqBl15xOGyyLwwqPaWZYj8vgJyCkq5qoBaT/Lm8s5c0aQ7dTwN/Sr3bsk/5LX9vWh4q6psZe1sXRXY1kVcW1dFNRI8UYexoVyV8omYVagdp7tQb1avSPVCe/lhU5Zf/1h//GtxfXrb+Nr41nhgNI2fjcfGc2PHaBmO8afxt/GP8e/ircXvF5uLP+TozRuFzZfG3Gfxt/8AMi7iZg==</latexit> <latexit sha1_base64="VTalW+3KthfPSuX7r/T/HHiOg+4=">AAAWgnicjZjvU9s2GMfdrtu67BfdbrzZG2+5/tiO6yVs3Xa77a6FUgqllBYSCJilsqPEAv+qLIeEnP/Gvd4fsrfbJNsJjvWFkTuIou/n0fPIsqRHsiOPxaLR+OvGzfduvf/Bh7c/qn38yaeffb5w54t2HCbcoS0n9EJ+YJOYeiygLcGERw8iTolve3TfPl1V+v6Q8piFwZ4YR/TYJ4OA9ZlDhKzqLjDLXzXv/W5ago7E5O3b1GKBMK2A2B4x+9bStPTHnixnzP37qWXVlIlNByyY2D4RnI3S2vIj857ZMC1L/rtnPlr+qWbRoDfTuwv1xsNG9jH1QrMo1I3is9O989UNqxc6iU8D4Xgkjo+ajUgcTwgXzPFoWrOSmEbEOSUDeiSLAfFpfDzJnklq3pU1PbMfcvkne5TVli0mxI/jsW9LUgboxlVNVSLtKBH9X44nLIgSQQMnd9RPPFOEpnrAZo9x6ghvLAvE4UzGajou4cQRchhqNSvi4ZD1qBP6PpGPx5Ie0qPm8cSyQ6+nIgq9Sb2ZpndrFqcBPZuBQ+rkoIrJ7mdQzaoQxMOt1Wp3zdWci2VRqWYSRZSbjnx79KjomKaZp8mG8qIF/USqNIgTTlU0OfkkhegKQFcwugrQVYw+BehTjK4BdA2jzwD6DKPrAF3H6HOAPsfoBkA3MLoJ0E2MvgDoC4xuAXQLoy8B+hKj2wDdxugrgL7C6A5AdzD6GqCvMfoGoG8wugvQXYzuAXQPoy2AtjDaBmgbo/sA3cfoAUAPMNoBaAejhwA9vOQllBtYjwDeKhQ8GmzgQ6NcgDbnlIfApKFoHQ8DGqfZOjuhqLlh1b0iseehDVAbow5AHYz2ANrDKAUoxWgfoH2MDgA6wKgLUBejDKAMoycAPcHoKUBPMeoB1MOoD1AfowFAA4xW31KFhhiNABph9B1A32GUA5RjNAZojFEBUIHRBKAJRocAHWL0DKBnGB0BdITRMUDHGD0H6PklAwumgUz4JlaUzQWZ18Uy44zEXGZXThDjJ3lS50ozla5V1JWSuqKpayV1TVM3SuqGpm6W1E1N3S6p25q6W1J3NfWgpB5oaqekdvJH5BIRfzOPuVni6aqtQpbSqrg+E9c1cbZTueXdqQLtuVTMmPyH5qQ1c9LSxfZMbOvi/kzc10RvLjwPhyfK4QkYXjZNMl2WNHE8E8e6eD4TzzWxU2gdTTkslENNiTuz/l4MbPnl98Iz/PKr+eu4LGOzA9CAU3o6D9nEi1z1MCRQHJ+svCp3EfrULA58zvQ4NdcAi9KjZXn+8mhfyAceDDxq1ptL1lJ92eJs4AqLZ5WVbq2NotLhzpYTLGvhqN7MrY4r/Go4zP2EcqYTEXJ1+p2o2qldyaVuzDm0ltXXMG8Tnsc6Z61qr455h4d27rbo5M7U4BJH9sXZeO6qQToo3yxUrHq9Ue6lLw/dcm1U9wXEk4OQXvwYdSf15bRqSbjME0e5xyToyfM6FZPMbq6nBYasWXAtaxZo1rKO/a+tgvzE143J6BrGEkLGsdqDs6udOLFPqCPUVUYVEvJZUjDqeX02jA+m4/5dNlXU1YtNxjRmJDBZUFxCzbcacRaqvIK7YbfScMS1QDN6fCm+pC0+mcH55QbaguSxU0wrQXvPnFDlGetE/u5mqXfVfZg9WuQ8jLXmnBx3rscyRap9v7I6DorFcaCtm1nykK2bKluotOfmDZaACjGgkczqBl15xOGyyLwwqPaWZYj8vgJyCkq5qoBaT/Lm8s5c0aQ7dTwN/Sr3bsk/5LX9vWh4q6psZe1sXRXY1kVcW1dFNRI8UYexoVyV8omYVagdp7tQb1avSPVCe/lhU5Zf/1h//GtxfXrb+Nr41nhgNI2fjcfGc2PHaBmO8afxt/GP8e/ircXvF5uLP+TozRuFzZfG3Gfxt/8AMi7iZg==</latexit> <latexit sha1_base64="VTalW+3KthfPSuX7r/T/HHiOg+4=">AAAWgnicjZjvU9s2GMfdrtu67BfdbrzZG2+5/tiO6yVs3Xa77a6FUgqllBYSCJilsqPEAv+qLIeEnP/Gvd4fsrfbJNsJjvWFkTuIou/n0fPIsqRHsiOPxaLR+OvGzfduvf/Bh7c/qn38yaeffb5w54t2HCbcoS0n9EJ+YJOYeiygLcGERw8iTolve3TfPl1V+v6Q8piFwZ4YR/TYJ4OA9ZlDhKzqLjDLXzXv/W5ago7E5O3b1GKBMK2A2B4x+9bStPTHnixnzP37qWXVlIlNByyY2D4RnI3S2vIj857ZMC1L/rtnPlr+qWbRoDfTuwv1xsNG9jH1QrMo1I3is9O989UNqxc6iU8D4Xgkjo+ajUgcTwgXzPFoWrOSmEbEOSUDeiSLAfFpfDzJnklq3pU1PbMfcvkne5TVli0mxI/jsW9LUgboxlVNVSLtKBH9X44nLIgSQQMnd9RPPFOEpnrAZo9x6ghvLAvE4UzGajou4cQRchhqNSvi4ZD1qBP6PpGPx5Ie0qPm8cSyQ6+nIgq9Sb2ZpndrFqcBPZuBQ+rkoIrJ7mdQzaoQxMOt1Wp3zdWci2VRqWYSRZSbjnx79KjomKaZp8mG8qIF/USqNIgTTlU0OfkkhegKQFcwugrQVYw+BehTjK4BdA2jzwD6DKPrAF3H6HOAPsfoBkA3MLoJ0E2MvgDoC4xuAXQLoy8B+hKj2wDdxugrgL7C6A5AdzD6GqCvMfoGoG8wugvQXYzuAXQPoy2AtjDaBmgbo/sA3cfoAUAPMNoBaAejhwA9vOQllBtYjwDeKhQ8GmzgQ6NcgDbnlIfApKFoHQ8DGqfZOjuhqLlh1b0iseehDVAbow5AHYz2ANrDKAUoxWgfoH2MDgA6wKgLUBejDKAMoycAPcHoKUBPMeoB1MOoD1AfowFAA4xW31KFhhiNABph9B1A32GUA5RjNAZojFEBUIHRBKAJRocAHWL0DKBnGB0BdITRMUDHGD0H6PklAwumgUz4JlaUzQWZ18Uy44zEXGZXThDjJ3lS50ozla5V1JWSuqKpayV1TVM3SuqGpm6W1E1N3S6p25q6W1J3NfWgpB5oaqekdvJH5BIRfzOPuVni6aqtQpbSqrg+E9c1cbZTueXdqQLtuVTMmPyH5qQ1c9LSxfZMbOvi/kzc10RvLjwPhyfK4QkYXjZNMl2WNHE8E8e6eD4TzzWxU2gdTTkslENNiTuz/l4MbPnl98Iz/PKr+eu4LGOzA9CAU3o6D9nEi1z1MCRQHJ+svCp3EfrULA58zvQ4NdcAi9KjZXn+8mhfyAceDDxq1ptL1lJ92eJs4AqLZ5WVbq2NotLhzpYTLGvhqN7MrY4r/Go4zP2EcqYTEXJ1+p2o2qldyaVuzDm0ltXXMG8Tnsc6Z61qr455h4d27rbo5M7U4BJH9sXZeO6qQToo3yxUrHq9Ue6lLw/dcm1U9wXEk4OQXvwYdSf15bRqSbjME0e5xyToyfM6FZPMbq6nBYasWXAtaxZo1rKO/a+tgvzE143J6BrGEkLGsdqDs6udOLFPqCPUVUYVEvJZUjDqeX02jA+m4/5dNlXU1YtNxjRmJDBZUFxCzbcacRaqvIK7YbfScMS1QDN6fCm+pC0+mcH55QbaguSxU0wrQXvPnFDlGetE/u5mqXfVfZg9WuQ8jLXmnBx3rscyRap9v7I6DorFcaCtm1nykK2bKluotOfmDZaACjGgkczqBl15xOGyyLwwqPaWZYj8vgJyCkq5qoBaT/Lm8s5c0aQ7dTwN/Sr3bsk/5LX9vWh4q6psZe1sXRXY1kVcW1dFNRI8UYexoVyV8omYVagdp7tQb1avSPVCe/lhU5Zf/1h//GtxfXrb+Nr41nhgNI2fjcfGc2PHaBmO8afxt/GP8e/ircXvF5uLP+TozRuFzZfG3Gfxt/8AMi7iZg==</latexit>
36. ### f( x ) ⇡ g(UT x ) What is U?

Define the error function: R(U) = 1 2 Z (f( x ) µ(UT x ))2 ⇢( x ) d x Minimize the error: minimize U R ( U ) subject to U 2 G ( n, m ) Grassmann manifold Constantine, Eftekhari, Hokanson, and Ward (2017) Ridge approximations best approximation
37. ### k ¯ rR(W 1)kF  L ⇣ 2m1 2 +

(m n)1 2 ⌘ ( n+1 + · · · + m)1 2 The active subspace is nearly stationary Assume (1) Lipschitz continuous function (2) Gaussian density function gradient on the Grassmann manifold active subspace Frobenius norm dimensions Lipschitz constant eigenvalues associated with inactive subspace Constantine, Eftekhari, Hokanson, and Ward (2017)
38. ### (1) Choose points: (2) Compute: fj = f( xj) (3)

Minimize the misfit Minimize over polynomials and subspaces xj ⇠ ⇢( x ) Estimate the optimal subspace with discrete least squares minimize g2P p(Rn) U2G(n,m) N X j=1 ⇣ fj g(UT xj) ⌘2 Constantine, Eftekhari, Hokanson, and Ward (2017), Hokanson and Constantine (2018)
39. ### Alternating minimization Constantine, Eftekhari, Hokanson, and Ward (2017), Hokanson and

Constantine (2018) Variable projection Given subspace, fit polynomial Given polynomial coefficients, minimize over subspace Repeat Use pseudoinverse of Vandermonde matrix to express optimal polynomial coefficients Compute the derivative of the pseudoinverse of the Vandermonde matrix [Golub & Pereyra (1973)] on the Grassmann manifold [Edelman et al. (1998)] Run Newton on loss function Two contenders for the least squares problem minimize g2P p(Rn) U2G(n,m) N X j=1 ⇣ fj g(UT xj) ⌘2

41. ### r · (aru) = 1, s 2 D u =

0, s 2 1 n · aru = 0, s 2 2 Input field Solution Hokanson and Constantine (2018)
42. ### Constantine, Eftekhari, Hokanson, and Ward (2017), Hokanson and Constantine (2018)

Details under the hood How do you choose points? How do you choose the subspace dimension? What is your polynomial basis for the ridge approximation? Grassmann gradient of pseudo-inverse is complicated but has some nice simplifications Open questions Can you use other bases (frames, RBFs, …) ? Is the continuous optimization problem well-posed? How does the discrete optimization relate to the continuous optimization?