Exploiting low-dimensional ridge structures for design under uncertainty PAUL CONSTANTINE Assistant Professor Department of Computer Science University of Colorado, Boulder activesubspaces.org! @DrPaulynomial! SLIDES AVAILABLE UPON REQUEST DISCLAIMER: These slides are meant to complement the oral presentation. Use out of context at your own risk. SEQUOIA TEAM Specifically: Jeff Hokanson (CU Boulder) Juan Alonso, Charbel Farhat, Rick Feinrich, Victorien Menier (Stanford) 

•  Unmanned combat vehicle aircraft demonstrator, capable of carrier take-off and landing •  Complex nozzle shape integrated into aft end of vehicle •  Advanced materials and significant heat environment and thermal management issues •  Nozzle weight is a substantial portion of the overall propulsion system weight •  Uncertainties in all areas of multi- physics problem •  Complex multi-physics analysis and design problem Empty weight (kg) 6,350 TOGW (kg) 20,215 L/D cruise 12.6 - 15.6 Top speed High subsonic Service ceiling (ft) 40,000 Engine type F100-PW-20 Northrop Grumman X-47B UCAS Data-sheet, 2015 Nozzle aero-thermal-structural design 

1-D engine model Coarse FEM structural model Axisymmetric Euler / RANS aero Adaptive meshing 1-D Heat Transfer Medium-fidelity model ✔ Heat load Mech load •  Coupled, multi-physics model “MULTIF” •  Several levels of fidelity, focus today on “medium” fidelity •  Python wrappers make the coupled model a “black box” Nozzle aero-thermal-structural design PARAMETERIZATION FOR DESIGN UNDER UNCERTAINTY 62 design variables affect geometric characteristics •  nozzle shape •  baffle locations •  wall thicknesses along nozzle 40 variables represent uncertainties •  flight conditions •  material properties 

1-D engine model Coarse FEM structural model Axisymmetric Euler / RANS aero Adaptive meshing 1-D Heat Transfer Medium-fidelity model ✔ Heat load Mech load Nozzle aero-thermal-structural design minimize average mass thrust constraint structural constraints bound constraints 

Commentary on the “curse” We do not know the optimization is convex à INTRACTABLE The constraints are high dimensional integrals à INTRACTABLE However, intractability is a mathematical statement about worst case cost for an algorithm on a class of problems Methods that claim to “beat” the curse limit the class of problems to those with exploitable structure This nozzle design under uncertainty is a particular problem Given a particular problem, can we identify exploitable low-dimensional structure? The best way to fight the curse of dimensionality is to reduce the dimension 

f ( x ) ⇡ r X k=1 fk,1( x1) · · · fk,m( xm) f( x ) ⇡ p X k=1 ak k( x ), k a k0 ⌧ p f ( x ) ⇡ f1( x1) + · · · + fm( xm) Structure-exploiting methods for high-dimensional approximation STRUCTURE METHODS Sparse grids [Bungartz & Griebel (2004)], Smolyak methods [Constantine et al. (2012), Conrad & Marzouk (2013)], HDMR [Sobol (2003)], ANOVA [Hoeffding (1948)], … Separation of variables [Beylkin & Mohlenkamp (2005)], Tensor-train [Oseledets (2011)], Adaptive cross approximation [Bebendorff (2011)], Proper generalized decomposition [Chinesta et al. (2011)], … Compressed sensing [Donoho (2006), Candès and Wakin (2008)], … 

Ridge functions f( x ) = g(AT x ) A ridge function is constant along directions in its domain For u 2 null (AT ) f( x + u ) = g(AT ( x + u )) = g(AT x ) = f( x ) The effective dimension is the number of columns of A 

Ridge functions f( x ) = g(AT x ) Some relevant literature Approximation: Pinkus (2015), Diaconis and Shahshahani (1984), Donoho and Johnstone (1989) Ridge recovery: Fornasier et al. (2012), Cohen et al. (2012), Tyagi and Cevher (2014) Statistical regression: Friedman and Stuetzle (1981), Cook (1998), Li (1989) Uncertainty quantification: Tipireddy and Ghanem (2014); Lei et al. (2015); Stoyanov and Webster (2015); Tripathy, Bilionis, and Gonzalez (2016); Li, Lin, and Li (2016) 

Ridge functions f( x ) = g(AT x ) Why ridge functions? The low-dimensional structure … •  enables model fitting with less data Constantine, Eftekhari, and Ward (arXiv, 2016); Hokanson and Constantine (arXiv, 2017) R(U, g) = Z ⇣ f( x ) g(UT x ) ⌘2 ⇢( x ) d x Define: Solve: minimize R ( U, g ) subject to U 2 G ( m, n ) g 2 P N Grassmann manifold polynomials of degree N 

Ridge functions f( x ) = g(AT x ) Why ridge functions? The low-dimensional structure … •  enables model fitting with less data •  makes the high-dimensional integration and optimization more efficient---often from impractical to possible Constantine, Eftekhari, and Ward (arXiv, 2016); Hokanson and Constantine (arXiv, 2017) 

Ridge functions f( x ) = g(AT x ) Why ridge functions? Dimensional analysis (i.e., Buckingham Pi Theorem) implies that ridge are present systems whose variables have units Constantine, del Rosario, and Iaccarino (arXiv, 2016) 

Ridge functions f( x ) = g(AT x ) Why ridge functions? Because we can check a given problem for this type of exploitable, low-dimensional structure 

“The greatest value of a picture is when it forces us to notice what we never expected to see.” “Even more understanding is lost if we consider each thing we can do to data only in terms of some set of very restrictive assumptions under which that thing is best possible---assumptions we know we CANNOT check in practice.” “Exploratory data analysis is detective work …” 

www.youtube.com/watch?v=mJvKzjT6lmY 

Design a jet nozzle under uncertainty (DARPA SEQUOIA project) 10-parameter engine performance model (See animation at https://youtu.be/Fek2HstkFVc) 

Ridge functions f( x ) = g(AT x ) Why ridge functions? Because we can check a given problem for this type of exploitable, low-dimensional structure Beyond 1 and 2d, look at eigenvalues from active subspaces or sufficient dimension reduction analysis coupled with standard training / testing errors Constantine (2015); Glaws, Constantine, and Cook (arXiv, 2017) 

Ridge functions f( x ) = g(AT x ) Why ridge functions? Because they keep showing up in real applications 

Constantine, Emory, Larsson, and Iaccarino (2015) Evidence of structure: Multiphysics hypersonic scramjet model 

Jefferson, Gilbert, Constantine, and Maxwell (2015); Jefferson, Constantine, and Maxwell (2017) Evidence of structure: Integrated hydrologic model 

Lukaczyk, Constantine, Palacios, and Alonso (2014); Constantine (2015); Grey and Constantine (arXiv, 2017) Evidence of structure: Transonic wing design Wing perturbations 

T-cell count Loudon and Pankavich (2017) Evidence of structure: In-host HIV dynamics 

−2 −1 0 1 2 0 0.05 0.1 0.15 0.2 0.25 Active Variable 1 P max (watts) Constantine, Zaharatos, and Campanelli (2015) Evidence of structure: Solar-cell circuit model 

Evidence of structure: Atmospheric re-entry vehicle −1 0 1 ˆ wT p x 20000 40000 60000 80000 100000 Stagnation pressure pst −1 0 1 ˆ wT q x 0.4 0.6 0.8 1.0 1.2 Stagnation heat flux qst ×107 Cortesi, Constantine, Magin, and Congedo (In prep.) 

-1 0 1 wT 1 x 0 5 10 15 f(x) Average velocity Glaws, Constantine, Shadid, and Wildey (2017) Evidence of structure: Magnetohydrodynamics generator model 

2 0 2 wT x 3.2 3.4 3.6 Voltage [V] Constantine and Doostan (2017) Evidence of structure: Lithium-ion battery model 

Othmer, Lukaczyk, Constantine, and Alonso (2016) Evidence of structure: Automobile design 

Gilbert, Jefferson, Constantine, and Maxwell (2016) Evidence of no 1-d structure: A subsurface hydrology problem 0 100 200 300 0 100 200 300 0 20 40 x (m) y (m) z (m) Student Version of MATLAB Domain Hydraulic conductivities 

Jupyter notebooks: github.com/paulcon/as-data-sets 

WHY RIDGE FUNCTIONS? f( x ) = g(AT x ) Low effective dimension enables otherwise impractical computational studies (dimension reduction) Theoretical arguments from dimensional analysis Computational tools for checking for ridge structure They keep showing up in applications 

1-D engine model Coarse FEM structural model Axisymmetric Euler / RANS aero Adaptive meshing 1-D Heat Transfer Medium-fidelity model ✔ Heat load Mech load HOW DO WE USE RIDGES FOR THE DUU? minimize average mass thrust constraint structural constraints bound constraints 

31 MULTIF model shows nonsmooth behavior in randomized parameter sweeps We are formalizing a randomized, signal processing- based method for measuring “computational noise” using random parameter sweeps Randomized parameter sweeps to assess noise in MULTIF combined design and uncertain space 

32 Exploiting ridges in DUU objective and constraint functions METHOD 1.  fit 1-d ridge functions to constraint and objective functions 2.  make 1-d plots to assess ridge structure and monotonicity 3.  if present, use ridge direction to transform DUU into linear program 4.  solve linear program for design point 5.  run forward UQ on ridge surrogate for probabilities 

33 Exploiting ridges in DUU objective and constraint functions METHOD 1.  fit 1-d ridge functions to constraint and objective functions 2.  make 1-d plots to assess ridge structure and monotonicity 3.  if present, use ridge direction to transform DUU into linear program 4.  solve linear program for design point 5.  run forward UQ on ridge surrogate for probabilities 

minimize x c T x subject to a T 1 x < b1 a T 2 x < b2 a T 3 x < b3 x 2 D 62 design vars, 40 uncertain vars, 13 prob constraints, 1 objective, 1k runs E( mass ) [kg] Baseline 118.0 DUU 37.6 smaller is better right is better DUU Base DUU Base DUU Base right is better left is better Initial pressure contours DUU pressure contours 

SUMMARY AND CLARIFICATIONS Not a generic method for DUU Exploratory analysis for exploitable structure with a given DUU problem Heuristics for exploiting this type of low-d structure within the given DUU problem TODO: “Comparison” with other DUU methods the SEQUOIA team is developing 

www.siam.org/meetings/dr17/ http://www.siam.org/journals/juq/juq_special.php 

How does this relate to active subspaces? Tell me again how you dealt with those probability constraints. Will this work on my problem? PAUL CONSTANTINE Assistant Professor University of Colorado Boulder activesubspaces.org! @DrPaulynomial! QUESTIONS? Active Subspaces SIAM (2015)