Learn Continuous Structure

A gentle introduction to equadratures.

1. LEARN CONTINUOUS STRUCTURE Pranay Seshadri Associate Professor in Aerospace Engineering Georgia Institute of Technology
2. LEARN CONTINUOUS STRUCTURE INTRODUCTION equadratures is an open-source python code that is purpose built for developing models using orthogonal polynomials. These models can be fit directly to real-world sensor data, simulation data, or even to other surrogate models. As you will discover, using orthogonal polynomials is very advantageous and can tell us a lot about the underlying data we are modelling. A FEW WORDS
3. LEARN CONTINUOUS STRUCTURE OVERVIEW 1. FOURIER SERIES 2. ORTHOGONAL POLYNOMIALS 3. PARAMETERS 4. PARAMETERS & ORTHOGONAL POLYNOMIALS
4. LEARN CONTINUOUS STRUCTURE Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a Fourier series JB Joseph Fourier FOURIER SERIES
5. LEARN CONTINUOUS STRUCTURE Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a Fourier series JB Joseph Fourier FOURIER SERIES where and
6. LEARN CONTINUOUS STRUCTURE Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a Fourier series JB Joseph Fourier FOURIER SERIES where and 3 very important observations!
7. LEARN CONTINUOUS STRUCTURE Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a Fourier series JB Joseph Fourier FOURIER SERIES where and The function sits inside the integral. 1
8. LEARN CONTINUOUS STRUCTURE Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a Fourier series JB Joseph Fourier FOURIER SERIES where and The basis terms are orthogonal. 2
9. LEARN CONTINUOUS STRUCTURE Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a Fourier series JB Joseph Fourier FOURIER SERIES where and Computing the basis terms requires integration. 3
10. LEARN CONTINUOUS STRUCTURE FOURIER SERIES The function sits inside the integral. 1 The basis terms are orthogonal. 2 Computing the basis terms requires integration. 3 But, what is orthogonality?
11. LEARN CONTINUOUS STRUCTURE FOURIER SERIES We define orthogonality as where
12. LEARN CONTINUOUS STRUCTURE OVERVIEW 1. FOURIER SERIES 2. ORTHOGONAL POLYNOMIALS 3. PARAMETERS 4. PARAMETERS & ORTHOGONAL POLYNOMIALS
13. LEARN CONTINUOUS STRUCTURE ORTHOGONAL POLYNOMIALS Rather than use trigonometric functions, we opt for orthogonal polynomials. where and
14. LEARN CONTINUOUS STRUCTURE ORTHOGONAL POLYNOMIALS Rather than use trigonometric functions, we opt for orthogonal polynomials. where and The function sits inside the integral. 1 The basis terms are orthogonal. 2 Computing the basis terms requires integration. 3
15. LEARN CONTINUOUS STRUCTURE ORTHOGONAL POLYNOMIALS Rather than use trigonometric functions, we opt for orthogonal polynomials. where and Now we introduce truncation! This leads to an approximation error
16. LEARN CONTINUOUS STRUCTURE To approximate a function over a basis of orthogonal polynomials, we need to integrate to estimate its coefficients. Moreover, we need to ascertain the level of truncation required. ORTHOGONAL POLYNOMIALS
17. LEARN CONTINUOUS STRUCTURE To approximate a function over a basis of orthogonal polynomials, we need to integrate to estimate its coefficients. Moreover, we need to ascertain the level of truncation required. But, before we move forward, we need to understand these polynomials more intimately. ORTHOGONAL POLYNOMIALS
18. LEARN CONTINUOUS STRUCTURE OVERVIEW 1. FOURIER SERIES 2. ORTHOGONAL POLYNOMIALS 3. PARAMETERS 4. PARAMETERS & ORTHOGONAL POLYNOMIALS
19. LEARN CONTINUOUS STRUCTURE PARAMETER A parameter is a point within a domain , i.e., .
20. LEARN CONTINUOUS STRUCTURE PARAMETER A parameter is a point within a domain , i.e., . This domain can be: closed semi-infinite infinite We will assume that this domain is a line .
21. LEARN CONTINUOUS STRUCTURE PARAMETER We will also assume that this domain is equipped with a positive weight function Also assume that it integrates to unity over the domain . .
22. LEARN CONTINUOUS STRUCTURE PARAMETER We will also assume that this domain is equipped with a positive weight function Also assume that it integrates to unity over the domain . . probability density function
23. LEARN CONTINUOUS STRUCTURE PARAMETER Putting the pieces together with code, we have a parameter which is a point from the domain and its associated interval, and equipped with a probability distribution . from equadratures import * x = Parameter(distribution=‘uniform’, lower=-1, upper=1, order=3) x = Parameter(distribution=‘gaussian’, shape_parameter_A=0, shape_parameter_B=3, \ order=3) Use parameters to define your input data points, uncertainties or design variables.
24. LEARN CONTINUOUS STRUCTURE OVERVIEW 1. FOURIER SERIES 2. ORTHOGONAL POLYNOMIALS 3. PARAMETERS 4. PARAMETERS & ORTHOGONAL POLYNOMIALS
25. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS There exists a set of orthogonal polynomials that are orthogonal with respect to In other words… where
26. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS There exists a set of orthogonal polynomials that are orthogonal with respect to In other words… where For example, if Legendre polynomials
27. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS There exists a set of orthogonal polynomials that are orthogonal with respect to In other words… where For example, if Jacobi polynomials
28. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS There exists a set of orthogonal polynomials that are orthogonal with respect to In other words… where For example, if Hermite polynomials
29. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Pairing each parameter with its corresponding orthogonal polynomial is important! This guarantees that for the small truncation in you can guarantee exponential convergence. This can be easily seen when we compute Formula for the mean!
30. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Pairing each parameter with its corresponding orthogonal polynomial is important! This guarantees that for the small truncation in you can guarantee exponential convergence. This can be easily seen when we compute Formula for the variance!
31. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Model evaluations Error in estimating the mean 10-16 10-10 10-4 10-1 10 20 30 40 50 0 10 20 30 40 50 10-7 10-13 0 Model evaluations Monte Carlo Using orthogonal polynomials
32. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS So, assuming we can find the coefficients of the polynomial approximation Computing statistical moments of is very easy! But
33. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS So, assuming we can find the coefficients of the polynomial approximation Computing statistical moments of is very easy! But How do we determine the order of the polynomial? How do we evaluate the integrals for the coefficients?
34. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Recall our command from earlier… Set by . User should know something about how non-linear the function is. from equadratures import * x = Parameter(distribution=‘uniform’, lower=-1, upper=1, order=3)
35. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS While it is easy to truncate over a line for approximating
36. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Not so easy to truncate over a plane
37. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Additionally, in the univariate case for approximating Where the coefficients were given by
38. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Additionally, in the univariate case for approximating Where the coefficients were given by which we can estimate via a quadrature rule Quadrature rule points weights
39. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Additionally, in the univariate case for approximating Where the coefficients were given by which we can estimate via a quadrature rule Quadrature rule points weights
40. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Now for the bivariate case, we have Where the coefficients are given by
41. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Now for the bivariate case, we have Where the coefficients are given by
42. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Now for the bivariate case, we have Where the coefficients are given by *by construction this assumes the distributions are independent!
43. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Now for the bivariate case, we have Where the coefficients are given by which we can estimate via a bivariate quadrature rule Weights
44. LEARN CONTINUOUS STRUCTURE PARAMETERS & ORTHGONAL POLYNOMIALS Now for the bivariate case, we have Where the coefficients are given by which we can estimate via a bivariate quadrature rule Weights
45. LEARN CONTINUOUS STRUCTURE OVERVIEW 1. FOURIER SERIES 2. ORTHOGONAL POLYNOMIALS 3. PARAMETERS 4. PARAMETERS & ORTHOGONAL POLYNOMIALS
46. LEARN CONTINUOUS STRUCTURE CODE
47. LEARN CONTINUOUS STRUCTURE Let’s crystallise a few of these ideas through code. CODE from equadratures import * import numpy as np x = Parameter(distribution=‘uniform’, lower=-1, upper=1, order=3) I = Basis(‘tensor-grid’) p = Poly([x,x], I, method=‘numerical-integration’) pts, wts = p.get_points_and_weights() def fun(x): return np.exp(x[0] + x[1]) p.set_model(fun)
48. LEARN CONTINUOUS STRUCTURE CODE But we aren’t restricted to tensorial quadrature rules… There are other quadrature rules — each with their degree of exactness! But to understand those, it is worthwhile to re-phrase the problem.
49. LEARN CONTINUOUS STRUCTURE CODE For notational clarity, let’s stick to the 1D case for now Equivalently written as
50. LEARN CONTINUOUS STRUCTURE CODE With a slight abuse in notation….we can write this as Which may be solved via or or some combination thereof method=‘least-squares’ method=‘compressed-sensing’ method=‘elastic-net’ method=‘relevance-vector-machine’
51. LEARN CONTINUOUS STRUCTURE CODE Classically, polynomial interpolation / regression requires at least as many model evaluations as unknown coefficients. 1D 2D For instance for a quadratic polynomial, there are three unknowns in 1D. 9 unknowns in 2D
52. LEARN CONTINUOUS STRUCTURE CODE Classically, polynomial interpolation / regression requires at least as many model evaluations as unknown coefficients. 1D 2D For instance for a quadratic polynomial, there are three unknowns in 1D. 9 unknowns in 2D 3D 27 unknowns in 3D
53. LEARN CONTINUOUS STRUCTURE CODE Classically, polynomial interpolation / regression requires at least as many model evaluations as unknown coefficients. 1D 2D For instance for a quadratic polynomial, there are three unknowns in 1D. 9 unknowns in 2D 3D 27 unknowns in 3D Rising cost!
54. LEARN CONTINUOUS STRUCTURE The goal is to essentially solve linear systems where reducing costly model evaluations. For more details, check out the next tutorial! CODE
55. LEARN CONTINUOUS STRUCTURE THANK YOU made by the equadratures team, with ❤