July 04, 2023
2.5k

# Learn Continuous Structure

July 04, 2023

## Transcript

1. LEARN CONTINUOUS STRUCTURE
Associate Professor in Aerospace Engineering
Georgia Institute of Technology

2. LEARN CONTINUOUS STRUCTURE
INTRODUCTION
equadratures is an open-source python code that is purpose built for developing
models using orthogonal polynomials.
These models can be fit directly to real-world sensor data, simulation data, or even
to other surrogate models.
As you will discover, using orthogonal polynomials is very advantageous and can
tell us a lot about the underlying data we are modelling.
A FEW WORDS

3. LEARN CONTINUOUS STRUCTURE
OVERVIEW
1. FOURIER SERIES
2. ORTHOGONAL POLYNOMIALS
3. PARAMETERS
4. PARAMETERS & ORTHOGONAL POLYNOMIALS

4. LEARN CONTINUOUS STRUCTURE
Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a
Fourier series
JB Joseph Fourier
FOURIER SERIES

5. LEARN CONTINUOUS STRUCTURE
Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a
Fourier series
JB Joseph Fourier
FOURIER SERIES
where
and

6. LEARN CONTINUOUS STRUCTURE
Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a
Fourier series
JB Joseph Fourier
FOURIER SERIES
where
and
3 very important observations!

7. LEARN CONTINUOUS STRUCTURE
Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a
Fourier series
JB Joseph Fourier
FOURIER SERIES
where
and
The function sits inside the integral.
1

8. LEARN CONTINUOUS STRUCTURE
Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a
Fourier series
JB Joseph Fourier
FOURIER SERIES
where
and
The basis terms are orthogonal.
2

9. LEARN CONTINUOUS STRUCTURE
Before introducing equadratures, and its building blocks, it will be worthwhile to re-visit a
Fourier series
JB Joseph Fourier
FOURIER SERIES
where
and
Computing the basis terms requires integration.
3

10. LEARN CONTINUOUS STRUCTURE
FOURIER SERIES
The function sits inside the integral.
1
The basis terms are orthogonal.
2
Computing the basis terms requires integration.
3
But, what is orthogonality?

11. LEARN CONTINUOUS STRUCTURE
FOURIER SERIES
We define orthogonality as
where

12. LEARN CONTINUOUS STRUCTURE
OVERVIEW
1. FOURIER SERIES
2. ORTHOGONAL POLYNOMIALS
3. PARAMETERS
4. PARAMETERS & ORTHOGONAL POLYNOMIALS

13. LEARN CONTINUOUS STRUCTURE
ORTHOGONAL POLYNOMIALS
Rather than use trigonometric functions, we opt for orthogonal polynomials.
where
and

14. LEARN CONTINUOUS STRUCTURE
ORTHOGONAL POLYNOMIALS
Rather than use trigonometric functions, we opt for orthogonal polynomials.
where
and
The function sits inside the integral.
1
The basis terms are orthogonal.
2
Computing the basis terms requires integration.
3

15. LEARN CONTINUOUS STRUCTURE
ORTHOGONAL POLYNOMIALS
Rather than use trigonometric functions, we opt for orthogonal polynomials.
where
and
Now we introduce
truncation!
approximation error

16. LEARN CONTINUOUS STRUCTURE
To approximate a function over a basis of orthogonal polynomials, we need to
integrate to estimate its coefficients. Moreover, we need to ascertain the level of
truncation required.
ORTHOGONAL POLYNOMIALS

17. LEARN CONTINUOUS STRUCTURE
To approximate a function over a basis of orthogonal polynomials, we need to
integrate to estimate its coefficients. Moreover, we need to ascertain the level of
truncation required.
But, before we move forward, we need to understand these polynomials more
intimately.
ORTHOGONAL POLYNOMIALS

18. LEARN CONTINUOUS STRUCTURE
OVERVIEW
1. FOURIER SERIES
2. ORTHOGONAL POLYNOMIALS
3. PARAMETERS
4. PARAMETERS & ORTHOGONAL POLYNOMIALS

19. LEARN CONTINUOUS STRUCTURE
PARAMETER
A parameter is a point within a domain , i.e., .

20. LEARN CONTINUOUS STRUCTURE
PARAMETER
A parameter is a point within a domain , i.e., .
This domain can be:
closed
semi-infinite
infinite
We will assume that this domain is a line .

21. LEARN CONTINUOUS STRUCTURE
PARAMETER
We will also assume that this domain is equipped with a positive weight function
Also assume that it integrates to unity over the domain
.
.

22. LEARN CONTINUOUS STRUCTURE
PARAMETER
We will also assume that this domain is equipped with a positive weight function
Also assume that it integrates to unity over the domain
.
.
probability density function

23. LEARN CONTINUOUS STRUCTURE
PARAMETER
Putting the pieces together with code, we have a parameter which is a point from
the domain and its associated interval, and
equipped with a probability distribution .
x = Parameter(distribution=‘uniform’, lower=-1, upper=1, order=3)
x = Parameter(distribution=‘gaussian’, shape_parameter_A=0, shape_parameter_B=3, \
order=3)
Use parameters to define your input data points, uncertainties or design variables.

24. LEARN CONTINUOUS STRUCTURE
OVERVIEW
1. FOURIER SERIES
2. ORTHOGONAL POLYNOMIALS
3. PARAMETERS
4. PARAMETERS & ORTHOGONAL POLYNOMIALS

25. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
There exists a set of orthogonal polynomials that are orthogonal with respect to
In other words…
where

26. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
There exists a set of orthogonal polynomials that are orthogonal with respect to
In other words…
where
For example, if
Legendre
polynomials

27. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
There exists a set of orthogonal polynomials that are orthogonal with respect to
In other words…
where
For example, if
Jacobi
polynomials

28. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
There exists a set of orthogonal polynomials that are orthogonal with respect to
In other words…
where
For example, if
Hermite
polynomials

29. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Pairing each parameter with its corresponding orthogonal polynomial is important!
This guarantees that for the small truncation in
you can guarantee exponential convergence. This can be easily seen when we compute
Formula for the mean!

30. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Pairing each parameter with its corresponding orthogonal polynomial is important!
This guarantees that for the small truncation in
you can guarantee exponential convergence. This can be easily seen when we compute
Formula for the
variance!

31. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Model evaluations
Error in estimating the mean
10-16
10-10
10-4
10-1
10 20 30 40 50
0 10 20 30 40 50
10-7
10-13
0
Model evaluations
Monte Carlo
Using orthogonal polynomials

32. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
So, assuming we can find the coefficients of the polynomial approximation
Computing statistical moments of is very easy! But

33. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
So, assuming we can find the coefficients of the polynomial approximation
Computing statistical moments of is very easy! But
How do we determine the order of the polynomial?
How do we evaluate the integrals for the coefficients?

34. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Recall our command from earlier…
Set by .
User should know something about how non-linear the function is.
x = Parameter(distribution=‘uniform’, lower=-1, upper=1, order=3)

35. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
While it is easy to truncate over a line for approximating

36. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Not so easy to truncate over a plane

37. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Additionally, in the univariate case for approximating
Where the coefficients were given by

38. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Additionally, in the univariate case for approximating
Where the coefficients were given by
which we can estimate via a quadrature rule
points
weights

39. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Additionally, in the univariate case for approximating
Where the coefficients were given by
which we can estimate via a quadrature rule
points
weights

40. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Now for the bivariate case, we have
Where the coefficients are given by

41. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Now for the bivariate case, we have
Where the coefficients are given by

42. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Now for the bivariate case, we have
Where the coefficients are given by
*by construction this assumes the
distributions are independent!

43. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Now for the bivariate case, we have
Where the coefficients are given by
which we can estimate via a bivariate quadrature rule
Weights

44. LEARN CONTINUOUS STRUCTURE
PARAMETERS & ORTHGONAL POLYNOMIALS
Now for the bivariate case, we have
Where the coefficients are given by
which we can estimate via a bivariate quadrature rule
Weights

45. LEARN CONTINUOUS STRUCTURE
OVERVIEW
1. FOURIER SERIES
2. ORTHOGONAL POLYNOMIALS
3. PARAMETERS
4. PARAMETERS & ORTHOGONAL POLYNOMIALS

46. LEARN CONTINUOUS STRUCTURE
CODE

47. LEARN CONTINUOUS STRUCTURE
Let’s crystallise a few of these ideas through code.
CODE
import numpy as np
x = Parameter(distribution=‘uniform’, lower=-1, upper=1, order=3)
I = Basis(‘tensor-grid’)
p = Poly([x,x], I, method=‘numerical-integration’)
pts, wts = p.get_points_and_weights()
def fun(x):
return np.exp(x[0] + x[1])
p.set_model(fun)

48. LEARN CONTINUOUS STRUCTURE
CODE
But we aren’t restricted to tensorial quadrature rules…
There are other quadrature rules — each with their degree of exactness!
But to understand those, it is worthwhile to re-phrase the problem.

49. LEARN CONTINUOUS STRUCTURE
CODE
For notational clarity, let’s stick to the 1D case for now
Equivalently written as

50. LEARN CONTINUOUS STRUCTURE
CODE
With a slight abuse in notation….we can write this as
Which may be solved via
or
or some combination thereof
method=‘least-squares’ method=‘compressed-sensing’
method=‘elastic-net’ method=‘relevance-vector-machine’

51. LEARN CONTINUOUS STRUCTURE
CODE
Classically, polynomial interpolation / regression requires at least as many model
evaluations as unknown coefficients.
1D 2D
polynomial, there are three
unknowns in 1D.
9 unknowns in 2D

52. LEARN CONTINUOUS STRUCTURE
CODE
Classically, polynomial interpolation / regression requires at least as many model
evaluations as unknown coefficients.
1D 2D
polynomial, there are three
unknowns in 1D.
9 unknowns in 2D
3D
27 unknowns in 3D

53. LEARN CONTINUOUS STRUCTURE
CODE
Classically, polynomial interpolation / regression requires at least as many model
evaluations as unknown coefficients.
1D 2D
polynomial, there are three
unknowns in 1D.
9 unknowns in 2D
3D
27 unknowns in 3D
Rising cost!

54. LEARN CONTINUOUS STRUCTURE
The goal is to essentially solve linear systems where reducing costly model
evaluations.
For more details, check out the next tutorial!
CODE

55. LEARN CONTINUOUS STRUCTURE
THANK YOU