Linear Algebra at
Large Scale
Elizabeth Ramirez
@eramirem
Slide 2
Slide 2 text
Computational Engineer
We model complex systems on
the planet, like forestry and
agriculture using satellite
imagery.
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
Top 10 Algorithms of the 20th
Century
Slide 5
Slide 5 text
Often the most expensive
computations in large-scale
codes.
Curse of Dimensionality
Slide 6
Slide 6 text
Linear Systems
Nonlinear Systems
Machine Learning
Deep Learning
Slide 7
Slide 7 text
Most ubiquitous problem in
Scientific Computing and
Data Analysis
Slide 8
Slide 8 text
What solves?
Systems of Equations
Polynomial Interpolation
Linear Least-Squares
Slide 9
Slide 9 text
What we know?
Gaussian Elimination
Complexity
Slide 10
Slide 10 text
HPC Alternative: Iterative
Methods
General Form
Slide 11
Slide 11 text
Jacobi
Gauss-Seidel
Slide 12
Slide 12 text
Convergence of Basic
Iterative Methods
Spectral radius
Slide 13
Slide 13 text
Krylov Subspaces
Slide 14
Slide 14 text
Conjugate Gradient
Method (CG)
i)
ii)
Slide 15
Slide 15 text
Conjugate Gradient (CG)
Slide 16
Slide 16 text
Bi-conjugate gradient (BiCG)
Any linear system
Slide 17
Slide 17 text
Deep Learning Primitives
Weights, inputs, outputs stored in
tensors
Matrix Multiplication
Convolution
Inner Product
Transposition
Rectified Linear Unit (ReLu)
Slide 18
Slide 18 text
Matrix Multiplication
Fundamental task
Naive:
Strassen:
Single Instruction Multiple
Data (SIMD)
Data-level parallelism
Incompatible with code designed
for sequential processors
Instruction set available in
commercial CPUs and GPGPUs
Slide 22
Slide 22 text
Intel® Math Kernel Library
(Intel® MKL)
Improved Matrix Multiplication Performance
in LAPACK
LU decomposition and inverse without
pivoting
Take advantage of SIMD instruction set
In summary: High Performance Linear Algebra