Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
900
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
19
LADL-Code Mesh V
eramirem
0
180
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
240
The Linear Algebra of Deep Learning
eramirem
2
700
Linear Algebra for FE Developers
eramirem
1
610
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
410
Numerical Analysis for Orbit Propagation
eramirem
0
210
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
380
Other Decks in Science
See All in Science
プロダクト開発を通して学んだナレッジマネジメントの哲学
sonod
0
150
Direct Preference Optimization
zchenry
0
280
インフラだけではない MLOps の話 @事例でわかるMLOps 機械学習の成果をスケールさせる処方箋 発売記念
icoxfog417
2
580
HAS Dark Site Orientation
astronomyhouston
0
5.3k
科学で迫る勝敗の法則(名城大学公開講座.2024年10月) / The principle of victory discovered by science (Open lecture in Meijo Univ. 2024)
konakalab
0
200
JSol'Ex : traitement d'images solaires en Java
melix
0
110
ベイズのはなし
techmathproject
0
290
大規模言語モデルの開発
chokkan
PRO
84
33k
Machine Learning for Materials (Lecture 7)
aronwalsh
0
810
AI科学の何が“哲学”の問題になるのか ~問いマッピングの試み~
rmaruy
1
2.2k
トラブルがあったコンペに学ぶデータ分析
tereka114
2
890
Sarcoptic Mange
uni_of_nomi
1
110
Featured
See All Featured
A designer walks into a library…
pauljervisheath
203
24k
Visualization
eitanlees
145
15k
How STYLIGHT went responsive
nonsquared
95
5.2k
Fireside Chat
paigeccino
34
3k
How to Think Like a Performance Engineer
csswizardry
20
1.1k
A Philosophy of Restraint
colly
203
16k
A better future with KSS
kneath
238
17k
Scaling GitHub
holman
458
140k
Side Projects
sachag
452
42k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
8
860
Intergalactic Javascript Robots from Outer Space
tanoku
269
27k
Product Roadmaps are Hard
iamctodd
PRO
49
11k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture