Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
900
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
18
LADL-Code Mesh V
eramirem
0
180
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
240
The Linear Algebra of Deep Learning
eramirem
2
700
Linear Algebra for FE Developers
eramirem
1
610
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
410
Numerical Analysis for Orbit Propagation
eramirem
0
210
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
380
Other Decks in Science
See All in Science
伊豆赤沢海洋深層水中からメラニン分解能を有する微生物の探索
eltociear
0
120
ベイズ最適化をゼロから
brainpadpr
2
740
Direct Preference Optimization
zchenry
0
270
インフラだけではない MLOps の話 @事例でわかるMLOps 機械学習の成果をスケールさせる処方箋 発売記念
icoxfog417
2
520
The thin line between reconstruction, classification, and hallucination in brain decoding
ykamit
1
880
Iniciativas independentes de divulgação científica: o caso do Movimento #CiteMulheresNegras
taisso
0
140
解説!データ基盤の進化を後押しする手順とタイミング
shomaekawa
1
330
Snowflake上でRを使う: RStudioセットアップとShinyアプリケーションのデプロイ
ktatsuya
0
390
Machine Learning for Materials (Lecture 6)
aronwalsh
0
500
第61回コンピュータビジョン勉強会「BioCLIP: A Vision Foundation Model for the Tree of Life」
x_ttyszk
1
1.5k
理論計算機科学における 数学の応用: 擬似ランダムネス
nobushimi
1
330
大規模画像テキストデータのフィルタリング手法の紹介
lyakaap
6
1.5k
Featured
See All Featured
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
27
1.9k
It's Worth the Effort
3n
183
27k
The World Runs on Bad Software
bkeepers
PRO
65
11k
A Tale of Four Properties
chriscoyier
156
23k
Optimizing for Happiness
mojombo
376
69k
Raft: Consensus for Rubyists
vanstee
136
6.6k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
7
150
How GitHub (no longer) Works
holman
311
140k
Unsuck your backbone
ammeep
668
57k
Building an army of robots
kneath
302
42k
Product Roadmaps are Hard
iamctodd
PRO
48
10k
Happy Clients
brianwarren
97
6.7k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture