Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
910
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
27
LADL-Code Mesh V
eramirem
0
200
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
250
The Linear Algebra of Deep Learning
eramirem
2
710
Linear Algebra for FE Developers
eramirem
1
620
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
430
Numerical Analysis for Orbit Propagation
eramirem
0
240
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
380
Other Decks in Science
See All in Science
Masseyのレーティングを用いたフォーミュラレースドライバーの実績評価手法の開発 / Development of a Performance Evaluation Method for Formula Race Drivers Using Massey Ratings
konakalab
0
160
Trend Classification of InSAR Displacement Time Series Using SAE–CNN
satai
3
460
3次元点群を利用した植物の葉の自動セグメンテーションについて
kentaitakura
2
1.3k
白金鉱業Meetup Vol.16_数理最適化案件のはじめかた・すすめかた
brainpadpr
3
1.8k
機械学習 - SVM
trycycle
PRO
1
850
Introd_Img_Process_2_Frequ
hachama
0
560
Hakonwa-Quaternion
hiranabe
1
110
機械学習 - K-means & 階層的クラスタリング
trycycle
PRO
0
990
データマイニング - ノードの中心性
trycycle
PRO
0
130
統計学入門講座 第4回スライド
techmathproject
0
150
テンソル分解による糖尿病の組織特異的遺伝子発現の統合解析を用いた関連疾患の予測
tagtag
2
200
研究って何だっけ / What is Research?
ks91
PRO
1
100
Featured
See All Featured
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.3k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
740
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
Side Projects
sachag
455
42k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.5k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.9k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
It's Worth the Effort
3n
185
28k
The Cost Of JavaScript in 2023
addyosmani
51
8.5k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture