Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
910
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
28
LADL-Code Mesh V
eramirem
0
200
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
260
The Linear Algebra of Deep Learning
eramirem
2
710
Linear Algebra for FE Developers
eramirem
1
620
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
440
Numerical Analysis for Orbit Propagation
eramirem
0
240
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
390
Other Decks in Science
See All in Science
データベース14: B+木 & ハッシュ索引
trycycle
PRO
0
450
科学で迫る勝敗の法則(電気学会・SICE若手セミナー講演 2024年12月) / The principle of victory discovered by science (Lecture for young academists in IEEJ-SICE))
konakalab
0
130
コンピュータビジョンによるロボットの視覚と判断:宇宙空間での適応と課題
hf149
1
320
モンテカルロDCF法による事業価値の算出(モンテカルロ法とベイズモデリング) / Business Valuation Using Monte Carlo DCF Method (Monte Carlo Simulation and Bayesian Modeling)
ikuma_w
0
250
データベース12: 正規化(2/2) - データ従属性に基づく正規化
trycycle
PRO
0
970
研究って何だっけ / What is Research?
ks91
PRO
1
120
データベース01: データベースを使わない世界
trycycle
PRO
1
780
データベース04: SQL (1/3) 単純質問 & 集約演算
trycycle
PRO
0
990
Symfony Console Facelift
chalasr
2
470
データベース15: ビッグデータ時代のデータベース
trycycle
PRO
0
350
ウェブ・ソーシャルメディア論文読み会 第25回: Differences in misinformation sharing can lead to politically asymmetric sanctions (Nature, 2024)
hkefka385
0
130
ランサムウェア対策にも考慮したVMware、Hyper-V、Azure、AWS間のリアルタイムレプリケーション「Zerto」を徹底解説
climbteam
0
110
Featured
See All Featured
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.9k
How to train your dragon (web standard)
notwaldorf
96
6.2k
YesSQL, Process and Tooling at Scale
rocio
173
14k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
The Cost Of JavaScript in 2023
addyosmani
53
8.9k
GraphQLの誤解/rethinking-graphql
sonatard
72
11k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Music & Morning Musume
bryan
46
6.8k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
How STYLIGHT went responsive
nonsquared
100
5.8k
Building Flexible Design Systems
yeseniaperezcruz
329
39k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture