Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
890
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
15
LADL-Code Mesh V
eramirem
0
180
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
210
The Linear Algebra of Deep Learning
eramirem
2
680
Linear Algebra for FE Developers
eramirem
1
600
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
400
Numerical Analysis for Orbit Propagation
eramirem
0
170
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.4k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
380
Other Decks in Science
See All in Science
同じデータでもP値が変わる話/key_considerations_in_NHST
florets1
1
1.1k
B-Cubed: Leveraging analysis-ready biodiversity datasets and cloud computing for timely and actionable biodiversity monitoring
peterdesmet
0
150
Onsager代数とその周辺 / Onsager algebra tsudoi
usamik26
0
360
データで課題を解決する -因果関係を調べる統計的因果推論-
sshimizu2006
4
1.3k
Pokemon Roughs
shoryuuken
0
350
早わかり W3C Community Group
takanorip
0
250
勉強会資料 / “Asymptotic Statistics” Section 2.1
asymptotic_minato
0
210
Machine Learning for Materials (Lecture 1)
aronwalsh
1
1.4k
LCG20
lcolladotor
0
170
Услуги лаборатории ТиМПИ
dscs
0
610
OptimizationNight~機械学習と数理最適化の融合~
hidenari
0
260
量子コンピュータとデータサイエンティスト
fuyu_quant0
0
110
Featured
See All Featured
The Cult of Friendly URLs
andyhume
73
5.6k
Designing on Purpose - Digital PM Summit 2013
jponch
109
6.4k
The Invisible Side of Design
smashingmag
293
49k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
1
3.3k
Optimising Largest Contentful Paint
csswizardry
7
2.3k
Testing 201, or: Great Expectations
jmmastey
27
6.3k
The Brand Is Dead. Long Live the Brand.
mthomps
48
21k
Navigating Team Friction
lara
177
13k
YesSQL, Process and Tooling at Scale
rocio
160
13k
Writing Fast Ruby
sferik
619
59k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
272
12k
GitHub's CSS Performance
jonrohan
1023
450k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture