Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
910
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
27
LADL-Code Mesh V
eramirem
0
200
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
250
The Linear Algebra of Deep Learning
eramirem
2
710
Linear Algebra for FE Developers
eramirem
1
620
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
430
Numerical Analysis for Orbit Propagation
eramirem
0
240
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
380
Other Decks in Science
See All in Science
機械学習 - K-means & 階層的クラスタリング
trycycle
PRO
0
1k
白金鉱業Meetup Vol.16_数理最適化案件のはじめかた・すすめかた
brainpadpr
4
1.8k
機械学習 - SVM
trycycle
PRO
1
860
Hakonwa-Quaternion
hiranabe
1
110
CV_5_3dVision
hachama
0
140
統計的因果探索: 背景知識とデータにより因果仮説を探索する
sshimizu2006
4
940
データベース04: SQL (1/3) 単純質問 & 集約演算
trycycle
PRO
0
880
研究って何だっけ / What is Research?
ks91
PRO
1
100
生成AIと学ぶPythonデータ分析再入門-Pythonによるクラスタリング・可視化をサクサク実施-
datascientistsociety
PRO
4
1.6k
モンテカルロDCF法による事業価値の算出(モンテカルロ法とベイズモデリング) / Business Valuation Using Monte Carlo DCF Method (Monte Carlo Simulation and Bayesian Modeling)
ikuma_w
0
200
01_篠原弘道_SIPガバニングボード座長_ポスコロSIPへの期待.pdf
sip3ristex
0
560
地質研究者が苦労しながら運用する情報公開システムの実例
naito2000
0
220
Featured
See All Featured
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
990
We Have a Design System, Now What?
morganepeng
53
7.7k
RailsConf 2023
tenderlove
30
1.1k
Done Done
chrislema
184
16k
Music & Morning Musume
bryan
46
6.7k
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.8k
The Invisible Side of Design
smashingmag
301
51k
GraphQLとの向き合い方2022年版
quramy
49
14k
Navigating Team Friction
lara
187
15k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
60k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.1k
Thoughts on Productivity
jonyablonski
69
4.7k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture