Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
910
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
28
LADL-Code Mesh V
eramirem
0
210
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
260
The Linear Algebra of Deep Learning
eramirem
2
710
Linear Algebra for FE Developers
eramirem
1
620
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
450
Numerical Analysis for Orbit Propagation
eramirem
0
250
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
390
Other Decks in Science
See All in Science
蔵本モデルが解き明かす同期と相転移の秘密 〜拍手のリズムはなぜ揃うのか?〜
syotasasaki593876
1
120
サイコロで理解する原子核崩壊と拡散現象 〜単純化されたモデルで本質を理解する〜
syotasasaki593876
0
100
AIによる科学の加速: 各領域での革新と共創の未来
masayamoriofficial
0
200
コンピュータビジョンによるロボットの視覚と判断:宇宙空間での適応と課題
hf149
1
390
生成AIと学ぶPythonデータ分析再入門-Pythonによるクラスタリング・可視化をサクサク実施-
datascientistsociety
PRO
4
1.8k
知能とはなにかーヒトとAIのあいだー
tagtag
0
150
Accelerated Computing for Climate forecast
inureyes
0
120
データベース01: データベースを使わない世界
trycycle
PRO
1
830
Explanatory material
yuki1986
0
410
Symfony Console Facelift
chalasr
2
480
mOrganic™ Holdings, LLC.
hyperlocalnetwork
0
120
Ignite の1年間の軌跡
ktombow
0
160
Featured
See All Featured
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.7k
Making Projects Easy
brettharned
120
6.4k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
9.7k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
359
30k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
9
930
Six Lessons from altMBA
skipperchong
29
4k
jQuery: Nuts, Bolts and Bling
dougneiner
65
7.9k
Faster Mobile Websites
deanohume
310
31k
Done Done
chrislema
185
16k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
230
22k
Documentation Writing (for coders)
carmenintech
75
5.1k
The Straight Up "How To Draw Better" Workshop
denniskardys
238
140k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture