Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
900
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
20
LADL-Code Mesh V
eramirem
0
190
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
250
The Linear Algebra of Deep Learning
eramirem
2
700
Linear Algebra for FE Developers
eramirem
1
610
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
410
Numerical Analysis for Orbit Propagation
eramirem
0
230
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
380
Other Decks in Science
See All in Science
解説!データ基盤の進化を後押しする手順とタイミング
shomaekawa
1
400
構造設計のための3D生成AI-最新の取り組みと今後の展開-
kojinishiguchi
0
890
学術講演会中央大学学員会大分支部
tagtag
0
120
ICRA2024 速報
rpc
3
6k
生成AI による論文執筆サポートの手引き(ワークショップ) / A guide to supporting dissertation writing with generative AI (workshop)
ks91
PRO
0
400
20分で分かる Human-in-the-Loop 機械学習におけるアノテーションとヒューマンコンピューターインタラクションの真髄
hurutoriya
5
2.8k
学術講演会中央大学学員会八王子支部
tagtag
0
280
Online Feedback Optimization
floriandoerfler
0
930
2024-06-16-pydata_london
sofievl
0
610
Introduction to Image Processing: 2.Frequ
hachama
0
480
As We May Interact: Challenges and Opportunities for Next-Generation Human-Information Interaction
signer
PRO
0
380
【健康&筋肉と生産性向上の関連性】 【Google Cloudを企業で運用する際の知識】 をお届け
yasumuusan
0
470
Featured
See All Featured
The Pragmatic Product Professional
lauravandoore
32
6.4k
Designing for humans not robots
tammielis
250
25k
YesSQL, Process and Tooling at Scale
rocio
172
14k
How GitHub (no longer) Works
holman
314
140k
Being A Developer After 40
akosma
89
590k
KATA
mclloyd
29
14k
Embracing the Ebb and Flow
colly
84
4.6k
Become a Pro
speakerdeck
PRO
26
5.2k
Making the Leap to Tech Lead
cromwellryan
133
9.1k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
330
21k
Faster Mobile Websites
deanohume
306
31k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture