Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
910
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
28
LADL-Code Mesh V
eramirem
0
200
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
260
The Linear Algebra of Deep Learning
eramirem
2
710
Linear Algebra for FE Developers
eramirem
1
620
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
440
Numerical Analysis for Orbit Propagation
eramirem
0
240
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
390
Other Decks in Science
See All in Science
KH Coderチュートリアル(スライド版)
koichih
1
46k
03_草原和博_広島大学大学院人間社会科学研究科教授_デジタル_シティズンシップシティで_新たな_学び__をつくる.pdf
sip3ristex
0
600
機械学習 - 決定木からはじめる機械学習
trycycle
PRO
0
1k
科学で迫る勝敗の法則(電気学会・SICE若手セミナー講演 2024年12月) / The principle of victory discovered by science (Lecture for young academists in IEEJ-SICE))
konakalab
0
130
データベース15: ビッグデータ時代のデータベース
trycycle
PRO
0
350
CV_5_3dVision
hachama
0
150
生成AIと学ぶPythonデータ分析再入門-Pythonによるクラスタリング・可視化をサクサク実施-
datascientistsociety
PRO
4
1.8k
Cross-Media Technologies, Information Science and Human-Information Interaction
signer
PRO
3
31k
[Paper Introduction] From Bytes to Ideas:Language Modeling with Autoregressive U-Nets
haruumiomoto
0
130
How To Buy, Verified Venmo Accounts in 2025 This year
usaallshop68
4
280
データベース05: SQL(2/3) 結合質問
trycycle
PRO
0
800
機械学習 - K近傍法 & 機械学習のお作法
trycycle
PRO
0
1.2k
Featured
See All Featured
Six Lessons from altMBA
skipperchong
28
4k
Typedesign – Prime Four
hannesfritz
42
2.8k
Speed Design
sergeychernyshev
32
1.1k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.9k
Docker and Python
trallard
46
3.6k
Automating Front-end Workflow
addyosmani
1370
200k
Navigating Team Friction
lara
189
15k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
850
StorybookのUI Testing Handbookを読んだ
zakiyama
31
6.1k
Thoughts on Productivity
jonyablonski
70
4.8k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1.1k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture