Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
910
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
28
LADL-Code Mesh V
eramirem
0
200
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
260
The Linear Algebra of Deep Learning
eramirem
2
710
Linear Algebra for FE Developers
eramirem
1
620
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
450
Numerical Analysis for Orbit Propagation
eramirem
0
240
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
390
Other Decks in Science
See All in Science
07_浮世満理子_アイディア高等学院学院長_一般社団法人全国心理業連合会代表理事_紹介資料.pdf
sip3ristex
0
620
論文紹介 音源分離:SCNET SPARSE COMPRESSION NETWORK FOR MUSIC SOURCE SEPARATION
kenmatsu4
0
330
ド文系だった私が、 KaggleのNCAAコンペでソロ金取れるまで
wakamatsu_takumu
2
1.4k
LayerXにおける業務の完全自動運転化に向けたAI技術活用事例 / layerx-ai-jsai2025
shimacos
2
1.6k
データベース11: 正規化(1/2) - 望ましくない関係スキーマ
trycycle
PRO
0
950
04_石井クンツ昌子_お茶の水女子大学理事_副学長_D_I社会実現へ向けて.pdf
sip3ristex
0
640
地質研究者が苦労しながら運用する情報公開システムの実例
naito2000
0
280
02_西村訓弘_プログラムディレクター_人口減少を機にひらく未来社会.pdf
sip3ristex
0
630
AIに仕事を奪われる 最初の医師たちへ
ikora128
0
980
mathematics of indirect reciprocity
yohm
1
190
academist Prize 4期生 研究トーク延長戦!「美は世界を救う」っていうけど、どうやって?
jimpe_hitsuwari
0
170
モンテカルロDCF法による事業価値の算出(モンテカルロ法とベイズモデリング) / Business Valuation Using Monte Carlo DCF Method (Monte Carlo Simulation and Bayesian Modeling)
ikuma_w
0
280
Featured
See All Featured
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
850
Practical Orchestrator
shlominoach
190
11k
Designing for humans not robots
tammielis
254
25k
4 Signs Your Business is Dying
shpigford
185
22k
The World Runs on Bad Software
bkeepers
PRO
71
11k
Making Projects Easy
brettharned
119
6.4k
We Have a Design System, Now What?
morganepeng
53
7.8k
Code Review Best Practice
trishagee
72
19k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
960
The Power of CSS Pseudo Elements
geoffreycrofte
79
6k
The Language of Interfaces
destraynor
162
25k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture