Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Linear Algebra at Large Scale
Search
Elizabeth Ramirez
April 27, 2018
Science
7
910
Linear Algebra at Large Scale
Elizabeth Ramirez
April 27, 2018
Tweet
Share
More Decks by Elizabeth Ramirez
See All by Elizabeth Ramirez
Maritime Transportation from Space: The most important industry you know nothing about.
eramirem
0
27
LADL-Code Mesh V
eramirem
0
200
Transition Matrix Estimation in High Dimensional Time Series.
eramirem
0
260
The Linear Algebra of Deep Learning
eramirem
2
710
Linear Algebra for FE Developers
eramirem
1
620
Top 10: Los mejores algoritmos del Siglo XX
eramirem
0
430
Numerical Analysis for Orbit Propagation
eramirem
0
240
A New Approach to Linear Filtering and Prediction Problems
eramirem
0
1.5k
Kalman Filters for non-rocket science - PyCon 2016
eramirem
2
380
Other Decks in Science
See All in Science
AI(人工知能)の過去・現在・未来 —AIは人間を超えるのか—
tagtag
1
110
07_浮世満理子_アイディア高等学院学院長_一般社団法人全国心理業連合会代表理事_紹介資料.pdf
sip3ristex
0
580
白金鉱業Meetup Vol.16_数理最適化案件のはじめかた・すすめかた
brainpadpr
4
1.9k
白金鉱業Meetup Vol.16_【初学者向け発表】 数理最適化のはじめの一歩 〜身近な問題で学ぶ最適化の面白さ〜
brainpadpr
11
2.3k
データマイニング - コミュニティ発見
trycycle
PRO
0
140
baseballrによるMLBデータの抽出と階層ベイズモデルによる打率の推定 / TokyoR118
dropout009
1
550
データマイニング - グラフデータと経路
trycycle
PRO
1
210
「美は世界を救う」を心理学で実証したい~クラファンを通じた新しい研究方法
jimpe_hitsuwari
1
140
データベース15: ビッグデータ時代のデータベース
trycycle
PRO
0
330
データベース09: 実体関連モデル上の一貫性制約
trycycle
PRO
0
980
LayerXにおける業務の完全自動運転化に向けたAI技術活用事例 / layerx-ai-jsai2025
shimacos
2
1.4k
サイゼミ用因果推論
lw
1
7.5k
Featured
See All Featured
[RailsConf 2023] Rails as a piece of cake
palkan
56
5.8k
GitHub's CSS Performance
jonrohan
1031
460k
BBQ
matthewcrist
89
9.8k
A Tale of Four Properties
chriscoyier
160
23k
Unsuck your backbone
ammeep
671
58k
How to Ace a Technical Interview
jacobian
279
23k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Building Adaptive Systems
keathley
43
2.7k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
Six Lessons from altMBA
skipperchong
28
4k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
30
9.6k
Transcript
Linear Algebra at Large Scale Elizabeth Ramirez @eramirem
Computational Engineer We model complex systems on the planet, like
forestry and agriculture using satellite imagery.
None
Top 10 Algorithms of the 20th Century
Often the most expensive computations in large-scale codes. Curse of
Dimensionality
Linear Systems Nonlinear Systems Machine Learning Deep Learning
Most ubiquitous problem in Scientific Computing and Data Analysis
What solves? Systems of Equations Polynomial Interpolation Linear Least-Squares
What we know? Gaussian Elimination Complexity
HPC Alternative: Iterative Methods General Form
Jacobi Gauss-Seidel
Convergence of Basic Iterative Methods Spectral radius
Krylov Subspaces
Conjugate Gradient Method (CG) i) ii)
Conjugate Gradient (CG)
Bi-conjugate gradient (BiCG) Any linear system
Deep Learning Primitives Weights, inputs, outputs stored in tensors Matrix
Multiplication Convolution Inner Product Transposition Rectified Linear Unit (ReLu)
Matrix Multiplication Fundamental task Naive: Strassen:
Low-Rank Approximation Accelerates matrix multiplication, therefore, accelerates convolution. Requires SVD:
Low-Rank Multiplication:
Single Instruction Multiple Data (SIMD) Data-level parallelism Incompatible with code
designed for sequential processors Instruction set available in commercial CPUs and GPGPUs
Intel® Math Kernel Library (Intel® MKL) Improved Matrix Multiplication Performance
in LAPACK LU decomposition and inverse without pivoting Take advantage of SIMD instruction set In summary: High Performance Linear Algebra
None
References http://www.siam.org/pdf/news/637.pdf https://software.intel.com/en-us/mkl https://software.intel.com/en-us/articles/t ensorflow-optimizations-on-modern-intel-arc hitecture