Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Distributed TensorFlow: Scaling Deep Learning L...
Search
mactiendinh
December 28, 2017
Technology
0
84
Distributed TensorFlow: Scaling Deep Learning Library
#tensorflow #scale #distributed
mactiendinh
December 28, 2017
Tweet
Share
More Decks by mactiendinh
See All by mactiendinh
Chapter 5: Good design = Flexible softwave
mactiendinh
0
14
Overview chapter 4 Head First Object Oriented Design and Analysis
mactiendinh
0
41
Overview chapter 3 Head First Object Oriented Design and Analysis
mactiendinh
0
64
Other Decks in Technology
See All in Technology
Oracle Cloud Infrastructure:2025年6月度サービス・アップデート
oracle4engineer
PRO
2
260
AWS テクニカルサポートとエンドカスタマーの中間地点から見えるより良いサポートの活用方法
kazzpapa3
2
550
GeminiとNotebookLMによる金融実務の業務革新
abenben
0
230
Delegating the chores of authenticating users to Keycloak
ahus1
0
130
【PHPカンファレンス 2025】PHPを愛するひとに伝えたい PHPとキャリアの話
tenshoku_draft
0
120
Clineを含めたAIエージェントを 大規模組織に導入し、投資対効果を考える / Introducing AI agents into your organization
i35_267
4
1.6k
Amazon ECS & AWS Fargate 運用アーキテクチャ2025 / Amazon ECS and AWS Fargate Ops Architecture 2025
iselegant
17
5.7k
AIエージェント最前線! Amazon Bedrock、Amazon Q、そしてMCPを使いこなそう
minorun365
PRO
15
5.3k
CI/CD/IaC 久々に0から環境を作ったらこうなりました
kaz29
1
180
セキュリティの民主化は何故必要なのか_AWS WAF 運用の 10 の苦悩から学ぶ
yoh
1
180
生成AI時代の開発組織・技術・プロセス 〜 ログラスの挑戦と考察 〜
itohiro73
1
270
~宇宙最速~2025年AWS Summit レポート
satodesu
1
1.9k
Featured
See All Featured
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
46
9.6k
KATA
mclloyd
29
14k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.5k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Documentation Writing (for coders)
carmenintech
72
4.9k
Bash Introduction
62gerente
614
210k
Being A Developer After 40
akosma
90
590k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.5k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
Why You Should Never Use an ORM
jnunemaker
PRO
57
9.4k
GraphQLとの向き合い方2022年版
quramy
48
14k
Transcript
Distributed TensorFlow Tien Dinh
None
None
None
None
TensorFlow: Expressing High-Level ML Computations Core in C++ • Very
• low overhead Different • front ends for specifying/driving the computation Python • and C++ today, easy to add more
Computation is a dataflow graph Graph of Nodes • ,
called Operations or ops Edges are N • -dimensional arrays: Tensors
Computation is a dataflow graph WITH STATE
Computation is a dataflow graph Distributed
Computation is a dataflow graph Assign Devices to Ops •
TensorFlow inserts Send/Recv Ops to transport tensors across devices • Recv ops pull data from Send ops
Computation is a dataflow graph Assign Devices to Ops TensorFlow
inserts Send/Recv Ops to transport tensors across devices • Recv • ops pull data from Send ops
Distrubuted Training with TensorFlow
Distrubuted Training with TensorFlow
Model Parallelism = split model, share data
Distrubuted Training
Distrubuted Training with TensorFlow
Data Parallelism
Data Parallelism
Data Parallelism
Data Parallelism
Data Parallelism
Data Parallelism
Distributed training mechanisms Graph structure and low-level graph primitives (queues)
allow us to play with synchronous vs. asynchronous update algorithms.
Thanks for your attention!