Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
150
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
80
Other Decks in Technology
See All in Technology
AIは変更差分からユニットテスト_結合テスト_システムテストでテストすべきことが出せるのか?
mineo_matsuya
5
2.5k
ABEMAにおける 生成AI活用の現在地 / The Current Status of Generative AI at ABEMA
dekatotoro
0
240
薬屋のひとりごとにみるトラブルシューティング
tomokusaba
0
390
AIが住民向けコンシェルジュに?Amazon Connectと生成AIで実現する自治体AIエージェント!
yuyeah
0
210
文字列の並び順 / String Collation
tmtms
1
110
信頼できる開発プラットフォームをどう作るか?-Governance as Codeと継続的監視/フィードバックが導くPlatform Engineeringの進め方
yuriemori
1
190
Amazon GuardDuty での脅威検出:脅威検出の実例から学ぶ
kintotechdev
0
130
AWSの最新サービスでAIエージェント構築に楽しく入門しよう
minorun365
PRO
8
470
オブザーバビリティ文化を組織に浸透させるには / install observability culture
mackerelio
0
320
AIエージェントを現場で使う / 2025.08.07 著者陣に聞く!現場で活用するためのAIエージェント実践入門(Findyランチセッション)
smiyawaki0820
7
1.3k
EKS Pod Identity における推移的な session tags
z63d
1
140
Intro to Software Startups: Spring 2025
arnabdotorg
0
280
Featured
See All Featured
Scaling GitHub
holman
462
140k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Imperfection Machines: The Place of Print at Facebook
scottboms
268
13k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
131
19k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
KATA
mclloyd
32
14k
Intergalactic Javascript Robots from Outer Space
tanoku
272
27k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
460
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Thoughts on Productivity
jonyablonski
69
4.8k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash