Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
130
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
79
Other Decks in Technology
See All in Technology
DevinはクラウドエンジニアAIになれるのか!? 実践的なガードレール設計/devin-can-become-a-cloud-engineer-ai-practical-guardrail-design
tomoki10
3
1.4k
Proxmox VE超入門 〜 無料で作れるご自宅仮想化プラットフォームブックマークする
devops_vtj
0
210
Re:VIEWで書いた「Compose で Android の edge-to-edge に対応する」をRoo Codeで発表資料にしてもらった
tomoya0x00
0
180
製造業の会計システムをDDDで開発した話
caddi_eng
3
1k
コドモンのQAの今までとこれから -XPによる成長と見えてきた課題-
masasuna
0
110
Amazon EKS Auto ModeでKubernetesの運用をシンプルにする
sshota0809
0
120
Tirez profit de Messenger pour améliorer votre architecture
tucksaun
1
160
Redefine_Possible
upsider_tech
0
370
20250328_RubyKaigiで出会い鯛_____RubyKaigiから始まったはじめてのOSSコントリビュート.pdf
mterada1228
0
320
モンテカルロ木探索のパフォーマンスを予測する Kaggleコンペ解説 〜生成AIによる未知のゲーム生成〜
rist
4
1.2k
【5分でわかる】セーフィー エンジニア向け会社紹介
safie_recruit
0
20k
アプリケーション固有の「ロジックの脆弱性」を防ぐ開発者のためのセキュリティ観点
flatt_security
35
14k
Featured
See All Featured
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.1k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
129
19k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
17
1.1k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
51
2.4k
GraphQLの誤解/rethinking-graphql
sonatard
70
10k
Fireside Chat
paigeccino
37
3.3k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.2k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
2.1k
Making the Leap to Tech Lead
cromwellryan
133
9.2k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.2k
Reflections from 52 weeks, 52 projects
jeffersonlam
349
20k
Git: the NoSQL Database
bkeepers
PRO
429
65k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash