Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
120
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
75
Other Decks in Technology
See All in Technology
Pwned Labsのすゝめ
ken5scal
0
190
プロダクトエンジニア 360°フィードバックを実施した話
hacomono
PRO
0
130
ディスプレイ広告(Yahoo!広告・LINE広告)におけるバックエンド開発
lycorptech_jp
PRO
0
200
データマネジメントのトレードオフに立ち向かう
ikkimiyazaki
6
1.2k
【内製開発Summit 2025】イオンスマートテクノロジーの内製化組織の作り方/In-house-development-summit-AST
aeonpeople
1
470
白金鉱業Meetup Vol.17_あるデータサイエンティストのデータマネジメントとの向き合い方
brainpadpr
7
990
NFV基盤のOpenStack更新 ~9世代バージョンアップへの挑戦~
vtj
0
320
2/18/25: Java meets AI: Build LLM-Powered Apps with LangChain4j
edeandrea
PRO
0
160
表現を育てる
kiyou77
1
230
Amazon Aurora のバージョンアップ手法について
smt7174
1
110
Helm , Kustomize に代わる !? 次世代 k8s パッケージマネージャー Glasskube 入門 / glasskube-entry
parupappa2929
0
290
スキルだけでは満たせない、 “組織全体に”なじむオンボーディング/Onboarding that fits “throughout the organization” and cannot be satisfied by skills alone
bitkey
0
130
Featured
See All Featured
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
30
4.6k
Thoughts on Productivity
jonyablonski
69
4.5k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Practical Orchestrator
shlominoach
186
10k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Typedesign – Prime Four
hannesfritz
40
2.5k
Optimizing for Happiness
mojombo
376
70k
Six Lessons from altMBA
skipperchong
27
3.6k
The Cost Of JavaScript in 2023
addyosmani
47
7.3k
Designing on Purpose - Digital PM Summit 2013
jponch
117
7.1k
Git: the NoSQL Database
bkeepers
PRO
427
65k
KATA
mclloyd
29
14k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash