Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
120
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
74
Other Decks in Technology
See All in Technology
Amazon S3 Tablesと外部分析基盤連携について / Amazon S3 Tables and External Data Analytics Platform
nttcom
0
130
Tech Blogを書きやすい環境づくり
lycorptech_jp
PRO
1
240
管理者しか知らないOutlookの裏側のAIを覗く#AzureTravelers
hirotomotaguchi
2
350
Nekko Cloud、 これまでとこれから ~学生サークルが作る、 小さなクラウド
logica0419
2
960
MC906491 を見据えた Microsoft Entra Connect アップグレード対応
tamaiyutaro
1
540
Platform Engineeringは自由のめまい
nwiizo
4
2.1k
技術負債の「予兆検知」と「状況異変」のススメ / Technology Dept
i35_267
1
1.1k
プロダクトエンジニア構想を立ち上げ、プロダクト志向な組織への成長を続けている話 / grow into a product-oriented organization
hiro_torii
0
110
Developers Summit 2025 浅野卓也(13-B-7 LegalOn Technologies)
legalontechnologies
PRO
0
670
データ資産をシームレスに伝達するためのイベント駆動型アーキテクチャ
kakehashi
PRO
2
520
7日間でハッキングをはじめる本をはじめてみませんか?_ITエンジニア本大賞2025
nomizone
2
1.8k
滅・サービスクラス🔥 / Destruction Service Class
sinsoku
6
1.6k
Featured
See All Featured
YesSQL, Process and Tooling at Scale
rocio
172
14k
A better future with KSS
kneath
238
17k
GraphQLの誤解/rethinking-graphql
sonatard
68
10k
The World Runs on Bad Software
bkeepers
PRO
67
11k
Thoughts on Productivity
jonyablonski
69
4.5k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Bootstrapping a Software Product
garrettdimon
PRO
306
110k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Raft: Consensus for Rubyists
vanstee
137
6.8k
The Cult of Friendly URLs
andyhume
78
6.2k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Practical Orchestrator
shlominoach
186
10k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash