Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
100
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
65
Other Decks in Technology
See All in Technology
Terraformあれやこれ/terraform-this-and-that
emiki
6
580
GraphQL 成熟度モデルの紹介と、プロダクトに当てはめた事例 / GraphQL maturity model
mh4gf
4
130
入社後初めてのタスクでk8sアップグレードした話.pdf
kkato1
1
380
SREとその組織類型
tatsuo48
8
1.5k
Aurora MySQL v3(MySQL8.0互換)の オンラインDDLの罠挙動を全バージョンで検証した
yutakikai
1
150
Garoon 開発チーム / Garoon development team
cybozuinsideout
PRO
2
2.9k
NgRx Signal Store
rainerhahnekamp
0
120
日本におけるデータエンジニアリングのこれまでとこれから
foursue
12
2.5k
Google Cloud の AI を支える裏側のインフラを垣間見る!
maroon1st
0
210
WebアプリケーションにおけるPDOの使い方入門 / phpcon odawara 2024
meihei3
2
430
検証を通して見えてきたTiDBの性能特性
lycorptech_jp
PRO
6
3.4k
[PlatformCon 24] Platform Orchestrators: The Missing Middle of Internal Developer Platforms?
danielbryantuk
1
180
Featured
See All Featured
The Straight Up "How To Draw Better" Workshop
denniskardys
227
130k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
18
1.7k
Building Your Own Lightsaber
phodgson
98
5.7k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
186
16k
Bash Introduction
62gerente
604
210k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
29
6k
The MySQL Ecosystem @ GitHub 2015
samlambert
242
12k
WebSockets: Embracing the real-time Web
robhawkes
59
7k
The Invisible Customer
myddelton
114
12k
Pencils Down: Stop Designing & Start Developing
hursman
117
11k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
104
6.6k
Web development in the modern age
philhawksworth
202
10k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash