Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
120
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
72
Other Decks in Technology
See All in Technology
Lambdaと地方とコミュニティ
miu_crescent
2
370
Can We Measure Developer Productivity?
ewolff
1
150
TypeScriptの次なる大進化なるか!? 条件型を返り値とする関数の型推論
uhyo
2
1.6k
Amplify Gen2 Deep Dive / バックエンドの型をいかにしてフロントエンドへ伝えるか #TSKaigi #TSKaigiKansai #AWSAmplifyJP
tacck
PRO
0
370
複雑なState管理からの脱却
sansantech
PRO
1
140
BLADE: An Attempt to Automate Penetration Testing Using Autonomous AI Agents
bbrbbq
0
290
信頼性に挑む中で拡張できる・得られる1人のスキルセットとは?
ken5scal
2
530
Why does continuous profiling matter to developers? #appdevelopercon
salaboy
0
180
Shopifyアプリ開発における Shopifyの機能活用
sonatard
4
250
Security-JAWS【第35回】勉強会クラウドにおけるマルウェアやコンテンツ改ざんへの対策
4su_para
0
170
初心者向けAWS Securityの勉強会mini Security-JAWSを9ヶ月ぐらい実施してきての近況
cmusudakeisuke
0
120
元旅行会社の情シス部員が教えるおすすめなre:Inventへの行き方 / What is the most efficient way to re:Invent
naospon
2
340
Featured
See All Featured
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
250
21k
Happy Clients
brianwarren
98
6.7k
Fantastic passwords and where to find them - at NoRuKo
philnash
50
2.9k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
38
1.8k
Building a Scalable Design System with Sketch
lauravandoore
459
33k
Building a Modern Day E-commerce SEO Strategy
aleyda
38
6.9k
Optimising Largest Contentful Paint
csswizardry
33
2.9k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
25
1.8k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
16
2.1k
Building Flexible Design Systems
yeseniaperezcruz
327
38k
Adopting Sorbet at Scale
ufuk
73
9.1k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash