Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
120
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
73
Other Decks in Technology
See All in Technology
C++26 エラー性動作
faithandbrave
2
700
5分でわかるDuckDB
chanyou0311
10
3.2k
re:Invent をおうちで楽しんでみた ~CloudWatch のオブザーバビリティ機能がスゴい!/ Enjoyed AWS re:Invent from Home and CloudWatch Observability Feature is Amazing!
yuj1osm
0
120
第3回Snowflake女子会_LT登壇資料(合成データ)_Taro_CCCMK
tarotaro0129
0
180
NilAway による静的解析で「10 億ドル」を節約する #kyotogo / Kyoto Go 56th
ytaka23
3
370
ガバメントクラウドのセキュリティ対策事例について
fujisawaryohei
0
530
社外コミュニティで学び社内に活かす共に学ぶプロジェクトの実践/backlogworld2024
nishiuma
0
260
生成AIのガバナンスの全体像と現実解
fnifni
1
180
GitHub Copilot のテクニック集/GitHub Copilot Techniques
rayuron
24
11k
2024年にチャレンジしたことを振り返るぞ
mitchan
0
130
KubeCon NA 2024 Recap: How to Move from Ingress to Gateway API with Minimal Hassle
ysakotch
0
200
日本版とグローバル版のモバイルアプリ統合の開発の裏側と今後の展望
miichan
1
130
Featured
See All Featured
RailsConf 2023
tenderlove
29
940
The Language of Interfaces
destraynor
154
24k
Docker and Python
trallard
42
3.1k
The Power of CSS Pseudo Elements
geoffreycrofte
73
5.4k
Git: the NoSQL Database
bkeepers
PRO
427
64k
Why You Should Never Use an ORM
jnunemaker
PRO
54
9.1k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.7k
GraphQLとの向き合い方2022年版
quramy
44
13k
Measuring & Analyzing Core Web Vitals
bluesmoon
4
170
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
Reflections from 52 weeks, 52 projects
jeffersonlam
347
20k
The Invisible Side of Design
smashingmag
298
50k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash