Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Seeing at the Speed of Thought: Empowering Othe...
Search
Greg Goltsov
March 08, 2017
Programming
0
260
Seeing at the Speed of Thought: Empowering Others Through Data Exploration
Talk I gave at Big Data Visualisation Sydney 2017
Greg Goltsov
March 08, 2017
Tweet
Share
More Decks by Greg Goltsov
See All by Greg Goltsov
Beginning ClojureScript: How not to learn a new language
ggoltsov
1
110
Full-stack Data Science: How to be a One-Man Data Team
ggoltsov
2
600
Kuranku - final game presentation
ggoltsov
0
280
Scalable agent-based simulations
ggoltsov
1
240
Procedural City Generator - Honours Presentation
ggoltsov
2
1.6k
Extracting the Meaning: Painless processing and analysis of image data with Fiji and Ruby
ggoltsov
0
160
Ninja Code
ggoltsov
1
2.4k
Other Decks in Programming
See All in Programming
プロダクト開発をAI 1stに変革する〜SaaS is dead時代で生き残るために〜 / AI 1st Product Development
kobakei
0
440
monorepo の Go テストをはやくした〜い!~最小の依存解決への道のり~ / faster-testing-of-monorepos
convto
2
130
気づいて!アプリからのSOS 〜App Store Connect APIで始めるパフォーマンス健康診断〜
waka12
0
260
WebエンジニアがSwiftをブラウザで動かすプレイグラウンドを作ってみた
ohmori_yusuke
0
170
Swiftビルド弾丸ツアー - Swift Buildが作る新しいエコシステム
giginet
PRO
0
1.6k
非同期jobをtransaction内で 呼ぶなよ!絶対に呼ぶなよ!
alstrocrack
0
430
Reduxモダナイズ 〜コードのモダン化を通して、将来のライブラリ移行に備える〜
pvcresin
2
670
Swift Concurrency - 状態監視の罠
objectiveaudio
2
430
Let's Write a Train Tracking Algorithm
twocentstudios
0
220
AIエージェント時代における TypeScriptスキーマ駆動開発の新たな役割
bicstone
4
1.4k
iOSアプリの信頼性を向上させる取り組み/ios-app-improve-reliability
shino8rayu9
0
140
議事録の要点整理を自動化! サーバレス Bot 構築術
penpeen
3
1.6k
Featured
See All Featured
Become a Pro
speakerdeck
PRO
29
5.5k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
Writing Fast Ruby
sferik
629
62k
Raft: Consensus for Rubyists
vanstee
139
7.1k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.7k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
Code Review Best Practice
trishagee
72
19k
GraphQLとの向き合い方2022年版
quramy
49
14k
Building Applications with DynamoDB
mza
96
6.6k
Reflections from 52 weeks, 52 projects
jeffersonlam
352
21k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Transcript
Seeing at the speed of thought Empowering others through data
exploration Greg Goltsov Senior Data Engineer @gregoltsov www.gregory.goltsov.info (will have link to slides)
Seeing at the speed of thought Empowering others through data
exploration
Seeing at the speed of thought Empowering others through data
exploration
Seeing at the speed of thought Empowering others through data
exploration yourself
Seeing at the speed of thought Empowering others through data
exploration yourself your team
Seeing at the speed of thought Empowering others through data
exploration yourself your team your company
Touch Surgery Built marketing/sales dashboards for Fortune 10 companies Built
educational dashboards for 4 of the top 10 world-rated medical universities All from scratch
Appear Here World’s biggest online marketplace for retail spaces Internal
recommendation system Highly visual debug interface for non-tech people
Southern Cross Austereo Modernising the data pipeline Spearheading data-driven culture
throughout the company Datasets covering 80% Australians weekly
BI/DW tools
BI/DW tools
Remove barriers Make feedback fast Remove yourself
Remove barriers
Remove barriers Catalogued datasets with one-line import in Python Messy
dataset in PDFs
Remove barriers Dashboard with right filters, Excel export “Can you
run a query?”
Remove barriers. Foster curiosity.
Make feedback fast
Make feedback fast Found a new trend via tinkering “Tomorrow
I’ll see results of the batch job”
Make feedback fast “Check the dash in 15 mins” “I
put your request into the backlog”
Make feedback fast. Let people tinker.
Remove yourself
Remove yourself Data pipeline + products Ad-hoc
None
None
Remove yourself. Don’t stand in the way.
Remove barriers Make feedback fast Remove yourself
The goal is to turn data into information, and information
into insight. – Carly Fiorina, former HP CEO
Insight Information Data
Insight Information Data Value ↑ Abundance
Insight Information Data Fraud Access pattern Logs
Insight Information Data Key influencers MOM trends Tweets
Ad-hoc queries Data pipeline Fast to develop Every query gets
thrown away after Upfront investment Every integration builds foundations
Visualise your ETL. Augment your Data Warehouses with Data Lakes.
None
Extract Transform Load Sources Data Warehouse
Extract Transform Load Sources Data Warehouse Data Insight Time
Volume Variety Velocity "3D Data Management: Controlling Data Volume, Velocity
and Variety”, Gartner Inc. 2001
Volume Variety Velocity "3D Data Management: Controlling Data Volume, Velocity
and Variety”, Gartner Inc. 2001
Analysis of Unstructured Data: Applications of Text Analytics and Sentiment
Mining ~80% of all data is unstructured
~80% of your data is unstructured
http://www.ft.com/cms/s/0/de15414e-ebad-11e1-985a-00144feab49a.html#axzz2F3CM6G7g “Making sense of unstructured data isn’t about technology, it’s
a business challenge”
Aberdeen Group research Don’t use unstructured data Use unstructured data
Happy with the ability to share data 18% 60% Pleased with the accessibility 20% 50%
Volume Variety Velocity Machine learning "3D Data Management: Controlling Data
Volume, Velocity and Variety”, Gartner Inc. 2001
Ingest quickly Real-time schema-on- read exploration Push vetted insights into
DW/BI Example: Spark, AWS Athena, Microsoft’s PowerBI
Collect Store Process/ Analyse Sources Data Warehouse Data Insight Insight
Time
Collect Store Process/ Analyse
Collect Store Process/ Analyse
Collect Store Process/ Analyse
None
Look at data. A lot.
Look at data. A lot. http:/ /www.forbes.com/sites/gilpress/2016/03/23/data- preparation-most-time-consuming-least-enjoyable- data-science-task-survey-says
None
None
Scale computation and storage separately Go from non-trivial data to
dashboard in minutes Spark is 20-100x faster than MapReduce Turnkey solution: www.databricks.com OSS: Apache Zeppelin on AWS EMR Spark
We made it! Now what?
We made it! Now what? Human scale.
AirBnB Scaling Tribal Knowledge
AirBnB Scaling Tribal Knowledge
AirBnB Scaling Tribal Knowledge
AirBnB Scaling Tribal Knowledge
AirBnB Scaling Tribal Knowledge
None
THANK YOU Speaker Name: Greg Goltsov Email:
[email protected]
Organized by
UNICOM Trainings & Seminars Pvt. Ltd.
[email protected]
http://www.unicomlearning.com/2017/Big_Data_Visualization_Summit_Sydney