Pro Yearly is on sale from $80 to $50! »

Seeing at the Speed of Thought: Empowering Others Through Data Exploration

Seeing at the Speed of Thought: Empowering Others Through Data Exploration

Talk I gave at Big Data Visualisation Sydney 2017

707d0934f77fc31048f004103e10c57f?s=128

Greg Goltsov

March 08, 2017
Tweet

Transcript

  1. Seeing at the speed of thought Empowering others through data

    exploration Greg Goltsov Senior Data Engineer @gregoltsov www.gregory.goltsov.info (will have link to slides)
  2. Seeing at the speed of thought Empowering others through data

    exploration
  3. Seeing at the speed of thought Empowering others through data

    exploration
  4. Seeing at the speed of thought Empowering others through data

    exploration yourself
  5. Seeing at the speed of thought Empowering others through data

    exploration yourself your team
  6. Seeing at the speed of thought Empowering others through data

    exploration yourself your team your company
  7. Touch Surgery Built marketing/sales dashboards for Fortune 10 companies Built

    educational dashboards for 4 of the top 10 world-rated medical universities All from scratch
  8. Appear Here World’s biggest online marketplace for retail spaces Internal

    recommendation system Highly visual debug interface for non-tech people
  9. Southern Cross Austereo Modernising the data pipeline Spearheading data-driven culture

    throughout the company Datasets covering 80% Australians weekly
  10. BI/DW tools

  11. BI/DW tools

  12. Remove barriers Make feedback fast Remove yourself

  13. Remove barriers

  14. Remove barriers Catalogued datasets with one-line import in Python Messy

    dataset in PDFs
  15. Remove barriers Dashboard with right filters, Excel export “Can you

    run a query?”
  16. Remove barriers. Foster curiosity.

  17. Make feedback fast

  18. Make feedback fast Found a new trend via tinkering “Tomorrow

    I’ll see results of the batch job”
  19. Make feedback fast “Check the dash in 15 mins” “I

    put your request into the backlog”
  20. Make feedback fast. Let people tinker.

  21. Remove yourself

  22. Remove yourself Data pipeline + products Ad-hoc

  23. None
  24. None
  25. Remove yourself. Don’t stand in the way.

  26. Remove barriers Make feedback fast Remove yourself

  27. The goal is to turn data into information, and information

    into insight. – Carly Fiorina, former HP CEO
  28. Insight Information Data

  29. Insight Information Data Value ↑ Abundance

  30. Insight Information Data Fraud Access pattern Logs

  31. Insight Information Data Key influencers MOM trends Tweets

  32. Ad-hoc queries Data pipeline Fast to develop Every query gets

    thrown away after Upfront investment Every integration builds foundations
  33. Visualise your ETL. Augment your Data Warehouses with Data Lakes.

  34. None
  35. Extract Transform Load Sources Data Warehouse

  36. Extract Transform Load Sources Data Warehouse Data Insight Time

  37. Volume Variety Velocity "3D Data Management: Controlling Data Volume, Velocity

    and Variety”, Gartner Inc. 2001
  38. Volume Variety Velocity "3D Data Management: Controlling Data Volume, Velocity

    and Variety”, Gartner Inc. 2001
  39. Analysis of Unstructured Data: Applications of Text Analytics and Sentiment

    Mining ~80% of all data is unstructured
  40. ~80% of your data is unstructured

  41. http://www.ft.com/cms/s/0/de15414e-ebad-11e1-985a-00144feab49a.html#axzz2F3CM6G7g “Making sense of unstructured data isn’t about technology, it’s

    a business challenge”
  42. Aberdeen Group research Don’t use unstructured data Use unstructured data

    Happy with the ability to share data 18% 60% Pleased with the accessibility 20% 50%
  43. Volume Variety Velocity Machine learning "3D Data Management: Controlling Data

    Volume, Velocity and Variety”, Gartner Inc. 2001
  44. Ingest quickly Real-time schema-on- read exploration Push vetted insights into

    DW/BI Example: Spark, AWS Athena, Microsoft’s PowerBI
  45. Collect Store Process/ Analyse Sources Data Warehouse Data Insight Insight

    Time
  46. Collect Store Process/ Analyse

  47. Collect Store Process/ Analyse

  48. Collect Store Process/ Analyse

  49. None
  50. Look at data. A lot.

  51. Look at data. A lot. http:/ /www.forbes.com/sites/gilpress/2016/03/23/data- preparation-most-time-consuming-least-enjoyable- data-science-task-survey-says

  52. None
  53. None
  54. Scale computation and storage separately Go from non-trivial data to

    dashboard in minutes Spark is 20-100x faster than MapReduce Turnkey solution: www.databricks.com OSS: Apache Zeppelin on AWS EMR Spark
  55. We made it! Now what?

  56. We made it! Now what? Human scale.

  57. AirBnB Scaling Tribal Knowledge

  58. AirBnB Scaling Tribal Knowledge

  59. AirBnB Scaling Tribal Knowledge

  60. AirBnB Scaling Tribal Knowledge

  61. AirBnB Scaling Tribal Knowledge

  62. None
  63. THANK YOU Speaker Name: Greg Goltsov Email: gregory@goltsov.info Organized by

    UNICOM Trainings & Seminars Pvt. Ltd.
 contact@unicomlearning.com http://www.unicomlearning.com/2017/Big_Data_Visualization_Summit_Sydney