Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Big Data Wrangling: Getting Down to Business

Big Data Wrangling: Getting Down to Business

Keynote talk for a "business" audience at Big Data London 2016

Joe Hellerstein

November 04, 2016
Tweet

More Decks by Joe Hellerstein

Other Decks in Technology

Transcript

  1. Big Data
    Getting Down to Business
    Joe Hellerstein, Co-Founder and Chief Strategy Officer
    Wrangling

    View full-size slide

  2. Data to the People!
    Celebrate Diversity
    Tech for the People
    1
    2
    3
    Outline

    View full-size slide

  3. Marketing
    Finance
    Product
    IT & Cyber
    Web &
    Social
    Customer
    Data
    Sales
    Support
    4
    See you at the Lake!

    View full-size slide

  4. A Common Platform to Store & Analyze Diverse Data
    Data
    Lake
    Finance
    Supply
    Chain
    IT & Cyber
    Web &
    Social
    Customer
    Satisfaction
    Marketing
    Support
    Sales

    View full-size slide

  5. Your business users feel…
    Data
    Lake
    Finance
    Supply
    Chain
    IT & Cyber
    Web &
    Social
    Customer
    Satisfaction
    Marketing
    Support
    Sales

    View full-size slide

  6. Data
    Scientists
    Data
    Engineers
    SQL Analysts
    Excel Data
    Analysts
    Business
    Users
    Data-Driven Population

    View full-size slide

  7. Data Accessibility
    Data
    Scientists
    Data
    Engineers
    SQL Analysts
    Excel Data
    Analysts
    Business
    Users
    Hundreds of
    Gigabytes
    Petabytes &
    Terabytes
    Megabytes &
    Gigabytes

    View full-size slide

  8. 9
    Your business users feel…

    View full-size slide

  9. Data
    Scientists
    Data
    Engineers
    SQL Analysts
    Business
    Users
    Put the data in the hands of the
    people who know it best.
    Unlock potential by empowering people.

    View full-size slide

  10. Celebrate Diversity
    1
    2
    3

    View full-size slide

  11. “There is no point in bringing data … into the data warehouse environment
    without integrating it.”
    — Bill Inmon, Building the Data Warehouse
    The Single Source of Truth

    View full-size slide

  12. “There is no point in bringing data … into the data warehouse environment
    without integrating it.”
    — Bill Inmon, Building the Data Warehouse
    Building

    View full-size slide

  13. “There is no point in bringing data … into the data warehouse environment
    without integrating it.”
    — Bill Inmon, Building the Data Warehouse
    Building

    View full-size slide

  14. “There is no point in bringing data … into the data warehouse environment
    without integrating it.”
    — Bill Inmon, Building the Data Warehouse

    View full-size slide

  15. Data
    Lake
    Data
    Lake
    Finance
    Supply
    Chain
    IT & Cyber
    Web &
    Social
    Marketing
    Support
    The Value of Diversity
    Customer
    Satisfaction
    Sales

    View full-size slide

  16. The Value of Diversity
    Data
    Lake
    Supply
    Chain
    Marketing
    Customer
    Satisfaction
    Inventory Management
    per supermarket chain!
    Customer Behavior 

    Inventory correlations with social
    media campaigns.
    Brand Management
    Buying patterns by
    demographics, politics.

    View full-size slide

  17. Data
    Lake
    Finance
    Supply
    Chain
    IT & Cyber
    Web &
    Social
    Marketing
    Support
    Data
    Lake
    Customer
    Satisfaction
    Sales
    Move beyond 

    the single source of truth.
    Foster agility to generate diverse value.

    View full-size slide

  18. 1
    2
    3
    Tech for the People

    View full-size slide

  19. Driving Value
    To drive more
    value here
    You have to make this more efficient
    20
    Optimizing &
    Publishing
    Enriching &
    Blending
    Cleaning
    80% of the time spent 20%
    Structuring
    Discovery
    Ingestion

    View full-size slide

  20. Why are Business Users Disconnected from Big Data?

    View full-size slide

  21. It’s an Interaction Problem

    View full-size slide

  22. Turn of the Century UX for Data Transformation
    - Schemas and annotations
    - Box-and-arrow programming
    - Batch execution

    View full-size slide

  23. Research Roots: Potter’s Wheel 2001
    + Real data, sampled on the fly
    + Menu-driven transforms
    + Immediate execution and
    feedback
    [Raman & Hellerstein, VLDB01]

    View full-size slide

  24. Research Roots: Open Source Data Wrangler, 2011
    + Predictive Transformation
    + Immediate feedback on

    multiple choices
    – Browser-sized data sets
    [Kandel, Heer & Hellerstein, CHI 11]

    View full-size slide

  25. 26
    Tech Challenge:
    Make Working with Raw Data at Scale…
    Provide immediate
    feedback for how
    transformations impact
    the data on-the-fly
    Interactive
    Suggestions and cues 

    guide based on data,
    activity and history. You
    decide based on visual
    feedback
    Intelligent
    See what’s in your data—
    use visualization to drive
    transformation
    Visual
    Directly manipulate data
    in collaboration with
    algorithms.
    Delightful

    View full-size slide

  26. All of This Led to

    View full-size slide

  27. Interactive Exploration
    28

    View full-size slide

  28. 29
    Predictive Transformation

    View full-size slide

  29. Data to the People!
    Celebrate Diversity
    Tech for the People
    1
    2
    3

    View full-size slide

  30. Context Matters
    4

    View full-size slide

  31. ?
    ?
    THE KNIGHT
    IT
    THE WARRIOR
    DATA SCIENCE
    PROBABILITY
    PREDICTION
    THE QUEEN
    MANAGEMENT
    !
    LOCH
    DATA
    Data Relativism and Context Services
    http://bit.ly/datarelativism16

    View full-size slide

  32. Open Source Context Services
    http://www.ground-context.org
    ground
    A broader context for big data
    THE WARRIOR
    DATA SCIENCE
    THE WARRIOR
    DATA SCIENCE
    Wrangle Aggressively
    Govern Collectively
    THE WARRIOR
    DATA SCIENCE
    THE WARRIOR
    DATA SCIENCE
    THE KNIGHT
    IT
    THE QUEEN
    MANAGEMENT

    View full-size slide

  33. Stop by Trifacta’s booth #306
    Download Trifacta Wrangler for Free
    trifacta.com/start-wrangling

    View full-size slide