Building Data Driven Organizations

Given at IT Weekend 2014 in Kiev

Transcript

1. @AbeStanway
   BUILDING A DATA
   DRIVEN ORGANIZATION
   

   View Slide

2. 1. why
   2. how
   

   View Slide

3. 1. why
   2. how
   

   View Slide

4. “DATA IS THE NEW GOLD”
   

   View Slide

5. Predict the future!
   

   View Slide

6. Retain Customers!
   

   View Slide

7. Grow the business!
   

   View Slide

8. Recommend content!
   

   View Slide

9. Drive Engagement!
   

   View Slide

10. unclear paths to $$$
    

    View Slide

11. IN IT, It’s clear.
    

    View Slide

12. Data are Dollars
    

    View Slide

13. .
    IT Working = +$$$
    IT Not Working = -$$$
    .
    .
    

    View Slide

14. How do you know if
    your IT is working
    right now?
    

    View Slide

15. How do you know if you
    are earning money
    right now?
    

    View Slide

16. KPIs. What are they?
    

    View Slide

17. Etsy:
    Literally a Money per
    second Graph
    

    View Slide

18. Planet Labs:
    Literally an Images
    per day graph
    

    View Slide

19. What are the
    Kpis for kips?
    

    View Slide

20. $ per
    second
    items
    bought per
    second page
    requests
    per second
    database
    queries per
    second memcache
    hits per
    second
    fread()
    per second
    

    View Slide

21. If you do not have the data
    about your infrastructure,
    it is already broken.
    LEsson:
    

    View Slide

22. View Slide

23. Test driven development
    ->
    data driven development
    

    View Slide

24. Without data, you
    are flying blind
    

    View Slide

25. How do you know you’re
    hitting your goals?
    

    View Slide

26. How do you know if You’re
    making the right ones in the
    first place?
    

    View Slide

27. How do you know if
    you’re still in
    business?
    

    View Slide

28. How do you even
    know what planet
    you live on?
    

    View Slide

29. Assumptions are death
    

    View Slide

30. You need data, yo.
    

    View Slide

31. 1. why
    2. how
    

    View Slide

32. 1. collect
    2. analyze
    3. ???
    4. Profit!
    

    View Slide

33. 1. collect
    2. analyze
    3. ACT
    4. Profit!
    

    View Slide

34. data that cannot be
    acted upon should
    not be analyzed.
    

    View Slide

35. View Slide

36. You are running a
    business, not an
    art museum
    

    View Slide

37. You are Trying to
    Win the market, not
    a fields medal
    

    View Slide

38. This can be
    disappointing
    

    View Slide

39. Data SCientist?
    

    View Slide

40. Data Scientist?
    Realist.
    

    View Slide

41. Find a way to Align your
    employees intellectual curiosity
    With your Real business needs.
    LEssoN:
    

    View Slide

42. Train your organization
    

    View Slide

43. you need a data culture.
    

    View Slide

44. “It’s not shipped
    until it’s monitored”
    

    View Slide

45. “If you are not looking
    at Dashboards, you are
    not doing your job”
    

    View Slide

46. Building instrumentation
    and watching dashboards
    are hard And Time consuming
    

    View Slide

47. App code -> statsD -> Graphite -> Dashboards -> Insights
    by hand by hand by hand by hand by hand
    

    View Slide

48. Developers just
    want to code
    

    View Slide

49. Let’s
    automate
    

    View Slide

50. Which is easier
    to automate?
    Insights or data
    collection?
    

    View Slide

51. Insights are
    sexy and fun
    

    View Slide

52. Collection is
    hard And unsexy
    

    View Slide

53. Collection is
    hard And Boring
    

    View Slide

54. Collection is
    hard And unsexy
    

    View Slide

55. We’re on track to
    have excellent
    automated insights
    

    View Slide

56. anomaly
    detection
    

    View Slide

57. App code -> statsD -> Graphite -> Dashboards -> Insights
    by hand by hand by hand AUTOMATIC!
    AUTOMATIC!
    

    View Slide

58. (…if only we had the data)
    

    View Slide

59. How do we
    automate data
    collection?
    

    View Slide

60. currently have
    ganglia, New relic,
    collectD, etc
    

    View Slide

61. NOT WHAT
    WE NEED
    

    View Slide

62. they provide data about
    your raw machines, not your
    CUSTOM DEVELOPED TECHNOLOGY
    And Application level logic
    

    View Slide

63. Healthy servers
    don’t make you money.
    Healthy services do.
    

    View Slide

64. enter
    LARIMAR
    

    View Slide

65. Full disclosure:
    this is my new PROJECT
    !
    we’re going to talk about it
    because i’m pretty excited and the
    beta is opening up soon.
    

    View Slide

66. LARIMAR uses raw
    machine metrics to infer
    App level architecture
    and inform developers
    about problems
    

    View Slide

67. A service:
    cpu resources
    disk io
    PCAP data
    ports Used
    syscalls
    

    View Slide

68. A service:
    cpu resources
    disk io
    PCAP data
    ports Used
    syscalls
    service fingerprint
    MACHINE LEARNING
    

    View Slide

69. A service:
    cpu resources
    disk io
    PCAP data
    ports Used
    syscalls
    ABNORMAL BEHAVIOR
    MACHINE LEARNING
    

    View Slide

70. a system:
    service
    service
    service
    service
    service
    MACHINE LEARNING
    graphical system
    fingerprint
    

    View Slide

71. a system:
    service
    service
    service
    service
    service
    MACHINE LEARNING
    Abnormal, holistic
    system behavior
    

    View Slide

72. Larimar automates
    both analysis And
    Relevant data
    collection
    

    View Slide

73. so your developers
    can focus on coding
    and acting on insights
    

    View Slide

74. No configuration!
    

    View Slide

75. App code -> statsD -> Graphite -> Dashboards -> Insights
    by hand AUTOMATIC!
    AUTOMATIC!
    AUTOMATIC! AUTOMATIC!
    

    View Slide

76. 1. collect
    2. analyze
    3. ACT
    4. Profit!
    

    View Slide

77. organizational
    shifts are still
    needed to inspire
    ACTION on Data
    

    View Slide

78. but ACTION is easier to
    inspire when there is
    lots of data and lots
    of insight everywhere
    

    View Slide

79. Create a culture where
    your developers create
    these kinds of tools
    

    View Slide

80. When a data driven
    mindset is the default,
    tools will build
    themselves.
    

    View Slide

81. Thanks!
    @abestanway
    !
    !
    larimar.io
    @larimarhq
    

    View Slide