$30 off During Our Annual Pro Sale. View Details »

Building Data Driven Organizations

Abe Stanway
September 13, 2014

Building Data Driven Organizations

Given at IT Weekend 2014 in Kiev

Abe Stanway

September 13, 2014
Tweet

More Decks by Abe Stanway

Other Decks in Programming

Transcript

  1. @AbeStanway
    BUILDING A DATA
    DRIVEN ORGANIZATION

    View Slide

  2. 1. why
    2. how

    View Slide

  3. 1. why
    2. how

    View Slide

  4. “DATA IS THE NEW GOLD”

    View Slide

  5. Predict the future!

    View Slide

  6. Retain Customers!

    View Slide

  7. Grow the business!

    View Slide

  8. Recommend content!

    View Slide

  9. Drive Engagement!

    View Slide

  10. unclear paths to $$$

    View Slide

  11. IN IT, It’s clear.

    View Slide

  12. Data are Dollars

    View Slide

  13. .
    IT Working = +$$$
    IT Not Working = -$$$
    .
    .

    View Slide

  14. How do you know if
    your IT is working
    right now?

    View Slide

  15. How do you know if you
    are earning money
    right now?

    View Slide

  16. KPIs. What are they?

    View Slide

  17. Etsy:
    Literally a Money per
    second Graph

    View Slide

  18. Planet Labs:
    Literally an Images
    per day graph

    View Slide

  19. What are the
    Kpis for kips?

    View Slide

  20. $ per
    second
    items
    bought per
    second page
    requests
    per second
    database
    queries per
    second memcache
    hits per
    second
    fread()
    per second

    View Slide

  21. If you do not have the data
    about your infrastructure,
    it is already broken.
    LEsson:

    View Slide

  22. View Slide

  23. Test driven development
    ->
    data driven development

    View Slide

  24. Without data, you
    are flying blind

    View Slide

  25. How do you know you’re
    hitting your goals?

    View Slide

  26. How do you know if You’re
    making the right ones in the
    first place?

    View Slide

  27. How do you know if
    you’re still in
    business?

    View Slide

  28. How do you even
    know what planet
    you live on?

    View Slide

  29. Assumptions are death

    View Slide

  30. You need data, yo.

    View Slide

  31. 1. why
    2. how

    View Slide

  32. 1. collect
    2. analyze
    3. ???
    4. Profit!

    View Slide

  33. 1. collect
    2. analyze
    3. ACT
    4. Profit!

    View Slide

  34. data that cannot be
    acted upon should
    not be analyzed.

    View Slide

  35. View Slide

  36. You are running a
    business, not an
    art museum

    View Slide

  37. You are Trying to
    Win the market, not
    a fields medal

    View Slide

  38. This can be
    disappointing

    View Slide

  39. Data SCientist?

    View Slide

  40. Data Scientist?
    Realist.

    View Slide

  41. Find a way to Align your
    employees intellectual curiosity
    With your Real business needs.
    LEssoN:

    View Slide

  42. Train your organization

    View Slide

  43. you need a data culture.

    View Slide

  44. “It’s not shipped
    until it’s monitored”

    View Slide

  45. “If you are not looking
    at Dashboards, you are
    not doing your job”

    View Slide

  46. Building instrumentation
    and watching dashboards
    are hard And Time consuming

    View Slide

  47. App code -> statsD -> Graphite -> Dashboards -> Insights
    by hand by hand by hand by hand by hand

    View Slide

  48. Developers just
    want to code

    View Slide

  49. Let’s
    automate

    View Slide

  50. Which is easier
    to automate?
    Insights or data
    collection?

    View Slide

  51. Insights are
    sexy and fun

    View Slide

  52. Collection is
    hard And unsexy

    View Slide

  53. Collection is
    hard And Boring

    View Slide

  54. Collection is
    hard And unsexy

    View Slide

  55. We’re on track to
    have excellent
    automated insights

    View Slide

  56. anomaly
    detection

    View Slide

  57. App code -> statsD -> Graphite -> Dashboards -> Insights
    by hand by hand by hand AUTOMATIC!
    AUTOMATIC!

    View Slide

  58. (…if only we had the data)

    View Slide

  59. How do we
    automate data
    collection?

    View Slide

  60. currently have
    ganglia, New relic,
    collectD, etc

    View Slide

  61. NOT WHAT
    WE NEED

    View Slide

  62. they provide data about
    your raw machines, not your
    CUSTOM DEVELOPED TECHNOLOGY
    And Application level logic

    View Slide

  63. Healthy servers
    don’t make you money.
    Healthy services do.

    View Slide

  64. enter
    LARIMAR

    View Slide

  65. Full disclosure:
    this is my new PROJECT
    !
    we’re going to talk about it
    because i’m pretty excited and the
    beta is opening up soon.

    View Slide

  66. LARIMAR uses raw
    machine metrics to infer
    App level architecture
    and inform developers
    about problems

    View Slide

  67. A service:
    cpu resources
    disk io
    PCAP data
    ports Used
    syscalls

    View Slide

  68. A service:
    cpu resources
    disk io
    PCAP data
    ports Used
    syscalls
    service fingerprint
    MACHINE LEARNING

    View Slide

  69. A service:
    cpu resources
    disk io
    PCAP data
    ports Used
    syscalls
    ABNORMAL BEHAVIOR
    MACHINE LEARNING

    View Slide

  70. a system:
    service
    service
    service
    service
    service
    MACHINE LEARNING
    graphical system
    fingerprint

    View Slide

  71. a system:
    service
    service
    service
    service
    service
    MACHINE LEARNING
    Abnormal, holistic
    system behavior

    View Slide

  72. Larimar automates
    both analysis And
    Relevant data
    collection

    View Slide

  73. so your developers
    can focus on coding
    and acting on insights

    View Slide

  74. No configuration!

    View Slide

  75. App code -> statsD -> Graphite -> Dashboards -> Insights
    by hand AUTOMATIC!
    AUTOMATIC!
    AUTOMATIC! AUTOMATIC!

    View Slide

  76. 1. collect
    2. analyze
    3. ACT
    4. Profit!

    View Slide

  77. organizational
    shifts are still
    needed to inspire
    ACTION on Data

    View Slide

  78. but ACTION is easier to
    inspire when there is
    lots of data and lots
    of insight everywhere

    View Slide

  79. Create a culture where
    your developers create
    these kinds of tools

    View Slide

  80. When a data driven
    mindset is the default,
    tools will build
    themselves.

    View Slide

  81. Thanks!
    @abestanway
    !
    !
    larimar.io
    @larimarhq

    View Slide