Building Data Driven Organizations

Slide 1

Slide 1 text

@AbeStanway BUILDING A DATA DRIVEN ORGANIZATION

Slide 2

Slide 2 text

1. why 2. how

Slide 3

Slide 3 text

1. why 2. how

Slide 4

Slide 4 text

“DATA IS THE NEW GOLD”

Slide 5

Slide 5 text

Predict the future!

Slide 6

Slide 6 text

Retain Customers!

Slide 7

Slide 7 text

Grow the business!

Slide 8

Slide 8 text

Recommend content!

Slide 9

Slide 9 text

Drive Engagement!

Slide 10

Slide 10 text

unclear paths to $$$

Slide 11

Slide 11 text

IN IT, It’s clear.

Slide 12

Slide 12 text

Data are Dollars

Slide 13

Slide 13 text

. IT Working = +$$$ IT Not Working = -$$$ . .

Slide 14

Slide 14 text

How do you know if your IT is working right now?

Slide 15

Slide 15 text

How do you know if you are earning money right now?

Slide 16

Slide 16 text

KPIs. What are they?

Slide 17

Slide 17 text

Etsy: Literally a Money per second Graph

Slide 18

Slide 18 text

Planet Labs: Literally an Images per day graph

Slide 19

Slide 19 text

What are the Kpis for kips?

Slide 20

Slide 20 text

$ per second items bought per second page requests per second database queries per second memcache hits per second fread() per second

Slide 21

Slide 21 text

If you do not have the data about your infrastructure, it is already broken. LEsson:

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Test driven development -> data driven development

Slide 24

Slide 24 text

Without data, you are flying blind

Slide 25

Slide 25 text

How do you know you’re hitting your goals?

Slide 26

Slide 26 text

How do you know if You’re making the right ones in the first place?

Slide 27

Slide 27 text

How do you know if you’re still in business?

Slide 28

Slide 28 text

How do you even know what planet you live on?

Slide 29

Slide 29 text

Assumptions are death

Slide 30

Slide 30 text

You need data, yo.

Slide 31

Slide 31 text

1. why 2. how

Slide 32

Slide 32 text

1. collect 2. analyze 3. ??? 4. Profit!

Slide 33

Slide 33 text

1. collect 2. analyze 3. ACT 4. Profit!

Slide 34

Slide 34 text

data that cannot be acted upon should not be analyzed.

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

You are running a business, not an art museum

Slide 37

Slide 37 text

You are Trying to Win the market, not a fields medal

Slide 38

Slide 38 text

This can be disappointing

Slide 39

Slide 39 text

Data SCientist?

Slide 40

Slide 40 text

Data Scientist? Realist.

Slide 41

Slide 41 text

Find a way to Align your employees intellectual curiosity With your Real business needs. LEssoN:

Slide 42

Slide 42 text

Train your organization

Slide 43

Slide 43 text

you need a data culture.

Slide 44

Slide 44 text

“It’s not shipped until it’s monitored”

Slide 45

Slide 45 text

“If you are not looking at Dashboards, you are not doing your job”

Slide 46

Slide 46 text

Building instrumentation and watching dashboards are hard And Time consuming

Slide 47

Slide 47 text

App code -> statsD -> Graphite -> Dashboards -> Insights by hand by hand by hand by hand by hand

Slide 48

Slide 48 text

Developers just want to code

Slide 49

Slide 49 text

Let’s automate

Slide 50

Slide 50 text

Which is easier to automate? Insights or data collection?

Slide 51

Slide 51 text

Insights are sexy and fun

Slide 52

Slide 52 text

Collection is hard And unsexy

Slide 53

Slide 53 text

Collection is hard And Boring

Slide 54

Slide 54 text

Collection is hard And unsexy

Slide 55

Slide 55 text

We’re on track to have excellent automated insights

Slide 56

Slide 56 text

anomaly detection

Slide 57

Slide 57 text

App code -> statsD -> Graphite -> Dashboards -> Insights by hand by hand by hand AUTOMATIC! AUTOMATIC!

Slide 58

Slide 58 text

(…if only we had the data)

Slide 59

Slide 59 text

How do we automate data collection?

Slide 60

Slide 60 text

currently have ganglia, New relic, collectD, etc

Slide 61

Slide 61 text

NOT WHAT WE NEED

Slide 62

Slide 62 text

they provide data about your raw machines, not your CUSTOM DEVELOPED TECHNOLOGY And Application level logic

Slide 63

Slide 63 text

Healthy servers don’t make you money. Healthy services do.

Slide 64

Slide 64 text

enter LARIMAR

Slide 65

Slide 65 text

Full disclosure: this is my new PROJECT ! we’re going to talk about it because i’m pretty excited and the beta is opening up soon.

Slide 66

Slide 66 text

LARIMAR uses raw machine metrics to infer App level architecture and inform developers about problems

Slide 67

Slide 67 text

A service: cpu resources disk io PCAP data ports Used syscalls

Slide 68

Slide 68 text

A service: cpu resources disk io PCAP data ports Used syscalls service fingerprint MACHINE LEARNING

Slide 69

Slide 69 text

A service: cpu resources disk io PCAP data ports Used syscalls ABNORMAL BEHAVIOR MACHINE LEARNING

Slide 70

Slide 70 text

a system: service service service service service MACHINE LEARNING graphical system fingerprint

Slide 71

Slide 71 text

a system: service service service service service MACHINE LEARNING Abnormal, holistic system behavior

Slide 72

Slide 72 text

Larimar automates both analysis And Relevant data collection

Slide 73

Slide 73 text

so your developers can focus on coding and acting on insights

Slide 74

Slide 74 text

No configuration!

Slide 75

Slide 75 text

App code -> statsD -> Graphite -> Dashboards -> Insights by hand AUTOMATIC! AUTOMATIC! AUTOMATIC! AUTOMATIC!

Slide 76

Slide 76 text

1. collect 2. analyze 3. ACT 4. Profit!

Slide 77

Slide 77 text

organizational shifts are still needed to inspire ACTION on Data

Slide 78

Slide 78 text

but ACTION is easier to inspire when there is lots of data and lots of insight everywhere