Alternate Titles
●
“Operations Eye for the Dev Person”
●
“Operations 101”
●
“Everything you want to know about Ops but
were afraid to ask”
●
“Foreign Exchange for make benefit of
glorious Nation of Operations”
●
“Care and Feeding of your Ops team”
●
“Ops is from Venus, Development is from
Mars”
Slide 3
Slide 3 text
But first a word from our “sponsors”
●
Hosted or On-premise cloud management
●
Abstracts all the Things
●
Governance
●
You should ask me about it
Slide 4
Slide 4 text
Who the hell am I?
●
@lusis every-effing-where
●
Atlanta Native
●
DevOps-y kind of guy
●
Does Ruby (most of the time)
●
Does Python (other times)
●
Does Java (well, once like 2 days ago)
●
#monitoringsucks
●
Hate Driven Development
Slide 5
Slide 5 text
Let's talk about Ops
Slide 6
Slide 6 text
Ops Tradition
Slide 7
Slide 7 text
Ops Tradition
You shall not pass! (lol)
Slide 8
Slide 8 text
@shitallspawsays
“If you care at all about these things, you're doing Ops”
●
Metrics collection
●
PaaS/IaaS evaluation/investigation
●
Automation (auto-build, auto-recovery)
●
Fault tolerance
●
Availability
●
Monitoring
●
Performance
●
Capex and Opex forecasting
●
Outage response
Slide 9
Slide 9 text
@shitallspawsays
“If you care at all about these things, you're doing Ops”
●
Metrics collection
●
PaaS/IaaS evaluation/investigation
●
Automation (auto-build, auto-recovery)
●
Fault tolerance
●
Availability
●
Monitoring
●
Performance
●
Capex and Opex forecasting
●
Outage response
Slide 10
Slide 10 text
What is “The Metric”
“an analytical measurement intended to quantify
the state of a system” - W. I. Pedia
Slide 11
Slide 11 text
Does this look familiar?
Started GET "/" for 127.0.0.1 at 2012-03-10 14:28:14 +0100
Processing by HomeController#index as HTML
Rendered text template within layouts/application (0.0ms)
Rendered layouts/_assets.html.erb (2.0ms)
Rendered layouts/_top.html.erb (2.6ms)
Rendered layouts/_about.html.erb (0.3ms)
Rendered layouts/_google_analytics.html.erb (0.4ms)
Completed 200 OK in 79ms (Views: 78.8ms | ActiveRecord:
0.0ms)
Slide 12
Slide 12 text
Does this look familiar?
Started GET "/" for 127.0.0.1 at 2012-03-10 14:28:14 +0100
Processing by HomeController#index as HTML
Rendered text template within layouts/application (0.0ms)
Rendered layouts/_assets.html.erb (2.0ms)
Rendered layouts/_top.html.erb (2.6ms)
Rendered layouts/_about.html.erb (0.3ms)
Rendered layouts/_google_analytics.html.erb (0.4ms)
Completed 200 OK in 79ms (Views: 78.8ms | ActiveRecord:
0.0ms)
Slide 13
Slide 13 text
So that's great for humans but
●
what if you want to do this?
Slide 14
Slide 14 text
Well you could....
●
grep ms | awk | sed | grep | sort | cry
●
Performant?
●
Realistic?
●
Scalable?
●
Do you seriously want to do this?
Slide 15
Slide 15 text
What about Business Metrics?
●
Failed Logins
●
Shopping Cart Abandon Rates
●
New Account creations
●
Slide 16
Slide 16 text
What about Business Metrics?
●
Failed Logins
●
Shopping Cart Abandon Rates
●
New Account creations
●
AND DID ANY OF IT GET BETTER OR
WORSE WITH A NEW CODE RELEASE?
Slide 17
Slide 17 text
Let's talk about Ops
Slide 18
Slide 18 text
Baking operations into your code
(easy)
●
Visibility – Implement a “ping” page
●
Low-impact
●
Exercise the full stack (call DB, perform
some safe action, Easy to Parse, Easy to
Check)
●
Please don't expect folks to robot a login
session
Slide 19
Slide 19 text
Baking operations into your code
(awesome)
●
Visibility – Push Metrics
●
StatsD + Graphite (DIY)
●
Librato, New Relic (aaS)
●
Read this -
http://codeascraft.etsy.com/2011/02/15/measure-an
●
Watch This -
http://pivotallabs.com/talks/139-metrics-metrics-e
●
Try This -
https://github.com/johnewart/ruby-metrics
Slide 20
Slide 20 text
Let's talk about Logs
Slide 21
Slide 21 text
Remember This?
Started GET "/" for 127.0.0.1 at 2012-03-10 14:28:14 +0100
Processing by HomeController#index as HTML
Rendered text template within layouts/application (0.0ms)
Rendered layouts/_assets.html.erb (2.0ms)
Rendered layouts/_top.html.erb (2.6ms)
Rendered layouts/_about.html.erb (0.3ms)
Rendered layouts/_google_analytics.html.erb (0.4ms)
Completed 200 OK in 79ms (Views: 78.8ms | ActiveRecord:
0.0ms)
Slide 22
Slide 22 text
Logstash
●
Started by @jordansissel and @fetep ~2004
●
Inputs → Filters → Outputs
●
Jruby (and increasingly better YARV support)
●
UNIX pipe with Radioactive Super Powers
Slide 23
Slide 23 text
Works like this
Slide 24
Slide 24 text
More like this
Slide 25
Slide 25 text
Can give you this
Slide 26
Slide 26 text
Config language
Not Ruby. It'll be okay.
Slide 27
Slide 27 text
Basic Config
Slide 28
Slide 28 text
Slightly Advanced Config
Slide 29
Slide 29 text
Filters all up in your shit
Slide 30
Slide 30 text
Remember This?
Started GET "/" for 127.0.0.1 at 2012-03-10 14:28:14 +0100
Processing by HomeController#index as HTML
Rendered text template within layouts/application (0.0ms)
Rendered layouts/_assets.html.erb (2.0ms)
Rendered layouts/_top.html.erb (2.6ms)
Rendered layouts/_about.html.erb (0.3ms)
Rendered layouts/_google_analytics.html.erb (0.4ms)
Completed 200 OK in 79ms (Views: 78.8ms | ActiveRecord:
0.0ms)
Slide 31
Slide 31 text
Lookie here
Slide 32
Slide 32 text
Annoying to parse though
●
Try this:
https://github.com/mattmatt/lograge
Slide 33
Slide 33 text
Remember This?
Started GET "/" for 127.0.0.1 at 2012-03-10 14:28:14 +0100
Processing by HomeController#index as HTML
Rendered text template within layouts/application (0.0ms)
Rendered layouts/_assets.html.erb (2.0ms)
Rendered layouts/_top.html.erb (2.6ms)
Rendered layouts/_about.html.erb (0.3ms)
Rendered layouts/_google_analytics.html.erb (0.4ms)
Completed 200 OK in 79ms (Views: 78.8ms | ActiveRecord:
0.0ms)
Slide 34
Slide 34 text
Lookie here
GET /jobs/833552.json format=json
action=jobs#show status=200
duration=58.33 view=40.43 db=15.26
●
yes that's a single line!
●
No more multiline
●
No more awkward parsing
Slide 35
Slide 35 text
Remember this?
Slide 36
Slide 36 text
Lookie here
Slide 37
Slide 37 text
Logs need to be readable by humans
AND machines
●
Single lines are better
●
JSON is readable by humans and Machines
●
Every language has JSON support
Slide 38
Slide 38 text
Baking operations into your code
●
Take little effort
●
Has great value
●
Makes everyone happy
●
Don't forget the non-functional
requirements (The “ilities”)
●
Stop inventing Timestamp formats (ISO8601 is
a thing)
●
Turn Kittens into PopTarts