Implementing Security
with DevOps
Matt Konda
Jemurai
Slide 2
Slide 2 text
Agenda
• Introductions
• What is DevOps to Security
• Case Study 1
• Case Study 2
• Pipeline
• Conceptual framework
• Maturity Model
• OWASP Projects
• How to Get Involved
Slide 3
Slide 3 text
Introduction
1997 2006 2014
Consultant
Engineer
Software
Architect
Director of
Engineering
Rabble Rouser:
Perl
Java Applet
C++
J2EE
J2EE
Spring
Analytics
Certificate Authority
Vulnerability Scanner
Penetration Test Manager
Pricing
Retail
Banking
Manufacturing
Pharma
Healthcare
Research
Ruby
Rails
Chicago BSides 2011, 2012
Defcon Skytalk
OWASP Chicago, MSP 2013
AppSec USA 2012, 2013
ChicagoRuby 2013
Secure 360
Lone Star Ruby 2013
WindyCityRails 2013
Chicago JUG 2014
RailsConf 2014
Converge 2014
MS in CS
Founder
Consultant
Agile
Clojure
Graph Database
Trying to hack a
business model that
succeeds while
helping developers.
Domains:
Projects:
DevOps / Automation
Training
Coaching
Code Review
Plugged in to SDLC
Consulting
Assessments
@mkonda
[email protected]
DevOps
Growing
Slide 4
Slide 4 text
Census?
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
What is DevOps?
Slide 7
Slide 7 text
In my opinion, DevOps is basically
Agile extended to include Ops as
first class stakeholders.
Slide 8
Slide 8 text
Of course, it is also a result of an
increasingly cloud oriented Ops
environment that is scripted with
Chef, Puppet, Ansible, etc.
Slide 9
Slide 9 text
automation
Slide 10
Slide 10 text
No content
Slide 11
Slide 11 text
visibility
Slide 12
Slide 12 text
mttr
Mean time to repair
Slide 13
Slide 13 text
mttd
Mean time to detect
Slide 14
Slide 14 text
empathy
Slide 15
Slide 15 text
accountability
Slide 16
Slide 16 text
culture
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
OK OK … but what does
this actually mean for
security?!?!?
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
We’re trying to get on
the bandwagon.
Slide 21
Slide 21 text
SecDevOps
DevSecOps
DevOpsSec
Rugged DevOps
Slide 22
Slide 22 text
Reinforces something I
learned as a developer:
naming is hard.
Slide 23
Slide 23 text
I think that security should just be
implied by DevOps and doesn’t
need to have another name.
Slide 24
Slide 24 text
Why do we care?
Slide 25
Slide 25 text
Being able to deploy quickly is
my #1 security feature.
- Nick Galbreath
Slide 26
Slide 26 text
Personally: I have never
seen anything change
development/IT like this.
Slide 27
Slide 27 text
Case Study #1
Slide 28
Slide 28 text
Credit: Matt Tesauro at AppSecEU 2015
http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
Slide 29
Slide 29 text
Need somewhere to keep
an inventory of
applications.
Slide 30
Slide 30 text
Credit: Matt Tesauro at AppSecEU 2015
http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
Slide 31
Slide 31 text
Always a way for
human intervention.
Slide 32
Slide 32 text
Credit: Matt Tesauro at AppSecEU 2015
http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
Slide 33
Slide 33 text
Automation
Slide 34
Slide 34 text
Credit: Matt Tesauro at AppSecEU 2015
http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
Slide 35
Slide 35 text
Digest and filter
Slide 36
Slide 36 text
Credit: Matt Tesauro at AppSecEU 2015
http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
Slide 37
Slide 37 text
Communicate with
developers
Slide 38
Slide 38 text
Credit: Matt Tesauro at AppSecEU 2015
http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
Slide 39
Slide 39 text
Visibility
Slide 40
Slide 40 text
Credit: Matt Tesauro at AppSecEU 2015
http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
Slide 41
Slide 41 text
Scripted provisioning
Slide 42
Slide 42 text
Credit: Matt Tesauro at AppSecEU 2015
http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
Slide 43
Slide 43 text
Credit: Matt Tesauro at AppSecEU 2015
http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
Slide 44
Slide 44 text
Case Study #2
Slide 45
Slide 45 text
App Inventory
Pipeline
CSV, JSON, Text
Git webhooks
Jenkins
Slide 46
Slide 46 text
Need somewhere to keep
an inventory of
applications.
Slide 47
Slide 47 text
App Inventory
Pipeline
CSV, JSON, Text
Git webhooks
Jenkins
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
Devs can interact and
trigger security checks.
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
Automation
Slide 52
Slide 52 text
App Inventory
Pipeline
CSV, JSON, Text
Git webhooks
Jenkins
Slide 53
Slide 53 text
Digest and filter
Slide 54
Slide 54 text
App Inventory
Pipeline
CSV, JSON, Text
Git webhooks
Jenkins
Slide 55
Slide 55 text
Communicate with
developers
Slide 56
Slide 56 text
App Inventory
Pipeline
CSV, JSON, Text
Git webhooks
Jenkins
Slide 57
Slide 57 text
Visibility
Slide 58
Slide 58 text
No content
Slide 59
Slide 59 text
App Inventory
Pipeline
CSV, JSON, Text
Git webhooks
Jenkins
Slide 60
Slide 60 text
Scripted provisioning
Slide 61
Slide 61 text
App Inventory
Pipeline
CSV, JSON, Text
Git webhooks
Jenkins
Slide 62
Slide 62 text
Most parts of this automation
toolchain are open source and
offer multiple ways to interact…
Slide 63
Slide 63 text
Pipeline
Slide 64
Slide 64 text
No content
Slide 65
Slide 65 text
Intended to make it easy
to do security automation.
Mounter
Currently: brakeman, bundler-audit,
owasp-dependency-check, secrets in
source, retire.js, scan.js
Future: many more possible.
Designed for extension.
Files Code
Slide 70
Slide 70 text
Mounter
Currently: ZAP (in progress)
Future: guantlt, etc.
Files Code App
Other Internals
• Within “Tasks”, each of the files, code and app
phases of the pipeline can be run selectively.
Mounter Files Code App Filter Reporter
“Tasks”
Slide 76
Slide 76 text
ruby bin/pipeline
-l code (Code analysis)
-d (Turn on debug)
-f text (Output format)
/area53/app/
Slide 77
Slide 77 text
Some valid…
Slide 78
Slide 78 text
Still noisy … but you can
dismiss and move on and
hopefully rarely see them.
Slide 79
Slide 79 text
What if it just automatically
ran against every
company github project?
Slide 80
Slide 80 text
Conceptual Framework
for Security in DevOps
Slide 81
Slide 81 text
Two passes
• First talk about security overlaid on continuous
delivery model
• Then talk about event based security and DevOps
related activities
Slide 82
Slide 82 text
Understand lifecycle
Slide 83
Slide 83 text
No content
Slide 84
Slide 84 text
continuous delivery
Slide 85
Slide 85 text
Security sees this
and wants to …
Slide 86
Slide 86 text
continuous delivery
Slide 87
Slide 87 text
But we should embrace it.
Slide 88
Slide 88 text
Think incremental
Slide 89
Slide 89 text
No content
Slide 90
Slide 90 text
continuous delivery
Code Review
Security Unit Tests
Security Requirements
Slide 91
Slide 91 text
Automate security tools
Slide 92
Slide 92 text
continuous delivery
Security Tool Automation:
Code analysis
Security unit tests
Dynamic scanning
etc.
Slide 93
Slide 93 text
continuous delivery
Security Tests Run
Exploratory Testing Includes Security
Slide 94
Slide 94 text
A detailed example:
• Let’s say a feature is being developed
• Then devs and testers are checking a new feature
• Let them browse through an attack proxy (like Burp
or ZAP) in passive mode
• At night or when the system is quiet, use the
browsing pattern as seeds for overnight attacks
Slide 95
Slide 95 text
Continuous feedback
Slide 96
Slide 96 text
continuous delivery
Feedback!
Slide 97
Slide 97 text
EVIL
False
Positives
Are
a Necessary
Slide 98
Slide 98 text
Optimize for relevance
Slide 99
Slide 99 text
Provisioning tools
Slide 100
Slide 100 text
continuous delivery
Since its easy to provision
we can do security testing
safely in a new env.
Slide 101
Slide 101 text
Audit tools
Slide 102
Slide 102 text
continuous delivery
Deployment checks
includes security
audit checks.
Slide 103
Slide 103 text
Self documenting for
regulatory and
compliance!
Slide 104
Slide 104 text
Collect important data
Slide 105
Slide 105 text
No content
Slide 106
Slide 106 text
Chaos tools
Slide 107
Slide 107 text
Change is good
Slide 108
Slide 108 text
continuous delivery
Change is happening.
It can be an
opportunity
instead of a hassle.
Slide 109
Slide 109 text
Complexity is an enemy
Slide 110
Slide 110 text
continuous delivery
Small releases reduce complexity.
Decomposition to micro-services reduces dependencies and complexity.
Right now, security hurts.
Slide 111
Slide 111 text
Shared responsibility
Slide 112
Slide 112 text
continuous delivery
Another principle of software delivery: build security in!
Done means
secure!
Empowered to
do security right!
Deploy
• Scripted Provisioning / Built in Change Control
• Provisioning Auditing (Chef Audit, hardening.io)
• Gauntlt
Slide 117
Slide 117 text
Periodic
• Full app analysis (static, manual pen test)
• Secure Development Training
• Baseline Security Requirements Review
• ASVS Review
• Data Science on Results
Slide 118
Slide 118 text
Security Incident
Slide 119
Slide 119 text
Maturity Model
Slide 120
Slide 120 text
How do we know what
to actually DO?
Slide 121
Slide 121 text
Belts
Slide 122
Slide 122 text
Defining Kata
Slide 123
Slide 123 text
Kata Name: Run ZAP Proxy Daily
Kata Detail: Automate a way to run ZAP Proxy against an
app on a daily basis and report issues to Jira
How to Do the Kata: Activate “XYZ” Jenkins Plugin
Training Resource: A web page with specifics about this
Kata.
Experts: A reference to people that can help with this Kata.
Difficulty: Belt level for the Kata.
Security Objectives: What security objectives does this Kata
help us to achieve?
Slide 124
Slide 124 text
Data Classification
OWASP Top 10 Training
Simple Developer Environment Setup
Continuous Integration and Testing
Source Code Repository and Proper Tags
Repeatable Deployment
HTTPS (TLS) Everywhere
Slide 125
Slide 125 text
Baseline Security Requirements
Test for Security Headers
Lockout to Prevent Brute Force
Consistent Output Encoding
Audit Records Written
IDE Style Checks
Villain Persona and Security Requirements
Operational Metrics
Unauthenticated Scanning
Slide 126
Slide 126 text
Storing Secrets
Solid SSL
Security of Dependencies
Audit Records Written
Anti DoS
Anti-CSRF Protection
Consistent SQL Injection Protection
Static Analysis
Vulnerability Scanning
Authenticated Application Scanning
Security Code Review
Slide 127
Slide 127 text
Logs Aggregated with Security Event
Incident Management System
Appropriate Encryption
Antivirus / Malware
Business Metrics
HSTS and Certificate Pinning
Application Penetration Testing
Slide 128
Slide 128 text
Attack Awareness
Behavioral Blocking
Centralized Security Service
Security Unit Tests for Business Logic
Hands on Developer Security Training
Dynamic Analysis
Runtime Application Security Analysis
Application Honeypot
The intent is not to force everyone to
do every one of those things.
The intent is to help to identify the things that can be
done and layer them so that they can be prioritized.
I recognize that software has become a foundation of our
modern world.
I recognize the awesome responsibility that comes with this
foundational role.
I recognize that my code will be used in ways I cannot
anticipate, in ways it was not designed, and for longer than it
was ever intended.
I recognize that my code will be attacked by talented and
persistent adversaries who threaten our physical, economic
and national security.
I recognize these things – and I choose to be rugged.
Slide 140
Slide 140 text
I recognize that software has become a foundation of our
modern world.
I recognize the awesome responsibility that comes with this
foundational role.
I recognize that my code will be used in ways I cannot
anticipate, in ways it was not designed, and for longer than it
was ever intended.
I recognize that my code will be attacked by talented and
persistent adversaries who threaten our physical,
economic and national security.
I recognize these things – and I choose to be rugged.