Our 7 Habits of DevOps Success
Lessons learnt from Microsoft’s journey to DevOps
Clemri Steyn
DevOps and Application Lifecycle Product Marketing lead
Visual Studio Product Marketing
Slide 6
Slide 6 text
W H Y D E V O P S ?
O U R D E V O P S J O U R N E Y
Slide 7
Slide 7 text
W H AT I S D E V O P S ?
O U R D E V O P S J O U R N E Y
It’s Development and
Operations collaboration
It’s a job title
It’s automation
It means faster and
smaller releases
Slide 8
Slide 8 text
A C O N V E R G I N G
O F L I F E C YC L E S
O U R D E V O P S J O U R N E Y
DEVELOPMENT
DEVELOP+TEST
PLAN+TRACK
OPERATIONS
MONITOR + LEARN
RELEASE
Slide 9
Slide 9 text
T H E D E V O P S
C O N V E R S AT I O N
O U R D E V O P S J O U R N E Y
Slide 10
Slide 10 text
O U R D E V O P S H A B I T S
A N D P R A C T I C E S
O U R D E V O P S J O U R N E Y
FLOW OF
CUSTOMER
VALUE
AUTONOMY
and
ALIGNMENT
BACKLOG
improved by
LEARNING
EVIDENCE
gathered in
PRODUCTION
MANAGED
TECHNICAL
DEBT
PRODUCTION
FIRST
MINDSET
INFRA is a
FLEXIBLE
RESOURCE
Slide 11
Slide 11 text
F LO W O F
C U S TO M E R VA LU E
O U R D E V O P S J O U R N E Y
Slide 12
Slide 12 text
C O N T I N U O U S I N T E G R AT I O N
C O N T I N U O U S D E P LOY M E N T
R E L E A S E M A N A G E M E N T
AU TO M AT E D T E S T I N G
O U R P R A C T I C E S F O R I M P R O V I N G
F LO W O F C U S TO M E R VA LU E
O U R D E V O P S J O U R N E Y
Slide 13
Slide 13 text
T E A M AU TO N O M Y
A N D A L I G N M E N T
O U R D E V O P S J O U R N E Y
Slide 14
Slide 14 text
O U R P R A C T I C E S F O R I N C R E A S I N G
A L I G N M E N T A N D AU TO N O M Y
O U R D E V O P S J O U R N E Y
S E L F - M A N A G I N G T E A M S
PL ANNING CHATS
F E AT U R E C R E W S
SPRINT RITUALS
SCALED AGILE
TE AM ROOMS
Slide 15
Slide 15 text
B A C K LO G I M P R O V E D
BY L E A R N I N G
O U R D E V O P S J O U R N E Y
Slide 16
Slide 16 text
O U R P R A C T I C E S F O R
C O N T I N U O U S I M P R O V E M E N T
O U R D E V O P S J O U R N E Y
U S E R T E L E M E T R Y
U S A G E M O N I TO R I N G
S TA K E H O L D E R F E E D B A C K
A / B T E S T I N G I N P R O D U C T I O N
Slide 17
Slide 17 text
E V I D E N C E G AT H E R E D
I N P R O D U C T I O N
O U R D E V O P S J O U R N E Y
Slide 18
Slide 18 text
O U R P R A C T I C E S F O R
G AT H E R I N G E V I D E N C E
O U R D E V O P S J O U R N E Y
T E S T I N G I N P R O D U C T I O N
STAKEHOLDER FEEDBACK
U S A G E M O N I TO R I N G
U S E R T E L E M E T R Y
F E AT U R E F L A G S
Slide 19
Slide 19 text
M A N A G E D
T E C H N I C A L D E BT
O U R D E V O P S J O U R N E Y
Slide 20
Slide 20 text
O U R P R A C T I C E S F O R
M A N A G I N G T E C H N I C A L D E BT
O U R D E V O P S J O U R N E Y
C O D E M E T R I C S
T E S T AU TO M AT I O N
P E E R C O D E R E V I E W S
C O N T I N U O U S I N T E G R AT I O N
C O N T I N U O U S U N I T T E S T I N G
Slide 21
Slide 21 text
P R O D U C T I O N - F I R S T
M I N D S E T
O U R D E V O P S J O U R N E Y
Slide 22
Slide 22 text
O U R P R A C T I C E S F O R A
P R O D U C T I O N - F I R S T M I N D S E T
O U R D E V O P S J O U R N E Y
A P P L I C AT I O N P E R F O R M A N C E M A N A G E M E N T
C O N F I G U R AT I O N M A N A G E M E N T
I N F R A S T R U C T U R E A S C O D E
AU TO M AT E D R E C O V E R Y
C O N T I N U O U S D E L I V E R Y
R E L E A S E M A N A G E M E N T
Slide 23
Slide 23 text
I N F R A S T R U C T U R E I S A
F L E X I B L E R E S O U R C E
O U R D E V O P S J O U R N E Y
Slide 24
Slide 24 text
O U R P R A C T I C E S F O R
F L E X I B L E I N F R A S T R U C T U R E
O U R D E V O P S J O U R N E Y
I N F R A S T R U C T U R E A S C O D E
D E V E LO P E R S A N D B OX I N G
C LO U D D E V / T E S T L A B S
C O N TA I N E R I Z AT I O N
M I C R O S E R V I C E S
AU TO S C A L I N G
FA I LO V E R
Slide 25
Slide 25 text
OLD WORLD
Focus on planning
Compete, not collaborate
Static hierarchies
Individual productivity
Efficiency of process
Assumptions, not data
NEW WORLD
Focus on delivering
Collaborate to win
Fluid and flexible teams
Collective value creation
Effectiveness of outcomes
Experiment, learn and respond
O U R S H I F T TO D E V O P S
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
No content
Slide 34
Slide 34 text
No content
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
No content
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
No content
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
No content
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
No content
Slide 47
Slide 47 text
No content
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
Beyond DevOps ?
What’s NEXT
Slide 52
Slide 52 text
The Method
We are what we repeatedly do. Excellence, then, is not an
act, but a habit.
~ Aristotle
Slide 53
Slide 53 text
Old School
Stable requirements
Known, mature technology
No new undertaking
Done this before
Plan
Require
ment
Develop
Test
Deploy
Operate
Functional Test
System Test
Performance Test
Reliability Test
Usability Test
Business need à 3 years à application
Slide 54
Slide 54 text
Complex projects ? Use Agile
Slide 55
Slide 55 text
Agile is Designed to deal with
• Ziv’s Law - specifications will never be fully understood.
• Humphrey’s law - the user will never know what they want until after the system
is in production (maybe not even then)
• Wegner’s lemma - an interactive system can never be fully specified nor can it
ever be fully tested.
• Langdon’s lemma - software evolves more rapidly as it approaches chaotic
regions (taking care not to spill over into chaos)
Slide 56
Slide 56 text
Agile Timeline
1992 - Crystal Methods
1993 – Refactoring
1995 – Scrum
1995 - Pair Development
1999 - Extreme Programming
1999 - The Pragmatic Programmer
2002 - Test Driven Development
1997 - Feature Driven Development
1999- Adaptive Software Development
2003 – Lean Software Development
Slide 57
Slide 57 text
The Agile Manifesto
In February 2001, the Manifesto for Agile Software Development was published.
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
While there is value in the items on the right, the value of the items on the left is more.
Slide 58
Slide 58 text
Agile
Business Design Development Ops Support
Plan Req. Develop Operate
Deploy
P
L
A
N
R
E
Q
.
D
E
V
.
T
E
S
T
P
L
A
N
R
E
Q
.
D
E
V
.
T
E
S
T
P
L
A
N
R
E
Q
.
D
E
V
.
T
E
S
T
Delay Delay
Slide 59
Slide 59 text
Beyond Agile Manifesto
Even a great manifesto needs to evolve:
•
Team vision and discipline over Individuals and interactions over processes and tools
•
Validated learning over Working software over comprehensive documentation
•
Customer discovery over Customer collaboration over contract negotiation
•
Initiating change over Responding to change over following a plan
Slide 60
Slide 60 text
DevOps Timeline
2008 - Google group called “Agile
System Administration”
June 2009 - 10+ Deploys per Day –
Presentation by Flickr
2009: #DevOps hashtag born on
Twitter
2010 First US DevOpsDays in
Mountain View, CA
Slide 61
Slide 61 text
DevOps
Business Design Development Ops Support
Plan Req. Develop Operate
Deploy
Deploy to production
Continuous Planning
Continuous Experimentation
Continuous Integration
Continuous Test
Manual Test
Continuous Deploy
Deploy to test and staging
(production-like)
Unit Test
Function Test
Performance Test
System Test
Reliability Test
Slide 62
Slide 62 text
DevOps Metrics
Time to Delivery
Deployment Frequency
Change Volume
Success Rate
MTTR (Mean Time to Recovery)
Slide 63
Slide 63 text
Innovation Management is in Crises
Slide 64
Slide 64 text
Beyond DevOps
Few teams went beyond classical DevOps using the full power of
the cloud, full-stack engineering, micro-services,
containerization where new term appear:
● NoOps
● DistributedOps
● BizDevOps,
Slide 65
Slide 65 text
Beyond DevOps
Development Ops Support
Develop Operate
Deploy
Continuous Planning
Continuous Experimentation
Continuous Integration
Continuous Test
Continuous Deploy
Continuous Operation
Deploy to staging and
production
Unit Test
Function Test
Performance Test
System Test
Reliability Test
Slide 66
Slide 66 text
Capital vs Innovation
Venture capital invested in the valley 15 Bilion USD every year.
SP 500 companies are expected only in 2016 to distribute into
share holder accounts 1,000,000,000,000,000,000 USD.
Slide 67
Slide 67 text
Pace of Innovation vs Adoption
Slide 68
Slide 68 text
Disruption
In 80s startup where looking for the Big company’s that was poorly run.
From mid 90s Startup main attack area was Big corporations well managed
In present even the startup’s as old as 5 year already expecting to be disrupted
by new comers.
Slide 69
Slide 69 text
What can we expect Next ?
Startup era mentality in enterprise (The Lean Startup)
Enterprise will start decoupling innovation (The Innovator's dilemma)
Blurred line between Business , Development and Operation (One Team)
Competition around the speed of innovation (shift main focus from sustaining
innovation to disruptive innovation)
Disruption at the core of short cycle of innovation (Disrupt older disruption)
Slide 70
Slide 70 text
We are hiring
Email: ion.mudreac@sc.com
Linkedin: https://sg.linkedin.com/in/mudrii
CI/CD
Standard pathway for all services
Unit Tests
Integration Tests
Slide 84
Slide 84 text
Engineering Onboarding
Start early, start better.
Raise the bar with new hires.
Slide 85
Slide 85 text
Consistent Naming
Service name = Docker name = Repo name
= monitoring namespace = load balancer name
= CI name
Slide 86
Slide 86 text
Standard Provisioning
Ansible/Chef etc.
Docker - use and throw.
Slide 87
Slide 87 text
Standard Configuration
Consul, Zookeeper etc.
Hot reloadable configuration
Deploy faster, fix faster
Slide 88
Slide 88 text
DevOps as a Shared Bus
Identify patterns across all services
Easier to handle complexity of microservices
Ability to debug at any level
Slide 89
Slide 89 text
DevOps as a Shared Bus
Small teams with more context and communication
are better than large teams with no communication
Work closely with developers
Slide 90
Slide 90 text
Enforce Standards
But with logic and common sense.
(and some love, for our developers)
Slide 91
Slide 91 text
Its all about
Give and Take.
(Take sudo, Give advice)
Slide 92
Slide 92 text
No content
Slide 93
Slide 93 text
No content
Slide 94
Slide 94 text
No content
Slide 95
Slide 95 text
Security at DevOps Speed
Stefan Streichsbier
CTO Vantage Point
Founder DevSecOps Singapore
Slide 96
Slide 96 text
What is
AppSec?
Slide 97
Slide 97 text
Why does
AppSec ==
Pain?
Slide 98
Slide 98 text
Pentesters after
turning a report
in...
Slide 99
Slide 99 text
Security
Slide 100
Slide 100 text
Meanwhile
outside the
security camp
...
Slide 101
Slide 101 text
The frequency of releases over time
0
30
60
90
120
2005 2010 2015 2020
Releases per app per year
Towards CD
From Waterfall
The frequency increased
Slide 102
Slide 102 text
102
So many releases?!
Slide 103
Slide 103 text
Security
DevOps
Slide 104
Slide 104 text
104
Agile + DevOps + Security = DevSecOps
Slide 105
Slide 105 text
Step 1:
Security as
part of Agile
Slide 106
Slide 106 text
1-4
Weeks
24 hours
Develop
Test
Design
Plan
Output
Shippable Increment
Product Backlog Sprint Backlog
Let’s look at SCRUM
Start with understanding the process
Vulnerability
Repository
• Security Unit Tests
• SAST
• SCA
• DAST
• IAST
• VA
• Security as Code
• RASP
• NG WAF
• Red Team
• GOPT
• Actual Attackers
• Sec Requirements
• Design Review
• Threat Modelling
AppSec Pipeline
Slide 113
Slide 113 text
Instead of this ...
Slide 114
Slide 114 text
...Let’s do this...
Slide 115
Slide 115 text
No content
Slide 116
Slide 116 text
No content
Slide 117
Slide 117 text
TEN TIPS
to create a
TOXIC
CULTURE
. . . . . . . . . . . . . . . .
Slide 118
Slide 118 text
DISCOURAGE
SALARY DISCUSSIONS
Slide 119
Slide 119 text
DISCOURAGE
SALARY DISCUSSIONS
Slide 120
Slide 120 text
HAVE ABSTRACT
VALUES &
EXPECTATIONS
Slide 121
Slide 121 text
CREATE SCAPEGOATS
Slide 122
Slide 122 text
CREATE SCAPEGOATS
Slide 123
Slide 123 text
ONLY HIRE
CRONIES
Slide 124
Slide 124 text
ONLY HIRE
CRONIES
Slide 125
Slide 125 text
PLAY
FAVORITES
Slide 126
Slide 126 text
OFFER
UNLIMITED
VACATION
Slide 127
Slide 127 text
OFFER
UNLIMITED
VACATION
Slide 128
Slide 128 text
OFFER
UNLIMITED
VACATION
Slide 129
Slide 129 text
DESTROY
WORK-LIFE
BALANCE
Slide 130
Slide 130 text
ELIMINATE
LOYALTY
Slide 131
Slide 131 text
ELIMINATE
LOYALTY
Slide 132
Slide 132 text
TREAT HUMANS AS
DATA POINTS
Slide 133
Slide 133 text
TREAT HUMANS AS
DATA POINTS
Slide 134
Slide 134 text
MAKE FAKE PROMISES
Slide 135
Slide 135 text
MAKE FAKE PROMISES
Slide 136
Slide 136 text
THANK YOU!
@sethvargo
. . . . . . . . . . . .
Slide 137
Slide 137 text
No content
Slide 138
Slide 138 text
No content
Slide 139
Slide 139 text
Wanna keep it simple?
Clemens Perz
Slide 140
Slide 140 text
Wanna keep it simple. Can?
Slide 141
Slide 141 text
How much simple is left?
Slide 142
Slide 142 text
1995: The era of „simple“
1995
# Internet: Easy to build websites
# Mostly static content
# Dynamic content on the way
# Rudimental CSS
# Deploy via tar -xzvf :)
Slide 143
Slide 143 text
2005: And then there was 2.0
1995 2005
# Internet: Web 2.0
# Everything must be dynamic
# First wave of API's
# CSS 3.0 kicks off
# Complex Web Applications on the way
# Database backends
# Frontend servers
# Load balancers
# Deployments get complex, too
Slide 144
Slide 144 text
2015: The Big Everything
1995 2005 2015
# Internet: What Web are we at?
# Everything is microsliced
# Meshed via tons of RestAPI's
# Applications are run on Clouds
# Workloads virtualized
# Applications containerized
# How the hell do we manage all this?
Slide 145
Slide 145 text
Complification!
1995 2005 2015
# The number of complex/complicated
# questions to answer
# problems to fix
# features to implement
# incidents to handle
# decisions to make
Slide 146
Slide 146 text
Complification!
1995 2005 2015
# The number of complex/complicated
# questions to answer
# problems to fix
# features to implement
# incidents to handle
# decisions to make
Slide 147
Slide 147 text
Cynefin Framework
Complex
Emergent Practice
Complicated
Good Practice
Chaotic
Novel Practice
Simple/Obvious
Best Practice
Slide 148
Slide 148 text
Cynefin Framework
Complex
Emergent Practice
Complicated
Good Practice
Chaotic
Novel Practice
Simple/Obvious
Best Practice
Slide 149
Slide 149 text
Cynefin Framework
Complex
Emergent Practice
Complicated
Good Practice
Chaotic
Novel Practice
Simple/Obvious
Best Practice
Slide 150
Slide 150 text
Cynefin Framework
Complex
Emergent Practice
Complicated
Good Practice
Chaotic
Novel Practice
Simple/Obvious
Best Practice
Slide 151
Slide 151 text
Cynefin Framework
Complex
Emergent Practice
Complicated
Good Practice
Chaotic
Novel Practice
Simple/Obvious
Best Practice
Slide 152
Slide 152 text
Cynefin Framework
Complex
Emergent Practice
Complicated
Good Practice
Chaotic
Novel Practice
Simple/Obvious
Best Practice
Slide 153
Slide 153 text
The variety of problem and solution.
Slide 154
Slide 154 text
Where we stumble over it:
# Customer support
# Car repairs
# Operations :D
Slide 155
Slide 155 text
Is it clever or even possible to ignore the variety?
Slide 156
Slide 156 text
Ashby's Law of Requisite Variety
„Only variety absorbs variety“
Slide 157
Slide 157 text
A simple system
0
0 1 2 3 4
Slide 158
Slide 158 text
Can't keep it simple, stupid!
#The variety of our systems grows towards complex!
#DevOps relies on
−crossfunctional teams to deal with complexity
−automation to deal with order
−continiuous improvement to grow requisite variety
Slide 159
Slide 159 text
No content
Slide 160
Slide 160 text
No content
Slide 161
Slide 161 text
Infrastructure as code
@ Snapdeal
Yagnik
Architect
Cloud & Infrastructure
Slide 162
Slide 162 text
….once upon a time
• microservices architecture
• ~ 5000 nodes in AWS
• devops handle provisioning, release and
operations
• no visibility
Slide 163
Slide 163 text
No content
Slide 164
Slide 164 text
“If you know the enemy and know
yourself, you need not fear the result of
a hundred battles”
- Sun Tzu, The Art of War
Slide 165
Slide 165 text
“If you know the enemy and know
yourself, you need not fear the result of
a hundred battles”
- Sun Tzu, The Art of War
Building Infrastructure for
Massive Open Online Courses
Rizky Ariestiyansyah
DevOpsDays Singapore 2016
Slide 184
Slide 184 text
HI!
RIZKY ARIESTIYANSYAH
HI!
RIZKY ARIESTIYANSYAH
MOZILLA TECH SPEAKER
CTO of INDONESIAX.CO.ID
@ariestiyansyah
Slide 185
Slide 185 text
What is MOOC
Massive
Student number
can be 100.000+
Open Online Course
Study any course,
anywhere at any
time
Blended course Learning units in
an academic
subject, life skill.
Slide 186
Slide 186 text
Python for App
- Minimal Setup
- Hierarchical Module System
- Available Libraries
Slide 187
Slide 187 text
LMS
1.Learning Management
System for User
2.Forum Discussions
3.Exam
4.Assesment
5.Student Development
Slide 188
Slide 188 text
LMS
1.Learning Management
System for User
2.Forum Discussions
3.Exam
4.Assesment
5.Student Development
CMS
1.Course Management System
2.Instructor Dashboard
Slide 189
Slide 189 text
The Architecture
APP
Slide 190
Slide 190 text
Focus: MySQL
Company
Process Username Port Function
mysqld mysql 3306 Database
● DB for: user profile, course enrollment,
certificate
● Authentication and Authorization data as well
Slide 191
Slide 191 text
Focus: MySQL
Company
Process Username Port Function
mysqld mysql 3306 Database
● DB for: user profile, course enrollment,
certificate
● Authentication and Authorization data as well
Slide 192
Slide 192 text
Focus: MongoDB
Company
● Easy to scale
● Storing course data
● NoSQL DB Backend
Process Username Port Function
mongodb mongo 27017 Database
Slide 193
Slide 193 text
Focus: MongoDB
Company
● Easy to scale
● Storing course data
● NoSQL DB Backend
Process Username Port Function
mongodb mongo 27017 Database
Slide 194
Slide 194 text
Focus: Memcached
Company
● High speed distributed key-value (KV) store
for object caching.
● Cache user session, course structure info,
ect..
Process Username Port Function
memcached memcache 11211 KV store
Slide 195
Slide 195 text
Focus: Memcached
Company
● High speed distributed key-value (KV) store
for object caching.
● Cache user session, course structure info,
ect..
Process Username Port Function
memcached memcache 11211 KV store
Slide 196
Slide 196 text
Focus: Nginx
Company
● Web Server.
● Provided proxied.
● Load Balanced access to static and dynamic
assets.
● Stateless and horizontally scalable.
Process Username Worker Port Function
nginx www-data 4 80 Course
nginx www-data 4 8010 Course
Manager
Slide 197
Slide 197 text
Focus: Nginx
Company
● Web Server.
● Provided proxied.
● Load Balanced access to static and dynamic
assets.
● Stateless and horizontally scalable.
Process Username Worker Port Function
nginx www-data 4 80 Course
nginx www-data 4 8010 Course
Manager
Slide 198
Slide 198 text
Focus: Gunicorn
Company
● Dynamic worker management.
● Stateless and horizontally scalable.
Process Username Worker Port Function
gunicorn www-data 4 8000 Course
gunicorn www-data 4 8010 Course
Manager
Slide 199
Slide 199 text
Focus: Gunicorn
Company
● Dynamic worker management.
● Stateless and horizontally scalable.
Process Username Worker Port Function
gunicorn www-data 4 8000 Course
gunicorn www-data 4 8010 Course
Manager
Slide 200
Slide 200 text
Focus: RabbitMQ
Company
● Large jobs are run through queue.
● Coordinated by celery.
● It is a software where queues can be defined,
applications may connect to the queue and
transfer a message onto it.
Process Username Port Function
beam.smp rabbitmq * :5672 Message
Queue
beam.smp rabbitmq 15672 Message
Queue
epmd rabbitmq 4369 Port Mapper
Slide 201
Slide 201 text
Focus: RabbitMQ
Company
● Large jobs are run through queue.
● Coordinated by celery.
● It is a software where queues can be defined,
applications may connect to the queue and
transfer a message onto it.
Process Username Port Function
beam.smp rabbitmq * :5672 Message
Queue
beam.smp rabbitmq 15672 Message
Queue
epmd rabbitmq 4369 Port Mapper
Slide 202
Slide 202 text
Thank You!
Rizky Ariestiyansyah
@ariestiyansyah
oonlab.com
See You!