Ignites - DevOpsDays Singapore 2016

You have five minutes to talk on a subject accompanied by 20 slides, for 15 seconds each, automatically advanced. What will you say?

Clemri Steyn - 7 Habits of successful DevOps
Benjamin Henshall - Are we bridging the cultural divide that is necessary for DevOps to succeed
Ion Mudreac - Beyond DevOps ?
Angad Singh - Devops and Standards
Stefan Streichsbier - Security at DevOps Speed
Seth Vargo - 10 Tips to Create a Toxic Tech Team
Clemens Perz - Can't keep it simple, stupid!
Yagnik Khanna - Infrastructure as code at Snapdeal
Rizky Ariestiyansyah - Building Infrastructure for MOOC

Transcript

5. Our 7 Habits of DevOps Success
 
 Lessons learnt from Microsoft’s journey to DevOps Clemri Steyn DevOps and Application Lifecycle Product Marketing lead Visual Studio Product Marketing
6. W H Y D E V O P S ? O U R D E V O P S J O U R N E Y
7. W H AT I S D E V O P S ? O U R D E V O P S J O U R N E Y It’s Development and
 Operations collaboration It’s a job title It’s automation It means faster and smaller releases
8. A C O N V E R G I N G 
 O F L I F E C YC L E S O U R D E V O P S J O U R N E Y DEVELOPMENT DEVELOP+TEST PLAN+TRACK OPERATIONS MONITOR + LEARN RELEASE
9. T H E D E V O P S C O N V E R S AT I O N O U R D E V O P S J O U R N E Y
10. O U R D E V O P S H A B I T S 
 A N D P R A C T I C E S O U R D E V O P S J O U R N E Y FLOW OF CUSTOMER VALUE AUTONOMY
 and
 ALIGNMENT BACKLOG improved by LEARNING EVIDENCE gathered in PRODUCTION MANAGED TECHNICAL DEBT PRODUCTION FIRST MINDSET INFRA is a FLEXIBLE RESOURCE
11. F LO W O F 
 C U S TO M E R VA LU E O U R D E V O P S J O U R N E Y
12. C O N T I N U O U S I N T E G R AT I O N 
 C O N T I N U O U S D E P LOY M E N T R E L E A S E M A N A G E M E N T AU TO M AT E D T E S T I N G O U R P R A C T I C E S F O R I M P R O V I N G 
 F LO W O F C U S TO M E R VA LU E O U R D E V O P S J O U R N E Y
13. T E A M AU TO N O M Y 
 A N D A L I G N M E N T O U R D E V O P S J O U R N E Y
14. O U R P R A C T I C E S F O R I N C R E A S I N G 
 A L I G N M E N T A N D AU TO N O M Y O U R D E V O P S J O U R N E Y S E L F - M A N A G I N G T E A M S PL ANNING CHATS F E AT U R E C R E W S SPRINT RITUALS SCALED AGILE TE AM ROOMS
15. B A C K LO G I M P R O V E D 
 BY L E A R N I N G O U R D E V O P S J O U R N E Y
16. O U R P R A C T I C E S F O R C O N T I N U O U S I M P R O V E M E N T O U R D E V O P S J O U R N E Y U S E R T E L E M E T R Y U S A G E M O N I TO R I N G S TA K E H O L D E R F E E D B A C K 
 A / B T E S T I N G I N P R O D U C T I O N
17. E V I D E N C E G AT H E R E D 
 I N P R O D U C T I O N O U R D E V O P S J O U R N E Y
18. O U R P R A C T I C E S F O R 
 G AT H E R I N G E V I D E N C E O U R D E V O P S J O U R N E Y T E S T I N G I N P R O D U C T I O N STAKEHOLDER FEEDBACK U S A G E M O N I TO R I N G U S E R T E L E M E T R Y F E AT U R E F L A G S
19. M A N A G E D 
 T E C H N I C A L D E BT O U R D E V O P S J O U R N E Y
20. O U R P R A C T I C E S F O R 
 M A N A G I N G T E C H N I C A L D E BT O U R D E V O P S J O U R N E Y C O D E M E T R I C S T E S T AU TO M AT I O N P E E R C O D E R E V I E W S C O N T I N U O U S I N T E G R AT I O N C O N T I N U O U S U N I T T E S T I N G
21. P R O D U C T I O N - F I R S T M I N D S E T O U R D E V O P S J O U R N E Y
22. O U R P R A C T I C E S F O R A P R O D U C T I O N - F I R S T M I N D S E T O U R D E V O P S J O U R N E Y A P P L I C AT I O N P E R F O R M A N C E M A N A G E M E N T C O N F I G U R AT I O N M A N A G E M E N T I N F R A S T R U C T U R E A S C O D E AU TO M AT E D R E C O V E R Y C O N T I N U O U S D E L I V E R Y R E L E A S E M A N A G E M E N T
23. I N F R A S T R U C T U R E I S A F L E X I B L E R E S O U R C E O U R D E V O P S J O U R N E Y
24. O U R P R A C T I C E S F O R F L E X I B L E I N F R A S T R U C T U R E O U R D E V O P S J O U R N E Y I N F R A S T R U C T U R E A S C O D E D E V E LO P E R S A N D B OX I N G C LO U D D E V / T E S T L A B S C O N TA I N E R I Z AT I O N M I C R O S E R V I C E S AU TO S C A L I N G FA I LO V E R
25. OLD WORLD Focus on planning Compete, not collaborate Static hierarchies Individual productivity Efficiency of process Assumptions, not data NEW WORLD Focus on delivering Collaborate to win Fluid and flexible teams Collective value creation Effectiveness of outcomes Experiment, learn and respond O U R S H I F T TO D E V O P S
51. Beyond DevOps ? What’s NEXT
52. The Method We are what we repeatedly do. Excellence, then, is not an act, but a habit. ~ Aristotle
53. Old School Stable requirements Known, mature technology No new undertaking Done this before Plan Require ment Develop Test Deploy Operate Functional Test System Test Performance Test Reliability Test Usability Test Business need à 3 years à application
54. Complex projects ? Use Agile
55. Agile is Designed to deal with • Ziv’s Law - specifications will never be fully understood. • Humphrey’s law - the user will never know what they want until after the system is in production (maybe not even then) • Wegner’s lemma - an interactive system can never be fully specified nor can it ever be fully tested. • Langdon’s lemma - software evolves more rapidly as it approaches chaotic regions (taking care not to spill over into chaos)
56. Agile Timeline 1992 - Crystal Methods 1993 – Refactoring 1995 – Scrum 1995 - Pair Development 1999 - Extreme Programming 1999 - The Pragmatic Programmer 2002 - Test Driven Development 1997 - Feature Driven Development 1999- Adaptive Software Development 2003 – Lean Software Development
57. The Agile Manifesto In February 2001, the Manifesto for Agile Software Development was published. Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan While there is value in the items on the right, the value of the items on the left is more.
58. Agile Business Design Development Ops Support Plan Req. Develop Operate Deploy P L A N R E Q . D E V . T E S T P L A N R E Q . D E V . T E S T P L A N R E Q . D E V . T E S T Delay Delay
59. Beyond Agile Manifesto Even a great manifesto needs to evolve: • Team vision and discipline over Individuals and interactions over processes and tools • Validated learning over Working software over comprehensive documentation • Customer discovery over Customer collaboration over contract negotiation • Initiating change over Responding to change over following a plan
60. DevOps Timeline 2008 - Google group called “Agile System Administration” June 2009 - 10+ Deploys per Day – Presentation by Flickr 2009: #DevOps hashtag born on Twitter 2010 First US DevOpsDays in Mountain View, CA
61. DevOps Business Design Development Ops Support Plan Req. Develop Operate Deploy Deploy to production Continuous Planning Continuous Experimentation Continuous Integration Continuous Test Manual Test Continuous Deploy Deploy to test and staging (production-like) Unit Test Function Test Performance Test System Test Reliability Test
62. DevOps Metrics Time to Delivery Deployment Frequency Change Volume Success Rate MTTR (Mean Time to Recovery)
63. Innovation Management is in Crises
64. Beyond DevOps Few teams went beyond classical DevOps using the full power of the cloud, full-stack engineering, micro-services, containerization where new term appear: • NoOps • DistributedOps • BizDevOps,
65. Beyond DevOps Development Ops Support Develop Operate Deploy Continuous Planning Continuous Experimentation Continuous Integration Continuous Test Continuous Deploy Continuous Operation Deploy to staging and production Unit Test Function Test Performance Test System Test Reliability Test
66. Capital vs Innovation 
 Venture capital invested in the valley 15 Bilion USD every year.
 SP 500 companies are expected only in 2016 to distribute into share holder accounts 1,000,000,000,000,000,000 USD.
67. Pace of Innovation vs Adoption
68. Disruption In 80s startup where looking for the Big company’s that was poorly run. From mid 90s Startup main attack area was Big corporations well managed In present even the startup’s as old as 5 year already expecting to be disrupted by new comers.

69. What can we expect Next ? Startup era mentality in enterprise (The Lean Startup) Enterprise will start decoupling innovation (The Innovator's dilemma) Blurred line between Business , Development and Operation (One Team) Competition around the speed of innovation (shift main focus from sustaining innovation to disruptive innovation) Disruption at the core of short cycle of innovation (Disrupt older disruption)
70. We are hiring Email: ion.mudreac@sc.com Linkedin: https://sg.linkedin.com/in/mudrii
73. Why standards matter Angad Singh @angadsg
75. DevOps Architecting (micro)services Infrastructure Automation Developer Productivity
76. DevOps Debugging all layers Incident Management Cost Management
79. Shared Libraries aka Reusability E.g. Redis/MySQL/xyz connection manager In memory caching Image transformations
80. Security E.g. Consistent Firewall rules Standard input sanitization Centralized Alerting
81. Developer Productivity No surprises when navigating the system e.g. All services log to /var/log One deploy toolchain to rule them all
82. Cost Optimization Engineering cost > Infrastructure cost Better Standards = Lesser Engineers
83. CI/CD Standard pathway for all services Unit Tests Integration Tests
84. Engineering Onboarding Start early, start better. Raise the bar with new hires.
85. Consistent Naming Service name = Docker name = Repo name = monitoring namespace = load balancer name = CI name
86. Standard Provisioning Ansible/Chef etc. Docker - use and throw.
87. Standard Configuration Consul, Zookeeper etc. Hot reloadable configuration Deploy faster, fix faster
88. DevOps as a Shared Bus Identify patterns across all services Easier to handle complexity of microservices Ability to debug at any level
89. DevOps as a Shared Bus Small teams with more context and communication are better than large teams with no communication Work closely with developers
90. Enforce Standards But with logic and common sense. (and some love, for our developers)
91. Its all about Give and Take. (Take sudo, Give advice)
95. Security at DevOps Speed Stefan Streichsbier CTO Vantage Point Founder DevSecOps Singapore
96. What is AppSec?
97. Why does AppSec == Pain?
98. 
 
 
 Pentesters after turning a report in...
99. Security
100. 
 
 
 Meanwhile outside the security camp
 ...
101. The frequency of releases over time 0 30 60 90 120 2005 2010 2015 2020 Releases per app per year Towards CD From Waterfall The frequency increased
102. 102 So many releases?!
103. Security DevOps
104. 104 Agile + DevOps + Security = DevSecOps
105. Step 1: Security as part of Agile
106. 1-4 Weeks 24 hours Develop Test Design Plan Output Shippable Increment Product Backlog Sprint Backlog Let’s look at SCRUM Start with understanding the process
107. 1-4 Weeks 24 hours Develop Test Design Plan Output Shippable Increment Product Backlog Sprint Backlog Secure SCRUM Security Training Security Requirements Security Activities Threat Modelling Design Review Pairing Manual Security Tests Automatic Security Tests Security Feature Demo Security Retrospective Security Acceptance Criteria
108. (Security) User Stories
109. (Security) Unit Tests
110. 0 25 50 75 100 Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5 Sprint 6 % Remaining Security work % App Robustness, Security Skills Security Debt Burndown
111. Step 2: DevSecOps
112. Vulnerability Repository • Security Unit Tests • SAST • SCA • DAST • IAST • VA • Security as Code • RASP • NG WAF • Red Team • GOPT • Actual Attackers • Sec Requirements • Design Review • Threat Modelling AppSec Pipeline
113. Instead of this ...
114. ...Let’s do this...
117. TEN TIPS to create a TOXIC CULTURE . . . . . . . . . . . . . . . .
118. DISCOURAGE SALARY DISCUSSIONS
119. DISCOURAGE SALARY DISCUSSIONS
120. HAVE ABSTRACT VALUES & EXPECTATIONS
121. CREATE SCAPEGOATS
122. CREATE SCAPEGOATS
123. ONLY HIRE CRONIES
124. ONLY HIRE CRONIES
125. PLAY FAVORITES
126. OFFER UNLIMITED VACATION
127. OFFER UNLIMITED VACATION
128. OFFER UNLIMITED VACATION
129. DESTROY WORK-LIFE BALANCE
130. ELIMINATE LOYALTY
131. ELIMINATE LOYALTY
132. TREAT HUMANS AS DATA POINTS
133. TREAT HUMANS AS DATA POINTS
134. MAKE FAKE PROMISES
135. MAKE FAKE PROMISES
136. THANK YOU! @sethvargo . . . . . . . . . . . .
139. Wanna keep it simple? Clemens Perz
140. Wanna keep it simple. Can?
141. How much simple is left?
142. 1995: The era of „simple“ 1995 # Internet: Easy to build websites # Mostly static content # Dynamic content on the way # Rudimental CSS # Deploy via tar -xzvf :)
143. 2005: And then there was 2.0 1995 2005 # Internet: Web 2.0 # Everything must be dynamic # First wave of API's # CSS 3.0 kicks off # Complex Web Applications on the way # Database backends # Frontend servers # Load balancers # Deployments get complex, too
144. 2015: The Big Everything 1995 2005 2015 # Internet: What Web are we at? # Everything is microsliced # Meshed via tons of RestAPI's # Applications are run on Clouds # Workloads virtualized # Applications containerized # How the hell do we manage all this?
145. Complification! 1995 2005 2015 # The number of complex/complicated # questions to answer # problems to fix # features to implement # incidents to handle # decisions to make
146. Complification! 1995 2005 2015 # The number of complex/complicated # questions to answer # problems to fix # features to implement # incidents to handle # decisions to make
147. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel Practice Simple/Obvious Best Practice
148. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel Practice Simple/Obvious Best Practice
149. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel Practice Simple/Obvious Best Practice
150. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel Practice Simple/Obvious Best Practice
151. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel Practice Simple/Obvious Best Practice
152. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel Practice Simple/Obvious Best Practice
153. The variety of problem and solution.
154. Where we stumble over it: # Customer support # Car repairs # Operations :D
155. Is it clever or even possible to ignore the variety?
156. Ashby's Law of Requisite Variety „Only variety absorbs variety“
157. A simple system 0 0 1 2 3 4
158. Can't keep it simple, stupid! #The variety of our systems grows towards complex! #DevOps relies on −crossfunctional teams to deal with complexity −automation to deal with order −continiuous improvement to grow requisite variety
161. Infrastructure as code @ Snapdeal Yagnik Architect Cloud & Infrastructure
162. ….once upon a time • microservices architecture • ~ 5000 nodes in AWS • devops handle provisioning, release and operations • no visibility
164. “If you know the enemy and know yourself, you need not fear the result of a hundred battles” - Sun Tzu, The Art of War
165. “If you know the enemy and know yourself, you need not fear the result of a hundred battles” - Sun Tzu, The Art of War
166. Infrastructure as code
167. ownership
168. service details
169. deployment
170. deployment
171. monitoring
172. fixed port
173. self heal
174. dependancies & security
175. dependancies & security
176. • ~120 components • ~350 subcomponents • explicit dependancy graph
178. OpenStack AWS Azure Infrastructure as Code Terraform SaltStack Application code Devops + Developer Happiness
179. OpenStack AWS Azure Infrastructure as Code Terraform SaltStack Application code Devops + Developer Happiness
180. TIL…
183. Building Infrastructure for Massive Open Online Courses Rizky Ariestiyansyah DevOpsDays Singapore 2016
184. HI! RIZKY ARIESTIYANSYAH HI! RIZKY ARIESTIYANSYAH MOZILLA TECH SPEAKER CTO of INDONESIAX.CO.ID @ariestiyansyah
185. What is MOOC Massive Student number can be 100.000+ Open Online Course Study any course, anywhere at any time Blended course Learning units in an academic subject, life skill.
186. Python for App - Minimal Setup - Hierarchical Module System - Available Libraries
187. LMS 1.Learning Management System for User 2.Forum Discussions 3.Exam 4.Assesment 5.Student Development
188. LMS 1.Learning Management System for User 2.Forum Discussions 3.Exam 4.Assesment 5.Student Development CMS 1.Course Management System 2.Instructor Dashboard
189. The Architecture APP
190. Focus: MySQL Company Process Username Port Function mysqld mysql 3306 Database • DB for: user profile, course enrollment, certificate • Authentication and Authorization data as well
191. Focus: MySQL Company Process Username Port Function mysqld mysql 3306 Database • DB for: user profile, course enrollment, certificate • Authentication and Authorization data as well
192. Focus: MongoDB Company • Easy to scale • Storing course data • NoSQL DB Backend Process Username Port Function mongodb mongo 27017 Database
193. Focus: MongoDB Company • Easy to scale • Storing course data • NoSQL DB Backend Process Username Port Function mongodb mongo 27017 Database
194. Focus: Memcached Company • High speed distributed key-value (KV) store for object caching. • Cache user session, course structure info, ect.. Process Username Port Function memcached memcache 11211 KV store
195. Focus: Memcached Company • High speed distributed key-value (KV) store for object caching. • Cache user session, course structure info, ect.. Process Username Port Function memcached memcache 11211 KV store
196. Focus: Nginx Company • Web Server. • Provided proxied. • Load Balanced access to static and dynamic assets. • Stateless and horizontally scalable. Process Username Worker Port Function nginx www-data 4 80 Course nginx www-data 4 8010 Course Manager
197. Focus: Nginx Company • Web Server. • Provided proxied. • Load Balanced access to static and dynamic assets. • Stateless and horizontally scalable. Process Username Worker Port Function nginx www-data 4 80 Course nginx www-data 4 8010 Course Manager
198. Focus: Gunicorn Company • Dynamic worker management. • Stateless and horizontally scalable. Process Username Worker Port Function gunicorn www-data 4 8000 Course gunicorn www-data 4 8010 Course Manager
199. Focus: Gunicorn Company • Dynamic worker management. • Stateless and horizontally scalable. Process Username Worker Port Function gunicorn www-data 4 8000 Course gunicorn www-data 4 8010 Course Manager
200. Focus: RabbitMQ Company • Large jobs are run through queue. • Coordinated by celery. • It is a software where queues can be defined, applications may connect to the queue and transfer a message onto it. Process Username Port Function beam.smp rabbitmq * :5672 Message Queue beam.smp rabbitmq 15672 Message Queue epmd rabbitmq 4369 Port Mapper
201. Focus: RabbitMQ Company • Large jobs are run through queue. • Coordinated by celery. • It is a software where queues can be defined, applications may connect to the queue and transfer a message onto it. Process Username Port Function beam.smp rabbitmq * :5672 Message Queue beam.smp rabbitmq 15672 Message Queue epmd rabbitmq 4369 Port Mapper
202. Thank You! Rizky Ariestiyansyah @ariestiyansyah oonlab.com See You!