Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ignites - DevOpsDays Singapore 2016

Ignites - DevOpsDays Singapore 2016

You have five minutes to talk on a subject accompanied by 20 slides, for 15 seconds each, automatically advanced. What will you say?

Clemri Steyn - 7 Habits of successful DevOps
Benjamin Henshall - Are we bridging the cultural divide that is necessary for DevOps to succeed
Ion Mudreac - Beyond DevOps ?
Angad Singh - Devops and Standards
Stefan Streichsbier - Security at DevOps Speed
Seth Vargo - 10 Tips to Create a Toxic Tech Team
Clemens Perz - Can't keep it simple, stupid!
Yagnik Khanna - Infrastructure as code at Snapdeal
Rizky Ariestiyansyah - Building Infrastructure for MOOC

Cc6ffa01992b5fa13e1bb5091a202b77?s=128

DevOpsDays Singapore

October 08, 2016
Tweet

More Decks by DevOpsDays Singapore

Other Decks in Technology

Transcript

  1. None
  2. None
  3. None
  4. None
  5. Our 7 Habits of DevOps Success
 
 Lessons learnt from

    Microsoft’s journey to DevOps Clemri Steyn DevOps and Application Lifecycle Product Marketing lead Visual Studio Product Marketing
  6. W H Y D E V O P S ?

    O U R D E V O P S J O U R N E Y
  7. W H AT I S D E V O P

    S ? O U R D E V O P S J O U R N E Y It’s Development and
 Operations collaboration It’s a job title It’s automation It means faster and smaller releases
  8. A C O N V E R G I N

    G 
 O F L I F E C YC L E S O U R D E V O P S J O U R N E Y DEVELOPMENT DEVELOP+TEST PLAN+TRACK OPERATIONS MONITOR + LEARN RELEASE
  9. T H E D E V O P S C

    O N V E R S AT I O N O U R D E V O P S J O U R N E Y
  10. O U R D E V O P S H

    A B I T S 
 A N D P R A C T I C E S O U R D E V O P S J O U R N E Y FLOW OF CUSTOMER VALUE AUTONOMY
 and
 ALIGNMENT BACKLOG improved by LEARNING EVIDENCE gathered in PRODUCTION MANAGED TECHNICAL DEBT PRODUCTION FIRST MINDSET INFRA is a FLEXIBLE RESOURCE
  11. F LO W O F 
 C U S TO

    M E R VA LU E O U R D E V O P S J O U R N E Y
  12. C O N T I N U O U S

    I N T E G R AT I O N 
 C O N T I N U O U S D E P LOY M E N T R E L E A S E M A N A G E M E N T AU TO M AT E D T E S T I N G O U R P R A C T I C E S F O R I M P R O V I N G 
 F LO W O F C U S TO M E R VA LU E O U R D E V O P S J O U R N E Y
  13. T E A M AU TO N O M Y

    
 A N D A L I G N M E N T O U R D E V O P S J O U R N E Y
  14. O U R P R A C T I C

    E S F O R I N C R E A S I N G 
 A L I G N M E N T A N D AU TO N O M Y O U R D E V O P S J O U R N E Y S E L F - M A N A G I N G T E A M S PL ANNING CHATS F E AT U R E C R E W S SPRINT RITUALS SCALED AGILE TE AM ROOMS
  15. B A C K LO G I M P R

    O V E D 
 BY L E A R N I N G O U R D E V O P S J O U R N E Y
  16. O U R P R A C T I C

    E S F O R C O N T I N U O U S I M P R O V E M E N T O U R D E V O P S J O U R N E Y U S E R T E L E M E T R Y U S A G E M O N I TO R I N G S TA K E H O L D E R F E E D B A C K 
 A / B T E S T I N G I N P R O D U C T I O N
  17. E V I D E N C E G AT

    H E R E D 
 I N P R O D U C T I O N O U R D E V O P S J O U R N E Y
  18. O U R P R A C T I C

    E S F O R 
 G AT H E R I N G E V I D E N C E O U R D E V O P S J O U R N E Y T E S T I N G I N P R O D U C T I O N STAKEHOLDER FEEDBACK U S A G E M O N I TO R I N G U S E R T E L E M E T R Y F E AT U R E F L A G S
  19. M A N A G E D 
 T E

    C H N I C A L D E BT O U R D E V O P S J O U R N E Y
  20. O U R P R A C T I C

    E S F O R 
 M A N A G I N G T E C H N I C A L D E BT O U R D E V O P S J O U R N E Y C O D E M E T R I C S T E S T AU TO M AT I O N P E E R C O D E R E V I E W S C O N T I N U O U S I N T E G R AT I O N C O N T I N U O U S U N I T T E S T I N G
  21. P R O D U C T I O N

    - F I R S T M I N D S E T O U R D E V O P S J O U R N E Y
  22. O U R P R A C T I C

    E S F O R A P R O D U C T I O N - F I R S T M I N D S E T O U R D E V O P S J O U R N E Y A P P L I C AT I O N P E R F O R M A N C E M A N A G E M E N T C O N F I G U R AT I O N M A N A G E M E N T I N F R A S T R U C T U R E A S C O D E AU TO M AT E D R E C O V E R Y C O N T I N U O U S D E L I V E R Y R E L E A S E M A N A G E M E N T
  23. I N F R A S T R U C

    T U R E I S A F L E X I B L E R E S O U R C E O U R D E V O P S J O U R N E Y
  24. O U R P R A C T I C

    E S F O R F L E X I B L E I N F R A S T R U C T U R E O U R D E V O P S J O U R N E Y I N F R A S T R U C T U R E A S C O D E D E V E LO P E R S A N D B OX I N G C LO U D D E V / T E S T L A B S C O N TA I N E R I Z AT I O N M I C R O S E R V I C E S AU TO S C A L I N G FA I LO V E R
  25. OLD WORLD Focus on planning Compete, not collaborate Static hierarchies

    Individual productivity Efficiency of process Assumptions, not data NEW WORLD Focus on delivering Collaborate to win Fluid and flexible teams Collective value creation Effectiveness of outcomes Experiment, learn and respond O U R S H I F T TO D E V O P S
  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None
  38. None
  39. None
  40. None
  41. None
  42. None
  43. None
  44. None
  45. None
  46. None
  47. None
  48. None
  49. None
  50. None
  51. Beyond DevOps ? What’s NEXT

  52. The Method We are what we repeatedly do. Excellence, then,

    is not an act, but a habit. ~ Aristotle
  53. Old School Stable requirements Known, mature technology No new undertaking

    Done this before Plan Require ment Develop Test Deploy Operate Functional Test System Test Performance Test Reliability Test Usability Test Business need à 3 years à application
  54. Complex projects ? Use Agile

  55. Agile is Designed to deal with • Ziv’s Law -

    specifications will never be fully understood. • Humphrey’s law - the user will never know what they want until after the system is in production (maybe not even then) • Wegner’s lemma - an interactive system can never be fully specified nor can it ever be fully tested. • Langdon’s lemma - software evolves more rapidly as it approaches chaotic regions (taking care not to spill over into chaos)
  56. Agile Timeline 1992 - Crystal Methods 1993 – Refactoring 1995

    – Scrum 1995 - Pair Development 1999 - Extreme Programming 1999 - The Pragmatic Programmer 2002 - Test Driven Development 1997 - Feature Driven Development 1999- Adaptive Software Development 2003 – Lean Software Development
  57. The Agile Manifesto In February 2001, the Manifesto for Agile

    Software Development was published. Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan While there is value in the items on the right, the value of the items on the left is more.
  58. Agile Business Design Development Ops Support Plan Req. Develop Operate

    Deploy P L A N R E Q . D E V . T E S T P L A N R E Q . D E V . T E S T P L A N R E Q . D E V . T E S T Delay Delay
  59. Beyond Agile Manifesto Even a great manifesto needs to evolve:

    • Team vision and discipline over Individuals and interactions over processes and tools • Validated learning over Working software over comprehensive documentation • Customer discovery over Customer collaboration over contract negotiation • Initiating change over Responding to change over following a plan
  60. DevOps Timeline 2008 - Google group called “Agile System Administration”

    June 2009 - 10+ Deploys per Day – Presentation by Flickr 2009: #DevOps hashtag born on Twitter 2010 First US DevOpsDays in Mountain View, CA
  61. DevOps Business Design Development Ops Support Plan Req. Develop Operate

    Deploy Deploy to production Continuous Planning Continuous Experimentation Continuous Integration Continuous Test Manual Test Continuous Deploy Deploy to test and staging (production-like) Unit Test Function Test Performance Test System Test Reliability Test
  62. DevOps Metrics Time to Delivery Deployment Frequency Change Volume Success

    Rate MTTR (Mean Time to Recovery)
  63. Innovation Management is in Crises

  64. Beyond DevOps Few teams went beyond classical DevOps using the

    full power of the cloud, full-stack engineering, micro-services, containerization where new term appear: • NoOps • DistributedOps • BizDevOps,
  65. Beyond DevOps Development Ops Support Develop Operate Deploy Continuous Planning

    Continuous Experimentation Continuous Integration Continuous Test Continuous Deploy Continuous Operation Deploy to staging and production Unit Test Function Test Performance Test System Test Reliability Test
  66. Capital vs Innovation 
 Venture capital invested in the valley

    15 Bilion USD every year.
 SP 500 companies are expected only in 2016 to distribute into share holder accounts 1,000,000,000,000,000,000 USD.
  67. Pace of Innovation vs Adoption

  68. Disruption In 80s startup where looking for the Big company’s

    that was poorly run. From mid 90s Startup main attack area was Big corporations well managed In present even the startup’s as old as 5 year already expecting to be disrupted by new comers.

  69. What can we expect Next ? Startup era mentality in

    enterprise (The Lean Startup) Enterprise will start decoupling innovation (The Innovator's dilemma) Blurred line between Business , Development and Operation (One Team) Competition around the speed of innovation (shift main focus from sustaining innovation to disruptive innovation) Disruption at the core of short cycle of innovation (Disrupt older disruption)
  70. We are hiring Email: ion.mudreac@sc.com Linkedin: https://sg.linkedin.com/in/mudrii

  71. None
  72. None
  73. Why standards matter Angad Singh @angadsg

  74. None
  75. DevOps Architecting (micro)services Infrastructure Automation Developer Productivity

  76. DevOps Debugging all layers Incident Management Cost Management

  77. None
  78. None
  79. Shared Libraries aka Reusability E.g. Redis/MySQL/xyz connection manager In memory

    caching Image transformations
  80. Security E.g. Consistent Firewall rules Standard input sanitization Centralized Alerting

  81. Developer Productivity No surprises when navigating the system e.g. All

    services log to /var/log One deploy toolchain to rule them all
  82. Cost Optimization Engineering cost > Infrastructure cost Better Standards =

    Lesser Engineers
  83. CI/CD Standard pathway for all services Unit Tests Integration Tests

  84. Engineering Onboarding Start early, start better. Raise the bar with

    new hires.
  85. Consistent Naming Service name = Docker name = Repo name

    = monitoring namespace = load balancer name = CI name
  86. Standard Provisioning Ansible/Chef etc. Docker - use and throw.

  87. Standard Configuration Consul, Zookeeper etc. Hot reloadable configuration Deploy faster,

    fix faster
  88. DevOps as a Shared Bus Identify patterns across all services

    Easier to handle complexity of microservices Ability to debug at any level
  89. DevOps as a Shared Bus Small teams with more context

    and communication are better than large teams with no communication Work closely with developers
  90. Enforce Standards But with logic and common sense. (and some

    love, for our developers)
  91. Its all about Give and Take. (Take sudo, Give advice)

  92. None
  93. None
  94. None
  95. Security at DevOps Speed Stefan Streichsbier CTO Vantage Point Founder

    DevSecOps Singapore
  96. What is AppSec?

  97. Why does AppSec == Pain?

  98. 
 
 
 Pentesters after turning a report in...

  99. Security

  100. 
 
 
 Meanwhile outside the security camp
 ...

  101. The frequency of releases over time 0 30 60 90

    120 2005 2010 2015 2020 Releases per app per year Towards CD From Waterfall The frequency increased
  102. 102 So many releases?!

  103. Security DevOps

  104. 104 Agile + DevOps + Security = DevSecOps

  105. Step 1: Security as part of Agile

  106. 1-4 Weeks 24 hours Develop Test Design Plan Output Shippable

    Increment Product Backlog Sprint Backlog Let’s look at SCRUM Start with understanding the process
  107. 1-4 Weeks 24 hours Develop Test Design Plan Output Shippable

    Increment Product Backlog Sprint Backlog Secure SCRUM Security Training Security Requirements Security Activities Threat Modelling Design Review Pairing Manual Security Tests Automatic Security Tests Security Feature Demo Security Retrospective Security Acceptance Criteria
  108. (Security) User Stories

  109. (Security) Unit Tests

  110. 0 25 50 75 100 Sprint 1 Sprint 2 Sprint

    3 Sprint 4 Sprint 5 Sprint 6 % Remaining Security work % App Robustness, Security Skills Security Debt Burndown
  111. Step 2: DevSecOps

  112. Vulnerability Repository • Security Unit Tests • SAST • SCA

    • DAST • IAST • VA • Security as Code • RASP • NG WAF • Red Team • GOPT • Actual Attackers • Sec Requirements • Design Review • Threat Modelling AppSec Pipeline
  113. Instead of this ...

  114. ...Let’s do this...

  115. None
  116. None
  117. TEN TIPS to create a TOXIC CULTURE . . .

    . . . . . . . . . . . . .
  118. DISCOURAGE SALARY DISCUSSIONS

  119. DISCOURAGE SALARY DISCUSSIONS

  120. HAVE ABSTRACT VALUES & EXPECTATIONS

  121. CREATE SCAPEGOATS

  122. CREATE SCAPEGOATS

  123. ONLY HIRE CRONIES

  124. ONLY HIRE CRONIES

  125. PLAY FAVORITES

  126. OFFER UNLIMITED VACATION

  127. OFFER UNLIMITED VACATION

  128. OFFER UNLIMITED VACATION

  129. DESTROY WORK-LIFE BALANCE

  130. ELIMINATE LOYALTY

  131. ELIMINATE LOYALTY

  132. TREAT HUMANS AS DATA POINTS

  133. TREAT HUMANS AS DATA POINTS

  134. MAKE FAKE PROMISES

  135. MAKE FAKE PROMISES

  136. THANK YOU! @sethvargo . . . . . . .

    . . . . .
  137. None
  138. None
  139. Wanna keep it simple? Clemens Perz

  140. Wanna keep it simple. Can?

  141. How much simple is left?

  142. 1995: The era of „simple“ 1995 # Internet: Easy to

    build websites # Mostly static content # Dynamic content on the way # Rudimental CSS # Deploy via tar -xzvf :)
  143. 2005: And then there was 2.0 1995 2005 # Internet:

    Web 2.0 # Everything must be dynamic # First wave of API's # CSS 3.0 kicks off # Complex Web Applications on the way # Database backends # Frontend servers # Load balancers # Deployments get complex, too
  144. 2015: The Big Everything 1995 2005 2015 # Internet: What

    Web are we at? # Everything is microsliced # Meshed via tons of RestAPI's # Applications are run on Clouds # Workloads virtualized # Applications containerized # How the hell do we manage all this?
  145. Complification! 1995 2005 2015 # The number of complex/complicated #

    questions to answer # problems to fix # features to implement # incidents to handle # decisions to make
  146. Complification! 1995 2005 2015 # The number of complex/complicated #

    questions to answer # problems to fix # features to implement # incidents to handle # decisions to make
  147. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel

    Practice Simple/Obvious Best Practice
  148. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel

    Practice Simple/Obvious Best Practice
  149. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel

    Practice Simple/Obvious Best Practice
  150. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel

    Practice Simple/Obvious Best Practice
  151. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel

    Practice Simple/Obvious Best Practice
  152. Cynefin Framework Complex Emergent Practice Complicated Good Practice Chaotic Novel

    Practice Simple/Obvious Best Practice
  153. The variety of problem and solution.

  154. Where we stumble over it: # Customer support # Car

    repairs # Operations :D
  155. Is it clever or even possible to ignore the variety?

  156. Ashby's Law of Requisite Variety „Only variety absorbs variety“

  157. A simple system 0 0 1 2 3 4

  158. Can't keep it simple, stupid! #The variety of our systems

    grows towards complex! #DevOps relies on −crossfunctional teams to deal with complexity −automation to deal with order −continiuous improvement to grow requisite variety
  159. None
  160. None
  161. Infrastructure as code @ Snapdeal Yagnik Architect Cloud & Infrastructure

  162. ….once upon a time • microservices architecture • ~ 5000

    nodes in AWS • devops handle provisioning, release and operations • no visibility
  163. None
  164. “If you know the enemy and know yourself, you need

    not fear the result of a hundred battles” - Sun Tzu, The Art of War
  165. “If you know the enemy and know yourself, you need

    not fear the result of a hundred battles” - Sun Tzu, The Art of War
  166. Infrastructure as code

  167. ownership

  168. service details

  169. deployment

  170. deployment

  171. monitoring

  172. fixed port

  173. self heal

  174. dependancies & security

  175. dependancies & security

  176. • ~120 components • ~350 subcomponents • explicit dependancy graph

  177. None
  178. OpenStack AWS Azure Infrastructure as Code Terraform SaltStack Application code

    Devops + Developer Happiness
  179. OpenStack AWS Azure Infrastructure as Code Terraform SaltStack Application code

    Devops + Developer Happiness
  180. TIL…

  181. None
  182. None
  183. Building Infrastructure for Massive Open Online Courses Rizky Ariestiyansyah DevOpsDays

    Singapore 2016
  184. HI! RIZKY ARIESTIYANSYAH HI! RIZKY ARIESTIYANSYAH MOZILLA TECH SPEAKER CTO

    of INDONESIAX.CO.ID @ariestiyansyah
  185. What is MOOC Massive Student number can be 100.000+ Open

    Online Course Study any course, anywhere at any time Blended course Learning units in an academic subject, life skill.
  186. Python for App - Minimal Setup - Hierarchical Module System

    - Available Libraries
  187. LMS 1.Learning Management System for User 2.Forum Discussions 3.Exam 4.Assesment

    5.Student Development
  188. LMS 1.Learning Management System for User 2.Forum Discussions 3.Exam 4.Assesment

    5.Student Development CMS 1.Course Management System 2.Instructor Dashboard
  189. The Architecture APP

  190. Focus: MySQL Company Process Username Port Function mysqld mysql 3306

    Database • DB for: user profile, course enrollment, certificate • Authentication and Authorization data as well
  191. Focus: MySQL Company Process Username Port Function mysqld mysql 3306

    Database • DB for: user profile, course enrollment, certificate • Authentication and Authorization data as well
  192. Focus: MongoDB Company • Easy to scale • Storing course

    data • NoSQL DB Backend Process Username Port Function mongodb mongo 27017 Database
  193. Focus: MongoDB Company • Easy to scale • Storing course

    data • NoSQL DB Backend Process Username Port Function mongodb mongo 27017 Database
  194. Focus: Memcached Company • High speed distributed key-value (KV) store

    for object caching. • Cache user session, course structure info, ect.. Process Username Port Function memcached memcache 11211 KV store
  195. Focus: Memcached Company • High speed distributed key-value (KV) store

    for object caching. • Cache user session, course structure info, ect.. Process Username Port Function memcached memcache 11211 KV store
  196. Focus: Nginx Company • Web Server. • Provided proxied. •

    Load Balanced access to static and dynamic assets. • Stateless and horizontally scalable. Process Username Worker Port Function nginx www-data 4 80 Course nginx www-data 4 8010 Course Manager
  197. Focus: Nginx Company • Web Server. • Provided proxied. •

    Load Balanced access to static and dynamic assets. • Stateless and horizontally scalable. Process Username Worker Port Function nginx www-data 4 80 Course nginx www-data 4 8010 Course Manager
  198. Focus: Gunicorn Company • Dynamic worker management. • Stateless and

    horizontally scalable. Process Username Worker Port Function gunicorn www-data 4 8000 Course gunicorn www-data 4 8010 Course Manager
  199. Focus: Gunicorn Company • Dynamic worker management. • Stateless and

    horizontally scalable. Process Username Worker Port Function gunicorn www-data 4 8000 Course gunicorn www-data 4 8010 Course Manager
  200. Focus: RabbitMQ Company • Large jobs are run through queue.

    • Coordinated by celery. • It is a software where queues can be defined, applications may connect to the queue and transfer a message onto it. Process Username Port Function beam.smp rabbitmq * :5672 Message Queue beam.smp rabbitmq 15672 Message Queue epmd rabbitmq 4369 Port Mapper
  201. Focus: RabbitMQ Company • Large jobs are run through queue.

    • Coordinated by celery. • It is a software where queues can be defined, applications may connect to the queue and transfer a message onto it. Process Username Port Function beam.smp rabbitmq * :5672 Message Queue beam.smp rabbitmq 15672 Message Queue epmd rabbitmq 4369 Port Mapper
  202. Thank You! Rizky Ariestiyansyah @ariestiyansyah oonlab.com See You!

  203. None