[ZendCon EU 2013] The Evolution of DevOps

With the rapid adoption of new software development methodologies and infrastructure as a service (IaaS), we’ve seen a new role emerge, DevOps.

The DevOp is a master of this new domain: The Cloud. But just as quickly as we’ve seen DevOps emerge, we’re seeing the role evolve as advanced platforms and services enter on to the scene.

During this talk, we’ll explore how DevOps is evolving, what that means to you, and your role in the ever changing landscape of development and the cloud.

Transcript

1. The Evolution of DevOps Where Developers Meet the Cloud
2. •Community Engineer at Engine Yard •Author of Zend PHP 5 Certification Study Guide, Sitepoints PHP Anthology: 101 Essential Tips, Tricks & Hacks & PHP Master: Write Cutting Edge Code •A contributor to Zend Framework 1 & 2, phpdocs, and PHP internals • @dshafik Davey Shafik
3. Apologies For the Accent(s) There many be 2 or 3…
4. Community
5. Community Engineer PHPMentoring.org #phpmentoring on irc.freenode.net
6. Community Engineer PHPWomen.org #phpwomen on irc.freenode.net
7. Community Engineer #phpc on irc.freenode.net
8. What do you do for a living?
9. What do you do for a living? I build the internet
10. We don’t build web pages anymore
11. We don’t build web pages anymore We build infrastructures
12. Scriptable Hardware
13. Scriptable Hardware Scriptable “Hardware”
14. Scriptable Hardware Sysadmin roles are changing Scriptable “Hardware”
15. Scriptable Hardware Sysadmin roles are changing Developer roles are changing Scriptable “Hardware”
17. Developers
18. Developers Application Features
19. Developers Application Features Application Architecture
20. Developers Sysadmins Application Features Application Architecture
21. Developers Sysadmins Application Features Application Architecture System Software
22. Developers Sysadmins Application Features Application Architecture System Software OS Level
23. Developers Sysadmins Application Features Application Architecture System Software OS Level Deployment
24. Developers Sysadmins Application Features Application Architecture System Software OS Level DevOps Deployment
25. DevOps Starts at 127.0.0.1
26. DevOps Starts at 127.0.0.1 Where Developers Are In Control
27. Vagrant vagrantup.com
28. PuPHPet PuPHPet.com
30. Deploy to Production
31. Deploy to Production Suddenly: You’re responsible for production!
32. Congratulations! You’re now a DevOp!
33. Deployment
34. Deployment It’s more than pushing bits to boxes
35. chef puppet apache aws azure bash bower cassandra composer config ec2 elastic ip elb fabric grunt ha haproxy heartbeat ini javascript json linux mariadb memcached mongodb mysql nginx node.js openstack oracle pear phing php php-fpm postgresql python redis riak ruby sass shell squid storage terramark vagrant varnish yaml pecl
36. chef puppet apache aws azure bash bower cassandra composer config ec2 elastic ip elb fabric grunt ha haproxy heartbeat ini javascript json linux mariadb memcached mongodb mysql nginx node.js openstack oracle pear phing php php-fpm postgresql python redis riak ruby sass shell squid storage terramark vagrant varnish yaml pecl
37. chef puppet apache aws azure bash bower cassandra composer config ec2 elastic ip elb fabric grunt ha haproxy heartbeat ini javascript json linux mariadb memcached mongodb mysql nginx node.js openstack oracle pear phing php php-fpm postgresql python redis riak ruby sass shell squid storage terramark vagrant varnish yaml pecl Both happen to be written in Ruby
38. DevOps are Polyglot
39. DevOps are Polyglot By Necessity
40. Rise of the “As A Service” *aaS
41. Application
42. Platform Application
43. Platform Varnish Application
44. Platform Nginx Varnish Application
45. Platform Nginx PHP Ruby/Rails Node.js Varnish Application
46. Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB MySQL Riak Cassandra Application
47. Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
48. Infrastructure Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
49. Infrastructure Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
50. Infrastructure Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
51. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
52. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
53. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
54. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
55. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
56. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
57. Infrastructure As A Service (IaaS) ! Application Platform
58. Application Platform Platform Application Infrastructure As A Service (IaaS) !
59. Application Platform Platform Application Infrastructure As A Service (IaaS) !
60. Application
61. Application As A Service
62. Application As A Service AKA: Service Oriented Architecture (SOA)
63. Service-oriented architecture (SOA) is […] based on discrete pieces of software providing application functionality as services to other applications. […] It is independent of any vendor, product or technology. Source: Wikipedia! (Emphasis Mine)
64. SOA = APIs
65. APIs = REST
66. Separation of Concerns (SoC)
67. Separation of concerns (SoC) is a design principle for separating a computer program into distinct sections […]. A concern is a set of information that affects the code of a computer program.! ! The value of separation of concerns is simplifying development and maintenance of computer programs. When concerns are well separated, individual sections can be developed and updated independently. Of especial value is the ability to later improve or modify one section of code without having to know the details of other sections, and without having to make corresponding changes to those sections. Source: Wikipedia! (Emphasis Mine)
68. SoC = Encapsulating Processes
69. Application Message Queues Telephony E-Mail E-Commerce Logging Translation
70. Application Authentication
73. Associates Associates
74. Associates Associates Search Search
75. Associates Associates Search Search Deals Deals
76. Associates Associates Search Search Deals Deals Recommendations Recommendations
77. Associates Associates Search Search Deals Deals Recommendations Recommendations Account Account
78. Associates Associates Search Search Deals Deals Recommendations Recommendations Account Account Wish List Wish List
79. Associates Associates Search Search Deals Deals Recommendations Recommendations Account Account Cart Cart Wish List Wish List
80. Associates Associates Search Search Deals Deals Recommendations Recommendations Account Account Cart Cart Wish List Wish List Ads Ads
83. Content Service Content Service
84. Related Content Service Related Content Service Content Service Content Service
85. Related Content Service Related Content Service Content Service Content Service That's 22°C btw... That's 22°C btw...
86. Five Rules of SOA
87. Isolation • Object Oriented Design at the Systems Level • Process Encapsulation • System Isolation Services should be independent of each other
88. Isolation • Plan well defined Boundaries • Code • Resources • Features • Use separate resource for each service, e.g. database • Use separate URLs for each • e.g. accounts.api.example.org & reviews.api.example.org • This allows you to eventually use separate servers and eventually, separate clusters for each !42
89. github.com/Netflix/SimianArmy
90. Reusable • Use the same service to get the same data for different purposes • e.g. Getting an items rating, and getting it’s reviews Services should encapsulate a concern entirely
91. Reusable • Services can be used like libraries • Expose as Public APIs !45
92. Interoperable Services should work with each other • Loosely couples integration from implementation • Allows mixing technology to meet your needs
93. Interoperable • Similar data should be consistent • Similar actions should be consistent • HTTP + JSON are supported everywhere !47
94. Scalable Services Allow for Easier Scaling • Independently scale each service • Scale teams
95. Scalable • Separated Resources • Similar to sharding your database !49
96. Robust Cache Everything • Loosely Coupled • Refactor without impacting clients
97. Robust • Consistent and informative errors • Treat all services the same as third-party • Trust nothing • Assume problems • Cache Everything !51
98. Performance All that extra HTTP traffic isn’t cheap
99. Distributed Resources Horizontal Scaling is Easier
100. Parallel HTTP Requests pecl/http or curl_multi_*
101. SPDY/HTTP 2.0
102. Thick Frontends Client-Side Applications Love SOA
103. Mobile is Easy
104. But I have Legacy Apps! (who doesn’t?)
105. Application Lifecycle
106. Application Lifecycle 80% Maintenance
107. Application Lifecycle 20% 80% Design, Building, QA, etc Maintenance
108. Maintaining Code
109. Maintaining Code 60% New Features
110. Maintaining Code 17% 60% Bug Fixes New Features
111. Maintaining Code 23% 17% 60% Other (QA, Docs, Deployment) Bug Fixes New Features
112. BULLET • Bullet point • Bullet point • Bullet point BULLET • Bullet point • Bullet point • Bullet point BULLET • Bullet point • Bullet point • Bullet point BULLET • Bullet point • Bullet point • Bullet point Reuse (What you want to) Refactor (What you can) Rewrite (What you need to) The 3 R’s of Software Development
113. Reuse • Models • Business Logic • Libraries • Data sources • Almost anything but your views Pros: Quickly piggy-back off your current code. Cons: Will likely lead to REST-like (or RESTy/RESTful) APIs that are RPC in reality.
114. Reuse: Best Practices for APIs • Design your API first • APIs are like URIs: They should be permanent • Don’t let bad legacy decision drive new development Do: Pretend your API is a re-write. Create the best API you can for your application. Then implement it using legacy code any way you can. Even if it sucks. Don’t: Make compromises based on old code. Instead, plan to later refactor the old code.
115. Refactor • Two types of refactoring: • Refactor your app to use your services instead of legacy code. Then make the APIs better by: • Refactor the underlying implementation of your APIs to migrate away from legacy code Do: Test. Unit Tests and Integration Tests. Don’t: change your API! If you designed it right during the Reuse phase, this should be easy!
116. Rewrite • Rewriting should be your last step • You now have separation of concerns, so you can re-write your application piece meal Do: Discard any and all bad legacy code. Treat it as a clean slate. Don’t: Rewrite just to rewrite! Legacy code isn’t inherently bad code. Focus on new features. Build them as services!
117. Step-By-Step 1. Design your API 2. Build by Reusing legacy code (Do whatever it takes!) 3. Release 4. Refactor Application to use API instead of Legacy code 5. Refactor (or Rewrite) API to not use Legacy code. 6. Reuse API in Legacy application Rewrite (or Reuse in an entirely new application!)
118. Where does that leave Developers?
119. Start Using VMs Vagrant is Awesome
120. Learn Chef or Puppet
121. Deep Dive into HTTP
122. Caching • E-Tags/If-None-Match • Last-Modified/If-Modified-Since • Cache-Control
123. Status Codes
124. Semantic Requests HTTP Requests with Meaning • HTTP Verbs (GET/POST/PUT/DELETE/OPTIONS/HEAD/PATCH)
 + • HTTP Status Codes (e.g. 405 Method Not Allowed)
125. Idempotent Requests Doing the same thing multiple times always has the same result • Easy To Test!
126. You Should Check it Out
127. Focus on Features Build Cool Stuff Leave as much work to others as you want
128. What about Sysadmins?
129. What about Sysadmins? Time to learn some Ruby…
130. ` Feedback: https://joind.in/9916 Twitter: @dshafik Email: davey@engineyard.com ! Slides: http://daveyshafik.com/slides