[ZendCon EU 2013] The Evolution of DevOps

Fee39f0c0ffb29d9ac21607ed188be6b?s=47 Davey Shafik
November 20, 2013

[ZendCon EU 2013] The Evolution of DevOps

With the rapid adoption of new software development methodologies and infrastructure as a service (IaaS), we’ve seen a new role emerge, DevOps.

The DevOp is a master of this new domain: The Cloud. But just as quickly as we’ve seen DevOps emerge, we’re seeing the role evolve as advanced platforms and services enter on to the scene.

During this talk, we’ll explore how DevOps is evolving, what that means to you, and your role in the ever changing landscape of development and the cloud.

Fee39f0c0ffb29d9ac21607ed188be6b?s=128

Davey Shafik

November 20, 2013
Tweet

Transcript

  1. The Evolution of DevOps Where Developers Meet the Cloud

  2. •Community Engineer at Engine Yard •Author of Zend PHP 5

    Certification Study Guide, Sitepoints PHP Anthology: 101 Essential Tips, Tricks & Hacks & PHP Master: Write Cutting Edge Code •A contributor to Zend Framework 1 & 2, phpdocs, and PHP internals • @dshafik Davey Shafik
  3. Apologies For the Accent(s) There many be 2 or 3…

  4. Community

  5. Community Engineer PHPMentoring.org #phpmentoring on irc.freenode.net

  6. Community Engineer PHPWomen.org #phpwomen on irc.freenode.net

  7. Community Engineer #phpc on irc.freenode.net

  8. What do you do for a living?

  9. What do you do for a living? I build the

    internet
  10. We don’t build web pages anymore

  11. We don’t build web pages anymore We build infrastructures

  12. Scriptable Hardware

  13. Scriptable Hardware Scriptable “Hardware”

  14. Scriptable Hardware Sysadmin roles are changing Scriptable “Hardware”

  15. Scriptable Hardware Sysadmin roles are changing Developer roles are changing

    Scriptable “Hardware”
  16. None
  17. Developers

  18. Developers Application Features

  19. Developers Application Features Application Architecture

  20. Developers Sysadmins Application Features Application Architecture

  21. Developers Sysadmins Application Features Application Architecture System Software

  22. Developers Sysadmins Application Features Application Architecture System Software OS Level

  23. Developers Sysadmins Application Features Application Architecture System Software OS Level

    Deployment
  24. Developers Sysadmins Application Features Application Architecture System Software OS Level

    DevOps Deployment
  25. DevOps Starts at 127.0.0.1

  26. DevOps Starts at 127.0.0.1 Where Developers Are In Control

  27. Vagrant vagrantup.com

  28. PuPHPet PuPHPet.com

  29. None
  30. Deploy to Production

  31. Deploy to Production Suddenly: You’re responsible for production!

  32. Congratulations! You’re now a DevOp!

  33. Deployment

  34. Deployment It’s more than pushing bits to boxes

  35. chef puppet apache aws azure bash bower cassandra composer config

    ec2 elastic ip elb fabric grunt ha haproxy heartbeat ini javascript json linux mariadb memcached mongodb mysql nginx node.js openstack oracle pear phing php php-fpm postgresql python redis riak ruby sass shell squid storage terramark vagrant varnish yaml pecl
  36. chef puppet apache aws azure bash bower cassandra composer config

    ec2 elastic ip elb fabric grunt ha haproxy heartbeat ini javascript json linux mariadb memcached mongodb mysql nginx node.js openstack oracle pear phing php php-fpm postgresql python redis riak ruby sass shell squid storage terramark vagrant varnish yaml pecl
  37. chef puppet apache aws azure bash bower cassandra composer config

    ec2 elastic ip elb fabric grunt ha haproxy heartbeat ini javascript json linux mariadb memcached mongodb mysql nginx node.js openstack oracle pear phing php php-fpm postgresql python redis riak ruby sass shell squid storage terramark vagrant varnish yaml pecl Both happen to be written in Ruby
  38. DevOps are Polyglot

  39. DevOps are Polyglot By Necessity

  40. Rise of the “As A Service” *aaS

  41. Application

  42. Platform Application

  43. Platform Varnish Application

  44. Platform Nginx Varnish Application

  45. Platform Nginx PHP Ruby/Rails Node.js Varnish Application

  46. Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB MySQL Riak

    Cassandra Application
  47. Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other: Memcached,

    Gearman, etc. MySQL Riak Cassandra Application
  48. Infrastructure Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other:

    Memcached, Gearman, etc. MySQL Riak Cassandra Application
  49. Infrastructure Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other:

    Memcached, Gearman, etc. MySQL Riak Cassandra Application
  50. Infrastructure Platform Nginx PHP Ruby/Rails Node.js Varnish PostgreSQL MongoDB Other:

    Memcached, Gearman, etc. MySQL Riak Cassandra Application
  51. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails

    Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
  52. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails

    Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
  53. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails

    Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
  54. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails

    Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
  55. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails

    Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
  56. Infrastructure As A Service (IaaS) ! Platform Nginx PHP Ruby/Rails

    Node.js Varnish PostgreSQL MongoDB Other: Memcached, Gearman, etc. MySQL Riak Cassandra Application
  57. Infrastructure As A Service (IaaS) ! Application Platform

  58. Application Platform Platform Application Infrastructure As A Service (IaaS) !

  59. Application Platform Platform Application Infrastructure As A Service (IaaS) !

  60. Application

  61. Application As A Service

  62. Application As A Service AKA: Service Oriented Architecture (SOA)

  63. Service-oriented architecture (SOA) is […] based on discrete pieces of

    software providing application functionality as services to other applications. […] It is independent of any vendor, product or technology. Source: Wikipedia! (Emphasis Mine)
  64. SOA = APIs

  65. APIs = REST

  66. Separation of Concerns (SoC)

  67. Separation of concerns (SoC) is a design principle for separating

    a computer program into distinct sections […]. A concern is a set of information that affects the code of a computer program.! ! The value of separation of concerns is simplifying development and maintenance of computer programs. When concerns are well separated, individual sections can be developed and updated independently. Of especial value is the ability to later improve or modify one section of code without having to know the details of other sections, and without having to make corresponding changes to those sections. Source: Wikipedia! (Emphasis Mine)
  68. SoC = Encapsulating Processes

  69. Application Message Queues Telephony E-Mail E-Commerce Logging Translation

  70. Application Authentication

  71. None
  72. None
  73. Associates Associates

  74. Associates Associates Search Search

  75. Associates Associates Search Search Deals Deals

  76. Associates Associates Search Search Deals Deals Recommendations Recommendations

  77. Associates Associates Search Search Deals Deals Recommendations Recommendations Account Account

  78. Associates Associates Search Search Deals Deals Recommendations Recommendations Account Account

    Wish List Wish List
  79. Associates Associates Search Search Deals Deals Recommendations Recommendations Account Account

    Cart Cart Wish List Wish List
  80. Associates Associates Search Search Deals Deals Recommendations Recommendations Account Account

    Cart Cart Wish List Wish List Ads Ads
  81. None
  82. None
  83. Content Service Content Service

  84. Related Content Service Related Content Service Content Service Content Service

  85. Related Content Service Related Content Service Content Service Content Service

    That's 22°C btw... That's 22°C btw...
  86. Five Rules of SOA

  87. Isolation • Object Oriented Design at the Systems Level •

    Process Encapsulation • System Isolation Services should be independent of each other
  88. Isolation • Plan well defined Boundaries • Code • Resources

    • Features • Use separate resource for each service, e.g. database • Use separate URLs for each • e.g. accounts.api.example.org & reviews.api.example.org • This allows you to eventually use separate servers and eventually, separate clusters for each !42
  89. github.com/Netflix/SimianArmy

  90. Reusable • Use the same service to get the same

    data for different purposes • e.g. Getting an items rating, and getting it’s reviews Services should encapsulate a concern entirely
  91. Reusable • Services can be used like libraries • Expose

    as Public APIs !45
  92. Interoperable Services should work with each other • Loosely couples

    integration from implementation • Allows mixing technology to meet your needs
  93. Interoperable • Similar data should be consistent • Similar actions

    should be consistent • HTTP + JSON are supported everywhere !47
  94. Scalable Services Allow for Easier Scaling • Independently scale each

    service • Scale teams
  95. Scalable • Separated Resources • Similar to sharding your database

    !49
  96. Robust Cache Everything • Loosely Coupled • Refactor without impacting

    clients
  97. Robust • Consistent and informative errors • Treat all services

    the same as third-party • Trust nothing • Assume problems • Cache Everything !51
  98. Performance All that extra HTTP traffic isn’t cheap

  99. Distributed Resources Horizontal Scaling is Easier

  100. Parallel HTTP Requests pecl/http or curl_multi_*

  101. SPDY/HTTP 2.0

  102. Thick Frontends Client-Side Applications Love SOA

  103. Mobile is Easy

  104. But I have Legacy Apps! (who doesn’t?)

  105. Application Lifecycle

  106. Application Lifecycle 80% Maintenance

  107. Application Lifecycle 20% 80% Design, Building, QA, etc Maintenance

  108. Maintaining Code

  109. Maintaining Code 60% New Features

  110. Maintaining Code 17% 60% Bug Fixes New Features

  111. Maintaining Code 23% 17% 60% Other (QA, Docs, Deployment) Bug

    Fixes New Features
  112. BULLET • Bullet point • Bullet point • Bullet point

    BULLET • Bullet point • Bullet point • Bullet point BULLET • Bullet point • Bullet point • Bullet point BULLET • Bullet point • Bullet point • Bullet point Reuse (What you want to) Refactor (What you can) Rewrite (What you need to) The 3 R’s of Software Development
  113. Reuse • Models • Business Logic • Libraries • Data

    sources • Almost anything but your views Pros: Quickly piggy-back off your current code. Cons: Will likely lead to REST-like (or RESTy/RESTful) APIs that are RPC in reality.
  114. Reuse: Best Practices for APIs • Design your API first

    • APIs are like URIs: They should be permanent • Don’t let bad legacy decision drive new development Do: Pretend your API is a re-write. Create the best API you can for your application. Then implement it using legacy code any way you can. Even if it sucks. Don’t: Make compromises based on old code. Instead, plan to later refactor the old code.
  115. Refactor • Two types of refactoring: • Refactor your app

    to use your services instead of legacy code. Then make the APIs better by: • Refactor the underlying implementation of your APIs to migrate away from legacy code Do: Test. Unit Tests and Integration Tests. Don’t: change your API! If you designed it right during the Reuse phase, this should be easy!
  116. Rewrite • Rewriting should be your last step • You

    now have separation of concerns, so you can re-write your application piece meal Do: Discard any and all bad legacy code. Treat it as a clean slate. Don’t: Rewrite just to rewrite! Legacy code isn’t inherently bad code. Focus on new features. Build them as services!
  117. Step-By-Step 1. Design your API 2. Build by Reusing legacy

    code (Do whatever it takes!) 3. Release 4. Refactor Application to use API instead of Legacy code 5. Refactor (or Rewrite) API to not use Legacy code. 6. Reuse API in Legacy application Rewrite (or Reuse in an entirely new application!)
  118. Where does that leave Developers?

  119. Start Using VMs Vagrant is Awesome

  120. Learn Chef or Puppet

  121. Deep Dive into HTTP

  122. Caching • E-Tags/If-None-Match • Last-Modified/If-Modified-Since • Cache-Control

  123. Status Codes

  124. Semantic Requests HTTP Requests with Meaning • HTTP Verbs (GET/POST/PUT/DELETE/OPTIONS/HEAD/PATCH)


    + • HTTP Status Codes (e.g. 405 Method Not Allowed)
  125. Idempotent Requests Doing the same thing multiple times always has

    the same result • Easy To Test!
  126. You Should Check it Out

  127. Focus on Features Build Cool Stuff Leave as much work

    to others as you want
  128. What about Sysadmins?

  129. What about Sysadmins? Time to learn some Ruby…

  130. ` Feedback: https://joind.in/9916 Twitter: @dshafik Email: davey@engineyard.com ! Slides: http://daveyshafik.com/slides