Hacking AWS OpsWorks

At Sport Ngin we use AWS OpsWorks heavily. We running 25 applications across our platform, 50 different OpsWorks stacks between our staging and production accounts. We've been using OpsWorks basically since it was announced 2 years ago. We built a command line utility(http://github.com/sportngin/opsicle) to help us along the way. I could give a presentation about OpsWorks in general and the evolution of how we've use it. If there's time start talking about the tradeoffs of using OpsWorks vs rolling your own chef server/provisioning setup.

Transcript

1. Hacking AWS OpsWorks
   Andy Fleener
   @andyfleener
   

   View Slide

2. Sport Ngin Platform
   January 2013

   March 2013

   June 2014

   February 11, 2015
   8 applications running on 2
   different “Paas”
   First new application launches on
   OpsWorks
   Decision is made to move all
   applications to OpsWorks
   All 25 web applications running on
   OpsWorks
   

   View Slide

3. Why Move
   Running on 3 different platforms was hard
   None of the 3 fully met our needs
   Consolidating allowed us to become experts in a 

   single platform
   

   View Slide

4. Why OpsWorks
   High level of customizability
   Did the low level work for us
   Stayed out of our way
   Good pricing model
   Stays fresh by releasing new features
   

   View Slide

5. What is OpsWorks
   

   View Slide

6. Rub Some DevOps
   Cloud
   DevOps Cloud
   Application Management
   On it!
   

   View Slide

7. No seriously WTF is it?
   

   View Slide

8. Application Manager
   Config Management - Chef/OpsWorks Lifecycle
   orchestration
   Automated Deployments - Via Chef’s deploy resource
   Application Stack definitions - Built-in or using custom
   defined stacks
   Resource Management - EIPs/EBS/ELBs
   

   View Slide

9. Primitives
   Stacks
   Layers
   Apps
   Instances
   

   View Slide

10. Chef/OpsWorks Agent
    Chef Solo/Chef Zero
    OpsWorks Agent
    Agent pings home to see if it needs to run Chef
    All life cycle events translate to a Chef run 

    with different run lists.
    

    View Slide

11. Life Cycle Events
    Setup
    Deploy
    Configure
    Undeploy
    Shutdown
    

    View Slide

12. Other Goodies
    EIPs
    EBS Volumes
    ELBs
    IAM User Management
    RDS Integration
    

    View Slide

13. Cool Story..
    

    View Slide

14. But will it blend?
    

    View Slide

15. The Good
    Everything is an API
    Direct integration with other Amazon tools
    Fast release cycle
    A bit of built in orchestration
    

    View Slide

16. The Bad
    User experience is OK-ish
    It’s not a typical Chef setup
    Auto Healing is a waste of a feature
    Not enough orchestration
    

    View Slide

17. Getting Started
    

    View Slide

18. Use built-in layers!
    

    View Slide

19. Questions to Ask
    Am I going to run this in production?
    Will this application have users?
    Do I need zero downtime deployments?
    Do I care about what software is running on my servers?
    

    View Slide

20. Don’t use built-in layers
    

    View Slide

21. Hybrid Approach
    

    View Slide

22. Custom Layers
    Reuse OpsWorks’ chef cookbooks
    Still have full control over the run list
    Replace parts that don’t meet your needs
    Use Layers as roles to attach run lists to layers
    Use custom security groups
    

    View Slide

23. Shipping all the bits
    

    View Slide

24. Updating Your Chef
    All Chef versions use a site-cookbooks pattern
    Berkshelf can help
    Watch out for indeterminate dependency resolution
    All the OpsWorks cookbooks are open source

    https://github.com/aws/opsworks-cookbooks
    

    View Slide

25. Deploying Your App
    What’s your deployment strategy?
    Built-in chef deploys with minimal downtime
    Work must be done to make them zero downtime
    

    View Slide

26. Zero Downtime Deploys
    Two reasonable techniques:
    - Zero downtime on each instance
    - Use orchestration to do rolling restarts
    

    View Slide

27. Leveling Up
    

    View Slide

28. Improving the UX
    Automation is key
    Building good abstractions
    CLI is faster
    MOAR Orchestration!!
    

    View Slide

29. Opsicle
    https://github.com/sportngin/opsicle
    

    View Slide

30. What does it do?
    Deployments
    Chef Cookbook updates
    Arbitrary Chef runs
    Monitor deployment activity
    ssh / ssh key management
    Moves management closer to the code
    

    View Slide

31. Primary Goals
    Improve the developer experience
    Build good layers of abstraction
    Add more automation
    

    View Slide

32. Road Map
    More visibility into deployments
    Instance management (start/stop/create/delete)
    Stack creation/configuration
    

    View Slide

33. DEMO TIME!
    https://github.com/anfleene/opsworks_rails_demo
    https://github.com/anfleene/opsworks_rails_demo_cookbooks
    

    View Slide

34. Why not normal Chef?
    

    View Slide

35. Everything has tradeoffs
    

    View Slide

36. OpsWorks
    Easy to get started
    Built-in server management
    OpsWorks runs the backend service
    Removes some of the complex parts of Chef
    It’s only kind of Chef
    

    View Slide

37. Chef Server
    It’s actually Chef
    Cookbooks from the community will work if you set
    them up right
    More freedom about how to run/maintain
    Bigger investment into ensuring you have a working
    Chef server
    Possible single point of failure
    

    View Slide

38. What Do You All Think?
    

    View Slide