[2016.04 Meetup #1] [TALK #3] Vasco Pinho - How Unbabel went from Heroku to AWS and kept deploying every day

Vasco Pinho's talk slides: How Unbabel went from Heroku to AWS and kept deploying every day

All rights belong to Vasco Pinho.

1. How Unbabel went from Heroku to AWS and kept deploying every day
2. whoami @vascogpinho DevOps @ Unbabel pinho@unbabel.com
3. Team A bunch of developers Some researchers One “ops” person
4. What we were doing git push heroku master
5. And a little bit of ssh + who knows what
6. Problems with Heroku Hard to know what’s going on No control over base images Expensive
7. A mess in AWS No configuration management Unable to easily reproduce server state No monitoring Running experiments in production machines
8. Deploying culture All developers can deploy to their own dev machine and/or staging environment A couple of people with deploying permissions / “keys to the kingdom” to production
9. Cleaning the house Design a deployment architecture Automate infrastructure provisioning and deployment in a delivery pipeline
10. Deploying Single command + developer initiated Zero Downtime deploys Allow rollbacks “mid” deploy Immutable servers (cattle vs pets)
11. Delivery Pipeline
12. Blue-Green Deployment Load Balancer Blue Green
13. Blue-Green Deployment Boots new fleet for deploy When machines are ready allow for automated smoke tests + any manual checks (how does the new feature look?) Proceed with deploy (add / remove machines from ELB) or rollback (kill new machines) Whole thing ideally under 5, realistically under 10 minutes
14. How do we do it Deploy unbabel.push:staging unbabel.push:production
15. Wrapping up the migration Developers can initiate deploys without needing an “ops” person present Multiple deploys a day with no downtime involved Deploy time actually came down from ~15min in Heroku to around ~7min now At the time of the change a single t2.medium replaced around 25 Heroku worker machines
16. Monolithic vs Microservices Started with just one big app in Heroku Smartcheck (spellchecking, consistency, grammar, etc) built as microservices Annotation tools, cost analysis tools now also microservices
17. Who does what Only some developers are able to deploy main app to production Other microservices are developed and deployed usually a single developer (ownership!) Ops role is supporting base system changes for new requirements, optimising deployments, monitoring machines, setting up new environments, etc…
18. The end Questions?