The release process at Tuenti

When a company grows up so fast like Tuenti did, the development and release process needs to evolve. From a manual, unstable, costly and prone to failures process, to a fully automated, fast, reliable and almost unattended one, being able to deploy or integrate code just clicking on a single button.

Transcript

1. The release process at Tuenti Víctor García (@V1KT) October 19th, 2013 http://blog.tuenti.com/dev
2. Tuenti 4 years ago, start-up way of life  Long releases, we woke up at 6am!  Many people involved  Deployment not reliable, unstable  Brittle tests  Nobody trusted the release  Bugfixing in live  Redeploying the bugfix  “Free commit bar”  Lot of effort  Risky  No repetitive  Stressful  Jenkins taking ages  Etc.
3. DevOps culture  January 2012: DevOps team creation  We are:  Developers with sysadmins skills  Sysadmins with developers skills  Release management background  The ideas  Build a platform to allow developers to self-service envs and services  Provide a toolchain so developers can build, test, deploy and run systems  Coach teams to move to this model
4. Developers start coding
5. Developers start coding
6. Developers start coding  Tuenti-in-a-box  Provides a self service full  development environment  Private  No shared resources  Vagrant + Puppet + VirtualBox  user@mylaptop:~# vagrant up  For us, better than classical server virtualization
7. Developers start coding
8. Jenkins tests  When  While developing  Before integrating  How:  Early stages, fast & short run: ~18000 tests in 13 min  Latests stages, slow & full run: ~29000 tests in 90 min  Release stage, fast & full run: ~29000 tests in 42 min  What tests:  Unit, integration, system, browser, JavaScript,...
9. Developers branches integration
10. Developers branches integration
11. Developers branches integration How do I integrate my branch??
12. Developers branches integration l Click a button  and go to play foosball!!
13. Flow l Flow does everything
14. Flow: integration pull request  Enqueues pull requests sent by developers  To process a pull request it does:  Create a temporary branch with the merge  Tell Jenkins to test it  If no tests failing, do the real merge  Otherwise, notify the developer and QA  Transition Jira tickets  Link Jira tickets
16. Release time!!
17. Release time!! Let's do a release, I'm getting in troubles...
18. Release time!!  Click a button  and go to play foosball!!
19. Flow: release management  Click a Jira button, and Flow does:  Creates SCM release branch  Build & Deploy code to preproduction  Tell Jenkins to test that branch  Notify everyone involved  Transition Jira tickets  Link Jira tickets  QA tests a little bit  Release to staging  The users tests for us  Release go live!  Deployment to hundreds of servers takes less than a minute
20. It's been a success  Now:  Almost no manual tasks  Developers throughput increased  Ageing of finished code not released greatly reduced  Fast reaction to product changes or bugs  Information transparency  How  Incremental changes, MVP (minimum viable product)  Evolved gradually  Did not involve rupture in processes
21. Some figures and notes  3.21 production deployments per day  67.64 preproduction deployments per day  7.19 pull requests per day  Releases takes about 1 hour  Almost unattended  Safe, reliable and revertible
22. Conclusion l You can change engineers mind  to adopt new methodologies
23. Questions? l Víctor García (@V1KT) http://blog.tuenti.com/dev