The release process at Tuenti

When a company grows up so fast like Tuenti did, the development and release process needs to evolve. From a manual, unstable, costly and prone to failures process, to a fully automated, fast, reliable and almost unattended one, being able to deploy or integrate code just clicking on a single button.

Transcript

1. The release process at Tuenti
   Víctor García (@V1KT)
   October 19th, 2013
   http://blog.tuenti.com/dev
   

   View Slide

2. Tuenti 4 years ago, start-up way of life
    Long releases, we woke up at 6am!
   
   Many people involved
   
   Deployment not reliable, unstable
   
   Brittle tests
   
   Nobody trusted the release
   
   Bugfixing in live
   
   Redeploying the bugfix
   
   “Free commit bar”
   
   Lot of effort
   
   Risky
   
   No repetitive
   
   Stressful
   
   Jenkins taking ages
   
   Etc.
   

   View Slide

3. DevOps culture
    January 2012: DevOps team creation
   
   We are:
   
   Developers with sysadmins skills
   
   Sysadmins with developers skills
   
   Release management background
   
   The ideas
   
   Build a platform to allow developers to self-service envs and services
   
   Provide a toolchain so developers can build, test, deploy and run systems
   
   Coach teams to move to this model
   

   View Slide

4. Developers start coding
   

   View Slide

5. Developers start coding
   

   View Slide

6. Developers start coding
   
   Tuenti-in-a-box
   
   Provides a self service full
   
   development environment
   
   Private
   
   No shared resources
   
   Vagrant + Puppet + VirtualBox
   
   user@mylaptop:~# vagrant up
   
   For us, better than classical server virtualization
   

   View Slide

7. Developers start coding
   

   View Slide

8. Jenkins tests
   
   When
   
   While developing
   
   Before integrating
   
   How:
   
   Early stages, fast & short run: ~18000 tests in 13 min
   
   Latests stages, slow & full run: ~29000 tests in 90 min
   
   Release stage, fast & full run: ~29000 tests in 42 min
   
   What tests:
   
   Unit, integration, system, browser, JavaScript,...
   

   View Slide

9. Developers branches integration
   

   View Slide

10. Developers branches integration
    

    View Slide

11. Developers branches integration
    How do I integrate my branch??
    

    View Slide

12. Developers branches integration
    l Click a button
    
    and go to play foosball!!
    

    View Slide

13. Flow
    l Flow does everything
    

    View Slide

14. Flow: integration pull request
    
    Enqueues pull requests sent by developers
    
    To process a pull request it does:
    
    Create a temporary branch with the merge
    
    Tell Jenkins to test it
    
    If no tests failing, do the real merge
    
    Otherwise, notify the developer and QA
    
    Transition Jira tickets
    
    Link Jira tickets
    

    View Slide

15. View Slide

16. Release time!!
    

    View Slide

17. Release time!!
    Let's do a release, I'm getting in troubles...
    

    View Slide

18. Release time!!
    
    Click a button
    
    and go to play foosball!!
    

    View Slide

19. Flow: release management
    
    Click a Jira button, and Flow does:
    
    Creates SCM release branch
    
    Build & Deploy code to preproduction
    
    Tell Jenkins to test that branch
    
    Notify everyone involved
    
    Transition Jira tickets
    
    Link Jira tickets
    
    QA tests a little bit
    
    Release to staging
    
    The users tests for us
    
    Release go live!
    
    Deployment to hundreds of servers takes less than a minute
    

    View Slide

20. It's been a success
    
    Now:
    
    Almost no manual tasks
    
    Developers throughput increased
    
    Ageing of finished code not released greatly reduced
    
    Fast reaction to product changes or bugs
    
    Information transparency
    
    How
    
    Incremental changes, MVP (minimum viable product)
    
    Evolved gradually
    
    Did not involve rupture in processes
    

    View Slide

21. Some figures and notes
    
    3.21 production deployments per day
    
    67.64 preproduction deployments per day
    
    7.19 pull requests per day
    
    Releases takes about 1 hour
    
    Almost unattended
    
    Safe, reliable and revertible
    

    View Slide

22. Conclusion
    l You can change engineers mind
    
    to adopt new methodologies
    

    View Slide

23. Questions?
    l Víctor García (@V1KT)
    http://blog.tuenti.com/dev
    

    View Slide