How a payment processing company turned into a software release factory - Ankur Trivedi

There was a company which had typical long running releases and the business was not happy. Business wanted change but IT was not sure how to deliver. Then they heard about Agile. It looked like the magic potion to all their problems.They started doing Agile but it just meant more work for the team and a chaos during the last days of the sprint. Operations was still not happy. Then they heard about another magic potion called Devops, which forced them to think about continuous delivery. I'll talk about the various tools being used in order to bring about this change and what were the challenges that they faced in this journey.The talk has two dimensions: First, The tool chain and Second, how the team was convinced to hop on this journey of continuos delivery. Let me begin by saying that both of the points are equally important and correlated because convincing a team to change is very difficult if the tool selection is not right. I'll also get into a demo of the working pipeline using this toolset.

Transcript

1. H O W A PAY R O L L P R O C E S S I N G C O M PA N Y T U R N E D I N T O A S O F T WA R E R E L E A S E FA C T O RY Ankur Trivedi
2. Xebialabs We build tools to solve problems around devops and continuous delivery at scale.
3. W H AT T H E C O M PA N Y I N T H E C A S E S T U D Y D O E S • Offer products to support business for payroll, human resources and benefit outsourcing. • Team of 1000+ associates in IT across various functions. • Have 5,40,000 customers ranging from SMBs to big corporates • Range of solutions from SaaS based solutions for SMBs to on-premise solutions for big corporates.
4. C H A L L E N G E S • Multiple tools • Support for multiple tools • Custom Integrations • Multiple processes • lack of project visibility • Manual and Outdated repoting
5. V I S I O N • Maximise the power and creativity of high- performing, cross-functional teams to deliver value to the business and increase the reliability of delivery through active engagement and collaboration
6. I N S P I R AT I O N Source: https://xebialabs.com//assets/files/whitepapers/ITRev_DevOps_Guide_5_2015.pdf
7. T H I N G S T O D O T O E N A B L E T H E V I S I O N • Agile transition • Agile portfolio and program management • Continuos Delivery enablement with right set of tools
8. • Agile with Collaborative lifecycle management tools • Established tracks with backlogs • SCM Migration to a single SCM tool • Better reporting with the lifecycle management tools. Agile transition
10. S T I L L O P E N Q U E S T I O N • Continuos Delivery • Automated Build • Automated Test • Automated Deployment • Automated Provisioning
11. B E N E F I T S O F C O N T I N U O S D E L I V E RY • Higher Productivity (less “waste”) • lower costs • better time to market • better co-operation and resource utilisation • elimination of human error
12. T E C H N O L O G Y L A N D S C A P E • Linux, Sun Solaris and Windows Servers • Weblogic Application Server Farm • Apache WebServers • Jenkins • IIS Servers • Oracle Database
13. J O U R N E Y • What are the technological challenges and solutions? • Environment creation is manual and time taking • Disparate tools across application. • What should the deployment pipeline look like and how do we streamline that across applications? • The entire process from dev to prod has a lot of manual steps and is time taking. • How do we keep improving upon the process without breaking anything? • No real data to retrospect on and improve upon.
14. C H O I C E O F T O O L S • Automated Deployment - XL Deploy • Automated Provisioning - Puppet • Release Automation - XL Release
15. A R C H I T E C T U R E D I A G R A M Automa'on) Jenkins) XLDeploy) XLRelease) ) DEV) Check)In/Out) Subversion,)Git) Extract/Build) Ant,) Maven,)Jenkins) ) Label) Git,)Subversion) Unit)Test) JUnit/Clover,) MSBuild/MSCover) App)Deploy) XLDeploy) Smoke) Test) JTAF/MSL) INT) QC) PROD) 5) Traceability)Per)Drop) ! )Build)Records) ! "Build"Time,"Test"Results/Coverage" ! )Deploy)Records) ! )Throughput,"Deployment";me" App)Monitoring) Nagios) Ar'facts) Nexus) 4) 6) 1) 2) 8) 3) Data)Centers) 7)
16. H U R D L E S F O R A D O P T I O N • Risk sensitive culture • Complex structure across the organisation • Conflicting ideas from the Dev and Ops organisations on automation of the dev to prod pipeline
17. The Pipeline
18. Q U A N T I TAT E B E N E F I T S W I T H O U T C D W I T H C D D E P L O Y M E N T D E V / T E S T 2 H O U R S 5 M I N U T E S D E P L O Y M E N T P R E P R O D 2 H O U R S 1 0 M I N U T E S D E P L O Y M E N T P R O D 4 H O U R S 2 0 M I N U T E S
19. O T H E R B E N E F I T S • Reduced preparation time • Improved communication • Capability to deploy faster • Better environment utilisation
20. Words of Wisdom
21. T E A M C O M P O S I T I O N • Release • Merging the “Release Implementation” and “Application Operations” on the roles. • Test • More focus on automation testing. • Continual focus on improving the automation capabilities. • CM/Build • Build engineers trained to take up devops roles with being trained in new tools. • Traditional Infrastructure/Ops • Gradually turning into provider of environments using more automation.
22. O N T H E B I G D AY • On the go live day all the teams were called to have a look at what we had achieved and what they can expect when they on-board on the CD platform.
23. The Celebration
24. C O N T I N U O S D E L I V E RY E V O L U T I O N Level 1 Base Level 2 Beginner Level 3 Average Level 4 Advanced Level 5 Complete DevOps Monitoring Testing Provisioning Deploying Building Code accompanied with release notes with which operations should install and manage the application. Operations engaged at the end of the project. Development and operations work together when this is required. An envoy of operations works along in project, an envoy of development works along with operations. Operations and development are both part of the same multidisciplinary delivery team and share responsibilities. Monitoring of application log files for errors. Reports generated on demand. Monitoring of system metrics (CPU, disk, memory, process). Reports accessible to Operations. Monitoring of software quality, application performance. Reports accessible through dashboard. Application Health and Build/Deploy dashboards available to teams, provides continuous insight into quality, health and performance metrics. Monitoring of business level quality metrics. Predictive failure monitoring. Monitoring data is used actively to improve the system. Automated tests are initiated as soon as code is checked in. Tests are focused on unit /component testing only. All tests require manual activity. Some tests are automated but have to be initiated by hand. Automated static code and security analysis after code check in. Automated dynamic quality tests like security scans, functional and performance tests guarantee quality of code. 100% fully automated tests all the way to production Scripted installations per component for each environment. Supporting systems manually configured. Manual installation and configuration of Network, OS and software for middleware, databases, application servers, etc. Environments are identical. Operating System is virtualized. Several tools used to provision and configure an environment. Environments created and torn down by a push of the button. Supporting systems automatically configured Self Service portal for requesting environments. New environments are created with each new release. Network automatically configured. Self service deployments to development and test. Manual to Staging and Production. Deployment through execution of separate deployment- and db scripts. Manual configurations and installs / env. Environments are identical. Roll out of applications performed by a push of the button. Test-gated deployments single applications. Deployments occur automatically over multiple environments. Continuous end-to-end Test-gated deployments of multiple (end to end) applications. . Build scripts are performed in a central area and activated manually. Builds are performed on local workstation by use of one or more separate build scripts. Build on commit. Archived components are made available for reuse by other teams. Central build environment. Teams actively reuse generic components in a secure and controlled manner. End-to-end automated gated builds.
25. Questions?
26. Thank you