De-mystifying DevOps A Practitioner’s Perspective Sept 2016 Promise DevOps The 

2 Joan Wrabetz CTO [email protected] @varbetz Edan Evantal VP, Engineering 

Agenda • Introduction • DevOps Framework • Sandboxes and Quali CloudShell • The Quali DevOps Journey • Challenges and Learnings • The Road Ahead • Summary and Best Practices 3 

01 02 03 Venture-funded start-up focused on DevOps and BizOps Automation Delivers “Sandboxes” - Replicas of production environments Strong Innovation DNA – Trusted by customers worldwide About Quali 4 300+ customers – Cloud providers, Telcos, Enterprises – across FSI, Retail, Healthcare and Government ~115 employees in USA, Israel and Europe focused on serving customers worldwide 300+ 

DevOps is A Journey Let’s Do Agile Self Service Automated Continuous 

How We View the DevOps Lifecycle 6 

Quali Cloud Sandboxes Building Blocks for DevOps and BizOps Automation Personalized replicas of production environments Reservation Self-Service with Sophisticated Modeling and Automation BI and Analytics Deploy Anywhere Set-up and tear down Environment GUI or API-driven Connectivity Applications and Data Physical or Virtual Infrastructure Orchestrate DevOps and BizOps workflows Security “Full Stack” On-Prem or Hybrid Cloud Multi-tenant 

Quali Cloud Shell – Industry –Leading Sandbox Hybrid Infrastructure Enable Self-Service Environments with BI and Analytics Provide physical/virtual Infrastructure & Applications (Any DC/ Cloud) Mode & Orchestrate Full-Stack replicas of Production Environments 

Sandboxes in Action 

ARCHITECTURE CLOUDSHELL Self-Service Portal Northbound APIs Shells: Infrastructure Automation Process/Workflow Orchestration Application Automation Visual Authoring Tool Open Automation APIs Script-based Authoring tools Sandbox Services Reporting/Business Intelligence DB Dash- boards REST APIs Python APIs Shell Shell Shell Shell WORKF LOW WORKF LOW WORKF LOW WORKF LOW AWS VCenter Openstack Azure • 300K LOC • C#, C++, JavaScript, Python • Microsoft .net • Sql Svr DB • SiSense BI DB • 3rd party add-ons • Scale-out Exec Servers • Web Portal(s) • Language support • White-label Capability • Any OS or cloud 

Quali DevOps Tools 50+ Engineers 15% DevOps tools Group Development Tools Source Control Build Virtualization / Cloud Test Dev/Test Sandbox ALM Collaboration 

Stage 1 – Implement Agile SCRUM Implement Agile Quali DevOps Journey 7 1 2 3 4 5 6 FIX ISSUE • 2 weeks “sprints” create planning overhead • QA and test cycle = 50% of release bloat. • Definition of “done” • Shift to 4 wk. sprints to automate tests • Implemented Kanban with short weekly planning We Implemented Agile Scrum framework fully 

Quali Lean Development Process Today Product Backlog Backlog grooming “Next“ stories Daily Scrum Meeting – Release Kickoff Weekly planning 4 weeks Sprint Review and Retrospective Potentially Shippable Product Increment 

14 Efficiency 2X productive • Agile Development • Kanban planning • Paired code reviews 1 

Stage 2 – Increase % Automation Implement Agile Increase Automation Quali DevOps Journey 7 1 2 3 4 5 6 FIX ISSUE • Only 40% tests automated • Escape defects • Reduce UI testing • Implemented zero bug policy • Quality sprints – zero bug baseline release 

Test Automation Composition UI Integration Unit 10’s 100’s 1000’s UI Integration Unit 100’s 10’s 100’s 3 Years Ago Today 

17 • Agile Development • Kanban planning • Paired code reviews • Zero bug policy 2 Test Automation 85% Continuous testing 2 

Stage 3 – Get Real Customer Input Implement Agile Increase Automation Customer Input Quali DevOps Journey 7 1 2 3 4 5 6 FIX ISSUE • INTERNAL sprint demos at end of 4 week releases. No customer input till EA  too late to make changes • EXTERNAL sprint demos to field and delivery • CloudShell sandbox for realistic config., training and customer roadshows 

Field Training and customer demos with CloudShell sandboxes 3 EA Feedback Months to Weeks 

Stage 4 – Move to DevOps Implement Agile Increase Automation Customer Input Move to DevOps Quali DevOps Journey 7 1 2 3 4 5 6 FIX ISSUE • Obtain immediate Developer feedback. • Integrate CI/CD • Create CI Pipeline  Shift left • Created CloudShell sandboxes mirroring customer environments  added to automated test flow • Invested in DevOps tools group Motto: Make the Developers Happy 

21 Continuous testing Developer Feedback Weeks to Minutes Continuous Integration Cloud Sandboxes 4 

Stage 5 – Engage Customer and Partner Dev. Implement Agile Increase Automation Customer Input Move to DevOps 3rd Party Developers Quali DevOps Journey 7 1 2 3 4 5 6 FIX ISSUE • Get customers and partners to contribute to “shell” development • Collaborate - Distributed teams • Open Source “Shells” on GitHub • DevOps pipeline moved to GitHub for open participation • Developer SDK • Online documentation • Docs and Code open for updates • API for 3rd party/ community 

23 Open Source 10X Innovation Velocity • Open Source Shells • Source Code in GitHub • Dev tools and docs • API driven 5 

Stage 6 – Getting to GA Faster Implement Agile Increase Automation Customer Input Move to DevOps 3rd Party Developers GA Release Quality Quali DevOps Journey 7 1 2 3 4 5 6 FIX ISSUE • 3-month releases • No continuous deployment • Impacts new feature velocity • Live staging for EA customers • Interim builds – with features in development • CloudShell demos on AWS – experiment on early-stage builds 

25 Customer Sat. 30% Increase Customer Staging Sites Demos of latest build to EA customers speeds GA 6 

Stage 7 – Increase Responsiveness Increase Responsiveness Implement Agile Increase Automation Customer Input Move to DevOps 3rd Party Developers GA Release Quality Quali DevOps Journey 7 1 2 3 4 5 6 FIX ISSUE • Continuous Deployment • New features and bugs cause disruption or get delayed by 3 months • Series of optimizations in progress 

The Quali DevOps Journey … So Far Increase Responsiveness Implement Agile Increase Automation Customer Input Move to DevOps 3rd Party Developers GA Release Quality Quali DevOps Journey 7 1 2 3 4 5 6 2X Efficiency Increase 85% Test Automation EA Feedback: Months to Weeks Developer Feedback: Week to Minutes Work in Progress 30% Increase in Customer Satisfaction 10X Innovation Velocity 

DevOps Progress Board To Do In Progress Done (?) Eliminate release planning - only feature level planning Flexible CD for non-SaaS Deployments Deliver customer code non-disruptively on- premise or in the cloud Fixed scrum team allocation. No pre- planned backlogs 100% Continuous Integration DevOps Style Innovation Customer and Partner Engagement Feature velocity and Faster EA customer feedback Community scaling. Test Automation 

DevOps Best Practices – It is a Journey! Focus on Agile Development 1 Take an “Automation-First” Approach – CI/ CD 2 Define end-goals. Leverage Cloud Sandboxes to Accelerate and De-risk the journey. 3 Successful pioneers like Netflix, Etsy and Amazon took 5+ years to get CD (We are in year 3). 

Q & A 

Quali Community 

Blogs www.quali.com Social Media @Quali @Qualisystems 32 Quali engages with its customers, partners and developers in many ways. Stay in Touch