Overview of DevOpsPhil Huang SDN Solution Engineer, Edgecore Networks CorporationDec 1, 2016
View Slide
Summary© 2016 Edgecore Networks. All rights reserved. Subject to errors and misprints. |www.edge-core.comOverview of DevOps010402Overview of NetDevOps03DevOps Case Studies
Who am I?3© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Phil Huang§ Edgecore SDN Solution Engineer§ ONOS/CORD Ambassador§ Familiar with§ Open Source / DevOps / NetDevOps§ SDN Open Source Committer§ ONOS / Mininet / ONIE§ Project§ Global ONOS SDN-IP Deployment, Taiwan§ GitHub§ pichuang
General Product Team4© 2016 Edgecore Networks. All rights reserved | www.edge-core.comRef: http://www.mindtheproduct.com/2016/02/what-the-hell-are-ci-cd-and-devops-a-cheatsheet-for-the-rest-of-us/You QA/QC PM Operator
Problem1: What’s wrong?5© 2016 Edgecore Networks. All rights reserved | www.edge-core.comRef: http://www.mindtheproduct.com/2016/02/what-the-hell-are-ci-cd-and-devops-a-cheatsheet-for-the-rest-of-us/
Operator got…6© 2016 Edgecore Networks. All rights reserved | www.edge-core.comRef: VMWare ESX Host Purple Diagnostic Screen
PM & QA think…7© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
You think…8© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
Problem 2: Lead time Too Long9© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Lead Time§ Time between request to deliveryØThis is what customer sees§ Cycle Time (or Process Time)§ Time between production start to completionØNot visible to customerRef: http://www.slideshare.net/Managewell/lean-and-kanbanbased-software-developmentTimeRequest DeliveryProduction StartLead TimeCycle Time
No Value, No Revenue10© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
Problem 3: Schedule Visibility11© 2016 Edgecore Networks. All rights reserved | www.edge-core.comResultVisible PartMindsetThinkingInvisible PartExperienceYouDirty work
Problem 4: Human Error Race• 80% of outages impacting mission-critical services will be caused bypeople and process issue• 20% of those outages will be caused bychange/configuration/release integration and hand-off issues© 2016 Edgecore Networks. All rights reserved | www.edge-core.com 12Ref: http://www.slideshare.net/giganati/is-orchestration-the-next-big-thing-in-devops
Problem 5: What happened in your system?13© 2016 Edgecore Networks. All rights reserved | www.edge-core.comAdministrator controlvia GUI and CLICrontab from systemForce majeure…Input Output
Why DevOps?14© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Collaboration§ Developer + Operator§ Reduce challenges related to Ops and Dev collaboration§ Deployment§ Minimize deployment related downtime§ Minimize roll-backs of deployed application§ Defect resolution§ Minimize MTTR§ Increase the ability to reproduce and fix defectsMTTR = Mean Time To Resolution
DevOps is about CALMS15© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§Culture§ More communication with team member§Automation§ The idea that you should program everything§Lean§ Keeping everything to a minimum§Measurement§ Try to have visibility into everything§Sharing§ Regular exchanging of ideas across team
Culture16© 2016 Edgecore Networks. All rights reserved | www.edge-core.comMore FeedbackCommunicationAutomation
Culture17© 2016 Edgecore Networks. All rights reserved | www.edge-core.comWaterfallPlan Code Build Test Release Deploy OperateTimePlanCodeBuildTestRelease PlanCodeBuildTestRelease PlanCodeBuildTestReleaseAgile
Automation18© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ DevOps LifecyclePlan Code Build Test Release Deploy OperateMonitoringMetricsAnalyticsFeedback
Automation19© 2016 Edgecore Networks. All rights reserved | www.edge-core.comPlan Commit Build Test Release Deploy OperateMonitoringMetricsAnalyticsFeedbackContinuous IntegrationContinuous Delivery
Lean20© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§MVP ( Minimal Viable Product )§ Set a goal§ Minimum cost§ Time, Money…etc§ Feedback from customer§ Continual ImprovementRef: http://justinmcgill.net/product-idea-validation-dont-build-mvp/
Lean21© 2016 Edgecore Networks. All rights reserved | www.edge-core.comRef: https://blog.engineyard.com/2015/actually-mvp
Measurement22© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Data collection§ Syslog, performance data§ Definition of measurement§ Data visualization§ A picture is worth athousand words§ Data analysis§ Find business value
Sharing23© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Focus on communication§ Any experiences for teamØFocus on people
2016 State of DevOps Report24© 2016 Edgecore Networks. All rights reserved | www.edge-core.comRef: https://puppet.com/resources/white-paper/2016-state-devops-report/
Periodic Table of DevOps Tools25© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
Develop workflow26© 2016 Edgecore Networks. All rights reserved | www.edge-core.comYouWorkflow1. Launch VM2. Setting environment3. Install applications4. Quick start5. Develop6. Compile7. Re-install applicationChoose Hypervisor, CPU, RAM, Disk size, locationBash, vim, dependency packageView INSTALL fileView README filesView Developer’s Guide
Develop workflow27© 2016 Edgecore Networks. All rights reserved | www.edge-core.comYouWorkflow1. Launch VM2. Setting environment3. Install applications4. Quick start5. Develop6. Compile7. Re-install applicationChoose Hypervisor, CPU, RAM, Disk size, locationBash, vim, dependency packageView INSTALL fileView README filesView Developer’s Guide
Baby Step 1 – Reduce Routine Manual Job$ cd $PROJECT_ROOT$ python setup.py build$ python setup.py install$ cd $WORK_ROOT$ ./service start• $ /usr/bin/rebuild.sh© 2016 Edgecore Networks. All rights reserved | www.edge-core.com 28Manually One-Click#!/bin/bashset -xcd $PROJECT_ROOTpython setup.py buildpython setup.py installcd $WORK_ROOT/service start
Baby Step 2 – Version Control Everything• $ /usr/bin/rebuild.sh© 2016 Edgecore Networks. All rights reserved | www.edge-core.com 29#!/bin/bashset -xcd $PROJECT_ROOTecho “Start build”python setup.py buildecho “Start install”python setup.py installcd $WORK_ROOT/service start
Develop workflow30© 2016 Edgecore Networks. All rights reserved | www.edge-core.comYouWorkflow1. Launch VM2. Setting environment3. Install applications4. Quick start5. Develop6. Compile7. Re-install applicationChoose Hypervisor, CPU, RAM, Disk size, locationBash, vim, dependency packageView INSTALL fileView README filesView Developer’s Guide
Baby Step 3 – Packer + Vagrant31© 2016 Edgecore Networks. All rights reserved | www.edge-core.comProvisioningRuntime VM
Case Study - OpenSource Contribution Workflow32© 2016 Edgecore Networks. All rights reserved | www.edge-core.comRef: http://docs.openstack.org/infra/manual/developers.htmlYou
Gerrit Code Review33© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
Jenkins34© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
Case Study - CISCO35© 2016 Edgecore Networks. All rights reserved | www.edge-core.comRef: http://www.slideshare.net/CiscoDevNet/enabing-devops-in-an-sdn-world
Case Study - Spirent36© 2016 Edgecore Networks. All rights reserved | www.edge-core.comRef: http://dw.connect.sys-con.com/session/2779/Marc_Hornbeek.pdf
Case Study – BigSwitch Chaos Monkey37© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
What is NetDevOps?38© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Infrastructure as Code (IaC)§ Bare Metal network§ VM network§ Container networkØ Configuration Management§ Network automationØRapid provisioning
Traditional Networking Methodology39© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Logging in manually with a password§ Typing commands§ Copy and paste over console or terminal§ Check, check and check..
Step 1: Using Git & GitHub40© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Configuration file repository§ File change and revision management§ Build for teams to work the same files
Step 2: Commit Your Configuration41© 2016 Edgecore Networks. All rights reserved | www.edge-core.com$ git add .$ git commit -a -m "Add new switch configuration" –s[master 8cf8a38] Add new switch configuration 1 filechanged, 1 insertion(+) create mode 100644ofswitch.conf$ git push -u origin master
Show Changes In Action42© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
Blame someone43© 2016 Edgecore Networks. All rights reserved | www.edge-core.com$ git blame ofswitch.conf69f5f511 (Phil Huang 2016-11-01 13:59:23 +0800 1) Phil Huang69f5f511 (Phil Huang 2016-11-01 13:59:23 +0800 2) is a60643262 (Phil Huang 2016-12-01 15:00:51 +0800 3) good69f5f511 (Phil Huang 2016-11-01 13:59:23 +0800 4) man
Step 3: Choose CM44© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
Ansible Network Modules45© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
Step 4: Templatize Configuration Files46© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Let you reuse code snippets§ Avoid human error and typosRef: https://github.com/CumulusNetworks/cldemo-netq-l3
Step 5: Virtualize Copy of Network47© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
Step 6: Continuous Integrations System48© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ Catch error or typos early§ Make sure all of the setting is sameas production
Summary49© 2016 Edgecore Networks. All rights reserved | www.edge-core.com§ More communication§ Get early feedback§ Automate everything§ Version control§ Visibility
Recommend Books50© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
51Open NetworkingfromFreedomControlInnovation© 2016 Edgecore Networks. All rights reserved | www.edge-core.com
© 2015 Edgecore Networks. All rights reserved. Subject to errors and misprints. | www.edge-core.com