Superpowers in Agile Software Development

These slides were part of a guest lecture in the Agile Software Engineering course (https://www.cs.manchester.ac.uk/study/undergraduate/courses/courseunitsyllabus/?courseunitcode=COMP33711), of the School of Computer Science, University of Manchester.

It focuses on the technical practices which are necessary ingredients to agility, such as Continuous Integration and Continuous Delivery, as well as basics like version control and automated testing.

The goal was to give students some information on practices they should _expect_ to see in their jobs as graduates, if their teams really want to be called "agile".

Transcript

1. the secret ingredients to agility superpowers in agile software development Yorgos Saslis | @gsaslis | Software Delivery Engineer
2. who r u? 2
3. yorgos saslis 3 community clean code open source automation
4. imperial college london Yorgos Saslis | @gsaslis | Software Delivery Engineer
7. journey better ways to write software 7
8. yorgos saslis 8
9. 9 07 MONITORING Most of it does actually take place on your monitor. agile What does it really mean? What makes you agile? background Who actually wrote the Agile Software Development manifesto feedback loops Yorgos Saslis | @gsaslis | Software Delivery Engineer
10. 10 Testing Strategies What to test, How much Why Continuous integration More than just tools: A practice technical practices Yorgos Saslis | @gsaslis | Software Delivery Engineer Deployment pipeline No pipes involved. Automation is.
11. 11 closing Q & A Yorgos Saslis | @gsaslis | Software Delivery Engineer VERSION CONTROL Never forget anything, ever again. 123 123 ANTI-PAtterns Red flags to watch out for in your next job.
14. by prof. Dave Snowden - Own creation, own work, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=12887809 cynefin framework
15. FEEDBACK ACTED UPON ASAP FEEDBACK RECEIVED ASAP EVERY CHANGE TRIGGERS FEEDBACK if it’s complex… how do i get it right? 15 Yorgos Saslis | @gsaslis | Software Delivery Engineer probe sense respond
16. feedback loop 16 feed back receive Deploy: don’t sit on it. Let your customers use it. trigger Monitoring: Observe, 
 don’t wait to be told. Understand. Prioritise. Implement more change. act
17. sensing through monitoring & Analytics 17 January February March April May June July August September October November December
18. Monitoring key performance indicators 18 Negative ? Positive ? some change How did this change affect my product? 
19. courtesy of https://www.cytechmobile.com/mcore/dashboards
20. courtesy of https://www.cytechmobile.com/mcore/dashboards
21. probing through a delivery pipeline 21 Yorgos Saslis | @gsaslis | Software Delivery Engineer
22. 22 delivery process Yorgos Saslis | @gsaslis | Software Delivery Engineer
23. 23 automation server How we deploy updates appropriate service rolling update deregister from load balancer upload new task definition ecs stop 1 “old version” container launch 1 “new version” container register on load balancer Yorgos Saslis | @gsaslis | Software Delivery Engineer artifact repository
24. CONTINUOUS INTEGRATION 24 Yorgos Saslis | @gsaslis | Software Delivery Engineer
25. 25 PROD ci server builds binary starts worker Tested WOrking binary where does the artifact come from? Runs ALL Tests against binary artifact repository
26. 26 ci server STARTS DOCker container maven:3 `mvn clean package` Unit tests Integration tests FUNCTIONAL tests create container img compile push docker image to repo How WE RUN THE TESTS final “working” container image Yorgos Saslis | @gsaslis | Software Delivery Engineer
27. where TO RUN THE TESTS 27 CI Pipeline example Yorgos Saslis | @gsaslis | Software Delivery Engineer
28. TESTING STRATEGIES 28 Yorgos Saslis | @gsaslis | Software Delivery Engineer
29. WHAT tests do we run 29 run with *EVERY* CHANGE run as often as possible acceptance tests Commit Stage test code works test app works fast (< 5 mins ) take longer in memory whole stack
30. WHAT tests do we write 30 https://martinfowler.com/bliki/TestPyramid.html
31. v 0.1 VERSION CONTROL 31 v 0.2 v 1.0 v 1.1 v 2.0 Yorgos Saslis | @gsaslis | Software Delivery Engineer
32. What TO KEEP IN VERSION CONTROL 32 Source Code, obviously. Unthinkable that your company’s software lives on your developers’ laptops alone. Configuration Any configuration files for your software, including their variations depending on the different environment the software is running on (e.g. for dev / test / staging / prod) Yorgos Saslis | @gsaslis | Software Delivery Engineer
33. What TO KEEP IN VERSION CONTROL 33 Dependencies This involves any 3rd-party libraries your software depends on to run. Whether as binaries or (preferrably) through a dependency management framework (e.g. Maven for Java). Environment (i.e. Servers !!! ) OS version. OS Configuration. 3rd-party software: •application servers •web servers •RDBMS •message brokers Yorgos Saslis | @gsaslis | Software Delivery Engineer
34. Infrastructure AS CODE 34 Yorgos Saslis | @gsaslis | Software Delivery Engineer
35. MUST DO’S 35 MANY PER DAY EXPLAIN WHY, NOT WHAT MEANINGFUL MESSAGES REGULAR COMMITS Yorgos Saslis | @gsaslis | Software Delivery Engineer
36. PEER REVIEW 36 at least 2 ppl know QA MERGE REQUESTS 4-eye principle Yorgos Saslis | @gsaslis | Software Delivery Engineer
37. code REVIEW 37 example Yorgos Saslis | @gsaslis | Software Delivery Engineer
38. CM IN A NUTSHELL 38 base image CM TOOL INSTALL & CONFIGURE 3rd party software: * App server * WEB SERVER * RDBMS * MSG BROKER * CACHE chef puppet ansible salt FINAL image environment config: * DEV * test * staging * prod packer
39. What TO INSTALL 39 base image CM TOOL packer
40. HOW TO INSTALL IT 40 chef recipes os commands
41. ANTI PATTERNS 41 Yorgos Saslis | @gsaslis | Software Delivery Engineer
42. #1: MANUAL DEPLOYMENTS 42 Common Signs: Extensive deployment docs Manually verify app is running correctly Frequent calls to dev team on release day Frequent corrections to the release process, *during* the release Unpredictable releases Roll-backs happen often it’s 2am. You’re still trying to figure out what went wrong. Yorgos Saslis | @gsaslis | Software Delivery Engineer
43. #1: MANUAL DEPLOYMENTS 43 Why this is bad: You cannot rule out human error in deployments Manual deployment requires documentation Documentation is expensive to write (collaboration between many ppl), yet VERY easy to become out-of-date Deployments require expertise, yet are boring tasks. Your best people are underutilized. Manual deployments cannot be tested prior to the deployment itself. Deployment expert is on holidays, or off sick. Good luck deploying! Yorgos Saslis | @gsaslis | Software Delivery Engineer
44. #2: Deploy TO STAGING at end of DEV cycle 44 Common Signs: System has so far only been tested on developers’ machines Production-like environment is “expensive” Deployment scripts, installers, configuration files, db migrations, etc. are passed on to ops by devs, entirely untested in production-like env. Development team is not working with Operations team on a day-to-day basis Yorgos Saslis | @gsaslis | Software Delivery Engineer
45. 45 Why this is bad: With the first release come the most bugs. Yet, it is already too close to the deadline. Last-minute patches, fixes, etc. impose a huge communication overhead The bigger the difference between `dev` and `prod` envs, the less realistic the assumptions the devs have to make: e.g. Windows on devs’ laptops, Linux in prod. The longer the release cycle, the harder it is to fix bugs. Developers do NOT remember the specifics of what they were working on 3 weeks ago (the same way they do “now”) Yorgos Saslis | @gsaslis | Software Delivery Engineer #2: Deploy TO STAGING at end of DEV cycle
46. #3: MANUAL CONFIGURATION MANAGEMENT 46 Common Signs: Even after many successful deploys to staging, deployment into production fails. Servers in a cluster behave differently. Some can serve less load than others. Operations team needs a long time to prepare a new environment for release Inability to go back to a previous known-working configuration (especially to older OS, RDBMS, application server version) System configuration is done by connecting directly (SSH) and modifying files by hand Yorgos Saslis | @gsaslis | Software Delivery Engineer
47. #3: MANUAL CONFIGURATION MANAGEMENT 47 INSTEAD! ALL aspects of testing, staging, production (including 3rd party libs) are configured through version control All means: OS, patch levels, OS config, your application stack, its config, infra config, etc. Every change to production recorded (auditable). No manual changes allowed. Only through automated process. Use same automated process to roll-back to previous version. Yorgos Saslis | @gsaslis | Software Delivery Engineer
48. The END 48 Thanks !! Yorgos Saslis | @gsaslis | Software Delivery Engineer
49. Yorgos Saslis Software Delivery Engineer @ Red Hat 49 twitter.com/gsaslis linkedin.com/in/gsaslis yorgo@protonmail.com github.com/gsaslis Q & a