Superpowers in Agile Software Development

These slides were part of a guest lecture in the Agile Software Engineering course (https://www.cs.manchester.ac.uk/study/undergraduate/courses/courseunitsyllabus/?courseunitcode=COMP33711), of the School of Computer Science, University of Manchester.

It focuses on the technical practices which are necessary ingredients to agility, such as Continuous Integration and Continuous Delivery, as well as basics like version control and automated testing.

The goal was to give students some information on practices they should _expect_ to see in their jobs as graduates, if their teams really want to be called "agile".

Transcript

1. the secret ingredients to agility
   superpowers
   in agile software development
   Yorgos Saslis | @gsaslis | Software Delivery Engineer
   

   View Slide

2. who r u?
   2
   

   View Slide

3. yorgos saslis
   3
   community clean
   code
   open
   source
   automation
   

   View Slide

4. imperial
   college london
   Yorgos Saslis | @gsaslis | Software Delivery Engineer
   

   View Slide

5. View Slide

6. View Slide

7. journey
   better ways to write software
   7
   

   View Slide

8. yorgos saslis
   8
   

   View Slide

9. 9
   07
   MONITORING
   Most of it does actually take place
   on your monitor.
   agile
   What does it really mean?
   What makes you agile?
   background
   Who actually wrote the
   Agile Software Development
   manifesto
   feedback
   loops
   Yorgos Saslis | @gsaslis | Software Delivery Engineer
   

   View Slide

10. 10
    Testing
    Strategies
    What to test,
    How much
    Why
    Continuous
    integration
    More than just tools:
    A practice
    technical
    practices
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    Deployment
    pipeline
    No pipes involved.
    Automation is.
    

    View Slide

11. 11
    closing
    Q & A
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    VERSION CONTROL
    Never forget anything,
    ever again.
    123
    123 ANTI-PAtterns
    Red flags to watch out for
    in your next job.
    

    View Slide

12. View Slide

13. View Slide

14. by prof. Dave Snowden - Own creation, own work, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=12887809
    cynefin
    framework
    

    View Slide

15. FEEDBACK
    ACTED UPON
    ASAP
    FEEDBACK
    RECEIVED
    ASAP
    EVERY CHANGE
    TRIGGERS
    FEEDBACK
    if it’s
    complex…
    how do i get
    it right?
    15
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    probe sense respond
    

    View Slide

16. feedback loop
    16
    feed
    back
    receive
    Deploy:
    don’t sit on it.
    Let your customers use it.
    trigger
    Monitoring:
    Observe, 

    don’t wait to be told.
    Understand.
    Prioritise.
    Implement more change.
    act
    

    View Slide

17. sensing
    through monitoring & Analytics
    17
    January February March April May June July August September October November December
    

    View Slide

18. Monitoring
    key
    performance
    indicators
    18
    Negative ?
    Positive ?
    some change
    How did this change affect my product?
    
    

    View Slide

19. courtesy of https://www.cytechmobile.com/mcore/dashboards
    

    View Slide

20. courtesy of https://www.cytechmobile.com/mcore/dashboards
    

    View Slide

21. probing through a delivery pipeline
    21
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

22. 22
    delivery
    process
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

23. 23
    automation
    server
    How we
    deploy
    updates appropriate service
    rolling update
    deregister from
    load balancer
    upload new task
    definition
    ecs
    stop 1 “old
    version” container
    launch 1 “new
    version” container
    register on load
    balancer
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    artifact
    repository
    

    View Slide

24. CONTINUOUS INTEGRATION
    24
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

25. 25
    PROD
    ci server
    builds
    binary
    starts worker
    Tested
    WOrking
    binary
    where
    does the
    artifact
    come from?
    Runs ALL Tests
    against binary
    artifact
    repository
    

    View Slide

26. 26
    ci server
    STARTS DOCker
    container
    maven:3
    `mvn clean package`
    Unit tests Integration
    tests
    FUNCTIONAL
    tests
    create
    container img
    compile
    push docker
    image to repo
    How WE
    RUN THE
    TESTS
    final “working”
    container image
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

27. where
    TO RUN THE
    TESTS
    27
    CI Pipeline example
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

28. TESTING STRATEGIES
    28
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

29. WHAT tests
    do we run
    29
    run with
    *EVERY* CHANGE
    run as often
    as possible
    acceptance tests
    Commit Stage
    test code
    works
    test app
    works
    fast
    (< 5 mins ) take longer
    in memory whole stack
    

    View Slide

30. WHAT tests
    do we write
    30
    https://martinfowler.com/bliki/TestPyramid.html
    

    View Slide

31. v 0.1
    VERSION
    CONTROL
    31
    v 0.2
    v 1.0
    v 1.1
    v 2.0
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

32. What TO KEEP IN VERSION CONTROL
    32
    Source Code, obviously.
    Unthinkable that your
    company’s software lives
    on your developers’
    laptops alone.
    Configuration
    Any configuration files
    for your software,
    including their variations
    depending on the
    different environment
    the software is running
    on (e.g. for dev / test /
    staging / prod)
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

33. What TO KEEP IN VERSION CONTROL
    33
    Dependencies
    This involves any 3rd-party
    libraries your software
    depends on to run. Whether
    as binaries or (preferrably)
    through a dependency
    management framework
    (e.g. Maven for Java).
    Environment
    (i.e. Servers !!! )
    OS version.
    OS Configuration.
    3rd-party software:
    •application servers
    •web servers
    •RDBMS
    •message brokers
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

34. Infrastructure AS CODE
    34
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

35. MUST
    DO’S
    35
    MANY PER
    DAY
    EXPLAIN WHY,
    NOT WHAT
    MEANINGFUL MESSAGES
    REGULAR COMMITS
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

36. PEER
    REVIEW
    36
    at least 2
    ppl know QA
    MERGE REQUESTS
    4-eye principle
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

37. code
    REVIEW
    37
    example
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

38. CM
    IN A NUTSHELL
    38
    base image
    CM TOOL
    INSTALL & CONFIGURE
    3rd party software:
    * App server
    * WEB SERVER
    * RDBMS
    * MSG BROKER
    * CACHE
    chef puppet ansible salt
    FINAL
    image
    environment config:
    * DEV
    * test
    * staging
    * prod
    packer
    

    View Slide

39. What
    TO INSTALL
    39
    base image
    CM TOOL
    packer
    

    View Slide

40. HOW
    TO INSTALL IT
    40
    chef
    recipes
    os
    commands
    

    View Slide

41. ANTI PATTERNS
    41
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

42. #1: MANUAL DEPLOYMENTS
    42
    Common Signs:
    Extensive deployment docs
    Manually verify app is running correctly
    Frequent calls to dev team on release day
    Frequent corrections to the release process, *during* the release
    Unpredictable releases
    Roll-backs happen often
    it’s 2am. You’re still trying to figure out what went wrong.
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

43. #1: MANUAL DEPLOYMENTS
    43
    Why this is bad:
    You cannot rule out human error in deployments
    Manual deployment requires documentation
    Documentation is expensive to write (collaboration between many ppl), yet VERY easy to
    become out-of-date
    Deployments require expertise, yet are boring tasks. Your best people are underutilized.
    Manual deployments cannot be tested prior to the deployment itself.
    Deployment expert is on holidays, or off sick. Good luck deploying!
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

44. #2: Deploy TO STAGING
    at end of DEV cycle
    44
    Common Signs:
    System has so far only been tested on developers’ machines
    Production-like environment is “expensive”
    Deployment scripts, installers, configuration files, db migrations, etc. are passed on to ops
    by devs, entirely untested in production-like env.
    Development team is not working with Operations team on a day-to-day basis
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

45. 45
    Why this is bad:
    With the first release come the most bugs. Yet, it is already too close to the deadline.
    Last-minute patches, fixes, etc. impose a huge communication overhead
    The bigger the difference between `dev` and `prod` envs, the less realistic the
    assumptions the devs have to make: e.g. Windows on devs’ laptops, Linux in prod.
    The longer the release cycle, the harder it is to fix bugs. Developers do NOT remember the
    specifics of what they were working on 3 weeks ago (the same way they do “now”)
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    #2: Deploy TO STAGING
    at end of DEV cycle
    

    View Slide

46. #3: MANUAL
    CONFIGURATION MANAGEMENT
    46
    Common Signs:
    Even after many successful deploys to staging, deployment into production fails.
    Servers in a cluster behave differently. Some can serve less load than others.
    Operations team needs a long time to prepare a new environment for release
    Inability to go back to a previous known-working configuration (especially to older OS,
    RDBMS, application server version)
    System configuration is done by connecting directly (SSH) and modifying files by hand
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

47. #3: MANUAL
    CONFIGURATION MANAGEMENT
    47
    INSTEAD!
    ALL aspects of testing, staging, production (including 3rd party libs) are configured
    through version control
    All means: OS, patch levels, OS config, your application stack, its config, infra config, etc.
    Every change to production recorded (auditable).
    No manual changes allowed. Only through automated process.
    Use same automated process to roll-back to previous version.
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

48. The
    END
    48
    Thanks !!
    Yorgos Saslis | @gsaslis | Software Delivery Engineer
    

    View Slide

49. Yorgos Saslis
    Software Delivery Engineer @ Red Hat
    49
    twitter.com/gsaslis
    linkedin.com/in/gsaslis
    yorgo@protonmail.com
    github.com/gsaslis
    Q & a
    

    View Slide