Pusheando en master, que es gerundio

Slides de la charla "Pusheando en master, que es gerundio": https://www.meetup.com/es-ES/CSTechHub/events/271540517/

Vídeo: https://www.youtube.com/watch?v=UvtaujgCNsI

Descripción:
En esta sesión me gustaría compartir mi experiencia trabajando con Trunk-Based Development (TBD) durante los últimos años, tras haber pasado muchos otros con diversas estrategias de branching.

Empezando por intentar entender “por qué hacemos lo que hacemos”, qué buscamos realmente con cada práctica y principio, la idea es ver durante la sesión qué ventajas ofrece trabajar con TBD, sus inconvenientes, cuándo considero que tiene sentido hacerlo… y cuándo no.

Tras una primera parte de la charla más “unidireccional”, habrá una segunda parte ya plenamente grupal y participativa, donde todo el mundo pueda compartir sus experiencias, dudas, inquietudes, etc., para reflexionar y aprender juntes :-)

Transcript

1. Pusheando en master, que es gerundio
   (Trunk-Based Development 101)
   CSTechHub
   9 de julio de 2020
   El vídeo de la charla
   

   View Slide

2. Ego slide
   Hemos venido a jugar...
   Desarrollador de software/producto
   @islomar
   islomar@gmail.com
   
   Teleco, Psicología, Antropología
   Social y Cultural, Executive MBA…
   
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. #NoTools
   #NoCode
   

   View Slide

8. What is Trunk-Based Development about?
   Just always push directly to your
   main (and only) branch
   Easy-peasy!!
   

   View Slide

9. View Slide

10. Agenda
    ● Continuous X
    ● Basic branching strategies & Pull/Merge Requests
    ● “Why” and “what for”: SW development principles
    ● Trunk-Based Development for the win!
    ● My personal experience
    ● When NOT to use it
    ● How to start with TBD
    ● Recommended people, books and resources
    ● Q&A
    

    View Slide

11. Continuous X
    ● Continuous Integration
    ● Continuous Delivery
    ● Continuous Deployment
    ● Continuous Release
    

    View Slide

12. Continuous Integration
    “Integrate and test changes after no more than a couple of hours.”
    “The integration step is unpredictable, but can easily take more time than the
    original programming.”
    “Continuous integration should be complete enough that the eventual first
    deployment of the system is no big deal”
    (“Extreme Programming Explained, Second Edition”, Kent Beck - 2005)
    

    View Slide

13. Continuous Delivery
    “Every push” (Isi©) leaves your code continuously ready to be deployed to
    Production with one single and simple manual step.
    ● Continuous Integration is a prerequisite
    ● Probably you will need feature flags (among other techniques)
    ● Deployments need to be automated
    

    View Slide

14. Continuous Delivery
    “Software delivers no revenue until it is in the hands of its users”
    “How long would it take your organization to deploy a change that involves
    just one single line of code?”
    “Branching should, in most circumstances, be avoided”
    (“Continuous Delivery: Reliable Software Releases through Build, Test, and
    Deployment Automation”, Jez Humble and David Farley - 2010)
    

    View Slide

15. Continuous Deployment
    Every change/push goes through the pipeline and automatically gets put into
    production.
    ● Continuous Delivery is a prerequisite
    ● No human intervention
    You need:
    ● Great safety net (different kind of automated tests, statics analysis,
    monitoring, etc.)
    ● You will certainly need feature flags (plus other parallel changes
    techniques)
    ● Easy way to rollback
    

    View Slide

16. https://opensenselabs.com/blog/articles/continuous-deployment-drupal
    

    View Slide

17. Continuous Release
    DEPLOYMENT != RELEASE
    ● Released: visible to the users
    ● Deployed: the code is “put” in the Production systems, maybe not visible.
    ○ With a feature toggle, or an unknown URL or a hidden HTML element or
    ● Decouple “deployment” and release!
    

    View Slide

18. Continuous Release
    Deploying to Production (without releasing) has value in itself:
    ● The whole pipeline works fine (e.g. tests, infrastructure, etc.)
    ● You check that nothing breaks (even not being active).
    ● You might want to partially activate it (e.g. with feature toggles/flags).
    ● You learn about the HOW.
    

    View Slide

19. How often is “continuous enough”?
    Online browser editor which triggers the deployment pipeline after clicking a
    specific button… why not?
    ● E.g. Dark
    ○ “A holistic programming language, editor and infrastructure for serverless backends”
    ○ “Deployless means that anything you type is instantly deployed and immediately
    usable in production.”
    ○ Native Continuous Delivery, using feature flags
    ○ https://darklang.com/
    ○ https://medium.com/darklang/how-dark-deploys-code-in-50ms-771c6dd60671
    

    View Slide

20. Source Code Management (SCM) tools
    1. No Version Control System (VCs) at all.
    2. Floppy disks.
    3. CVS (no branches at all)
    4. SubVersion
    a. Creating a branch was highly costly
    5. Git
    a. Feature branching: https://martinfowler.com/bliki/FeatureBranch.html
    b. GitFlow
    i. Master, Develop, Feature, Release and Hotfix branches
    

    View Slide

21. Branching strategies (over-simplified)
    ● Feature branching
    ○ One branch per feature (sometimes it means per developer)
    ● GitFlow
    ● Other combined strategies
    ○ Feature + Main + Release branches
    ○ Release trains
    

    View Slide

22. GitFlow
    ● Develop branch
    ● Feature branches
    ● Release branches
    ● Master branch
    ● Hotfix branches
    https://nvie.com/posts/a-successful-git-branching-model/
    

    View Slide

23. https://nvie.com/posts/a-successful-git-branching-model/
    

    View Slide

24. Why do people do Pull/Merge Requests?
    ● For feeling safer
    ○ Peer review: Someone else can take a look to it and detect problems
    ● For rising the code quality (e.g. design solution)
    ● For sharing/spreading knowledge
    ● For getting the control:
    ○ "Safe" way to accept contributions from other people/teams
    ○ Final review centralized in a few people (e.g. QA, PM, committee, etc.)
    

    View Slide

25. Problems with Pull/Merge Requests
    ● It completely breaks the flow
    ○ Increased lead time: no Continuous Delivery/Deployment
    ○ You better wait for that PR to be merged before creating a new one… or fight with Git
    ● Increased risk
    ○ It’s too late: someone already worked several hours (days?) on it…. Waste.
    ○ More probability of things breaking or not fitting… or not being needed anymore.
    

    View Slide

26. Problems with Pull/Merge Requests (cont.)
    ● Code reviews:
    ○ you don’t have the context of the person for every decision made.
    ○ tenths/hundreds of lines changed... are you kidding?
    SOURCE OF INEFFICIENCIES AND HUMAN CONFLICTS
    

    View Slide

27. View Slide

28. (IMO) When do pull/merge requests make sense?
    ● Distributed team with very different time zones
    ○ Not really possible to do pair programming
    ● OSS: very async, worldwide distributed people
    ● Contributions between teams inside a company
    ○ Though, even better: pair with someone from the other team
    ● For some exceptional reason, someone worked on their own
    

    View Slide

29. https://martinfowler.com/articles/branching-patterns.html#continuous-integration
    This is what it REALLY happens when branching...
    https://twitter.com/jahnnie/status/937917022247120898
    

    View Slide

30. But wait… let’s take several steps back...
    

    View Slide

31. Why and what for do we develop software professionally?
    ● From an individual/personal level: para ganarnos la vida, para
    divertirnos, para sentirnos útiles, para evitar el horror vacui vital...
    ● From a company perspective: to earn money, to “make the world
    a better place” (Silicon Valley dixit)
    Para resolver problemas que no resulta igual de
    óptimo/eficiente resolver sin software
    

    View Slide

32. View Slide

33. ● “Early and continuous”
    ○ How “early”?
    ○ How “continuous”? (“... from a
    couple of weeks…”)
    ● “Working software” + “Valuable
    software”
    ○ Does it break?
    ○ Fits business requirements?
    ○ Is it used?
    ○ Does it satisfy the customer/user?
    ○ Does it generate “profit”?
    

    View Slide

34. “agile” is about…
    ● Focus on people
    ○ Communication
    ○ Needs and expectations
    ● Risk management
    ○ “Short” feedback cycles, in order to improve in either:
    ■ Business (WHAT): user feedback
    ■ Technology (HOW): system feedback
    ○ Learn fast (not “fail fast”)
    

    View Slide

35. Which branching strategy
    would “better” fit with
    reducing the risk management,
    continuous learning and the
    continuous delivery of value?
    (and less people suffering!!)
    

    View Slide

36. Trunk-Based Development for the win!!
    https://unsplash.com/photos/abkEAOjnY0s
    

    View Slide

37. https://twitter.com/CodelyTV/status/1248216553138552834
    

    View Slide

38. https://svgsilh.com/image/2028004.html
    Technical
    excellence
    Continuous
    delivery of
    value to the
    business
    https://freesvg.org/unicorn-vector-clipart-pdv
    “Agile”
    People first!
    Make the
    world a
    better place!
    

    View Slide

39. Trunk-Based Development is about...
    Pushing to master
    

    View Slide

40. https://unsplash.com/photos/MggK54YixfU
    

    View Slide

41. Trunk-Based Development is about...
    Keeping one single branch “by default” and pushing your code there
    

    View Slide

42. Trunk-Based Development is about… (cont.)
    +
    Pair/mob programming
    +
    Parallel changes techniques
    +
    TDD (or at least “test first”, including baby commits)
    +
    A proper product management vision
    

    View Slide

43. Trunk-Based Development is about… (cont.)
    A “safe”, easy and automated deployment/release pipeline (CD)
    +
    Easy way to rollback
    +
    “Loosely-coupled” teams/services
    +
    An engineering culture which supports it
    +
    etc...
    

    View Slide

44. “A proper product management vision”
    ● Product Manager working in the day by day with the rest of the team
    ● Vertical slicing:
    ○ Simplified and small end-to-end chunks
    ○ Business (user stories): GREAT (release) > we learn the WHAT
    ○ Technical: good (deployment) > we learn the HOW
    ● Continuous prioritization
    ● Focus on value (aligned with “higher company goals”)
    ○ Measure the “value” delivered
    ● Pull system (keep it flowing!!)
    ○ Guided by the system/team capacity + JIT (last responsible moment)
    

    View Slide

45. Some other IMPORTANT practices (not only) for TBD
    ● Canary releases
    ○ https://martinfowler.com/bliki/CanaryRelease.html
    ○ First, deployed only to some representative environment
    ● Feature toggles (a.k.a “feature flags”)
    ○ https://martinfowler.com/articles/feature-toggles.html
    ● Other parallel changes techniques
    ○ https://martinfowler.com/bliki/ParallelChange.html
    ○ Branch by abstraction
    ○ Postel’s law: “Be conservative in what you do, be liberal in what you accept from others”
    ○ Strangler Application
    ○ Anti-Corruption Layer
    

    View Slide

46. What if I can not do ALL of that?
    ● Keep calm and do not push to master (yet)
    ● It is sometimes a sign of deeper inefficiencies and improvement
    opportunities, it just makes them “explicit” (if we have enough self-critic
    and humility)
    Congratulations! We have a great opportunity of improvement ahead!
    

    View Slide

47. If TBD is not about “pushing to master”... what is it
    about?
    ● Lean Product Management
    ○ Risk management
    ○ “Fast” feedback
    ● REALLY continuous and sustainable delivery of value
    ● Technical excellence (e.g. many XP/DevOps principles and practices)
    

    View Slide

48. We want a feedback cycle “as short as possible”
    Longer delay
    between
    deployments
    https://www.goodreads.com/book/show/34695798-release-it
    Higher risk of
    bugs and
    downtime
    More changes in
    each deployment
    Review
    processes
    A great vicious cycle
    

    View Slide

49. Another way of saying it...
    Reduce your lead time: from a customer making a request to the customer
    being satisfied.
    Reduce your product delivery lead time: from code committed to code
    successfully running in production.
    https://www.goodreads.com/book/show/35747076-accelerate
    

    View Slide

50. https://cloud.google.com/blog/products/devops-sre/the-2019-accelerate-state-of-devops-elite-performance-productivity-and-scaling
    

    View Slide

51. https://cloud.google.com/blog/products/devops-sre/the-2019-accelerate-state-of-devops-elite-performance-productivity-and-scaling
    

    View Slide

52. Code is not an asset, it is liability!
    ● The best line of code is the one not written.
    ○ “Fewest elements”: https://martinfowler.com/bliki/BeckDesignRules.html
    ● Code needs to be maintained.
    ● Code needs to be tested.
    ● Code needs to be understood.
    ● A line of code is one more possible point of failure.
    ● Software has a COST.
    https://wiki.c2.com/?SoftwareAsLiability
    Dan North: Microservices software that fits in your head
    

    View Slide

53. Code is not an asset, it is liability!
    “Between the time a developer commits code to the repository and the time it
    runs in production, code is a pure liability. Undeployed code is unfinished
    inventory. It has unknown bugs. It may break scaling or cause production
    downtime. It might be a great implementation of a feature nobody wants.”
    (“Release it! Second Edition”, Michael T. Nygard - 2018)
    

    View Slide

54. Does it require to work with monorepo?
    NOT at all
    Previous session in this Meetup: Monorepo: uno para gobernarlos a todos
    

    View Slide

55. My personal experience
    ● Two full years at Alea Soluciones
    ○ TDD + pairing + feature toggles + vertical slicing simplification (EVERYDAY)
    ○ Canary releases
    ○ Multiple services (multirepo)
    ○ TBD was the way to work (only a couple of branches for spikes or something like that)
    ○ Lucky me, I forgot how to use Git beyond commit, pull and push
    ● Six months at Spotify
    ○ Context: pure backend. People working with PRs all the time.
    ○ I started introducing TDD + pairing + baby-commits. No need for feature toggles.
    ○ That enabled to start experimenting TBD in a a couple of simple services (not too
    critical).
    

    View Slide

56. My personal experience
    ● Currently (last 3 months)
    ○ TDD + pair/mob + baby-commits + vertical slicing + TBD (EVERYDAY)
    ○ … but still not released to users (client needs).
    ○ Hopefully we’ll start with Continuous Delivery
    

    View Slide

57. When NOT to do TBD?
    Any time where you don’t follow “all” of the practices mentioned:
    ● pair/mob programming
    ● parallel changes techniques
    ● TDD (or at least “test first”)
    ● A proper product management vision
    ○ Simplified end-to-end “user stories”
    ○ Vertical slicing: small end-to-end chunks
    ■ Business: GREAT
    ■ Technical: good
    ○ Business ability to move in baby steps
    

    View Slide

58. When NOT to do TBD?
    ● Spikes
    ○ After the spike, start from scratch in the main branch
    ● Most OSS projects (super-async, no pairing, distributed timezones…)
    ● When someone worked on their own (no pair/mob)
    ● When the team can not work quite “independently” from other teams
    ○ 'No Man is an Island' (by John Donne)
    ● When the team doesn’t feel “comfortable enough”
    ○ Really hard to achieve if nobody in the team has previous experience.
    

    View Slide

59. How to start? (I)
    ● Get help (hire people with experience in this approach)
    ● Work in pairs (so much as possible; even better: do mob!!)
    ● Write tests (isolated + sociable, aka “unit and integration”)
    ● Build a continuous integration server that can monitor the main
    repository and run the tests automatically for every new commits pushed.
    ● Limit the work in progress a.k.a. WIP (e.g. half the number of developers)
    

    View Slide

60. How to start? (II)
    ● Automate your deployment to Production
    ● Get a staging/test server where to improve the safety net
    ○ Acceptance tests
    ○ End to end tests
    ○ Load tests, etc.
    ● Configure a release/deployment pipeline which breaks if the different
    checks/validations do not pass (Andon a.k.a. “Stop the line”)
    ● Make your branches ages visible and limit it to 24 hours (e.g. with Panda
    Pull Reminders)
    

    View Slide

61. How to start? (III)
    ● Start with TDD (again: get help)
    ● Improve your baby steps and baby commits skills
    ● Improve your vertical slicing and business simplification skills
    ● Reduce the branches lifetime to 2 hours.
    ● Reminder: this is not about tools or technologies!!
    ● Pick up a non-critical service (e.g. a simple API) and start trying
    Continuous Delivery (push deploy to the staging environment).
    In the “worst case scenario”, if TBD doesn’t “succeed”, you will have improved
    in many other things!!!
    

    View Slide

62. Main key points
    ● TBD is much more than “pushing to master”
    ○ Risk management
    ○ Reduced lead time
    ■ continuously delivering value to the BUSINESS
    ■ Fast feedback (users and systems)
    ● TBD is a great excuse to detect many inefficiencies:
    ○ why wouldn’t you be able to start working like that?
    ○ What should be “fixed” or “improved” to achieve it?
    ○ Everything is “connected”!! (systems thinking)
    

    View Slide

63. Main key points
    ● It requires to live together with many other practices and principles
    (not only technical ones, but also related with product management).
    ● Keep calm and start in baby steps… but start!
    ○ Probably your context or most of your activity is not “superexceptional”
    ○ Though often it requires a HUGE change in your engineering culture… everything is
    connected!
    

    View Slide

64. Last reflection: renaming “master”?
    Whatever your first or second thought is… give it a third
    ● IETF terminology proposal (from 2018):
    https://tools.ietf.org/id/draft-knodel-terminology-00.html#rfc.section.1.1
    ● Probable naming origin: Bitkeeper and its master/slave
    ○ https://git-scm.com/book/en/v2/Getting-Started-A-Short-History-of-Git
    ● Easily rename your default branch from master to main
    

    View Slide

65. Recommended books
    

    View Slide

66. Recommended books
    

    View Slide

67. Resources (I)
    ● https://martinfowler.com/articles/branching-patterns.html
    ● https://martinfowler.com/articles/continuousIntegration.html
    ● https://martinfowler.com/bliki/ContinuousDelivery.html
    ● https://www.atlassian.com/continuous-delivery/principles/continuous-int
    egration-vs-delivery-vs-deployment
    ● https://wiki.c2.com/?SoftwareAsLiability
    ● Dan North: Microservices software that fits in your head
    

    View Slide

68. Resources (II)
    ● Workshops about parallel changes:
    ○ http://www.eferro.net/2018/03/slides-taller-parallel-changes-we-code.html
    ○ https://paucls.wordpress.com/2020/02/26/ci-cd-non-breaking-changes-exercise/
    

    View Slide

69. Please, give me feedback!!! :-)
    https://islomar.typeform.com/to/rnZ82aVu
    

    View Slide

70. View Slide

71. View Slide