Pusheando en master, que es gerundio

Slides de la charla "Pusheando en master, que es gerundio": https://www.meetup.com/es-ES/CSTechHub/events/271540517/

Vídeo: https://www.youtube.com/watch?v=UvtaujgCNsI

Descripción:
En esta sesión me gustaría compartir mi experiencia trabajando con Trunk-Based Development (TBD) durante los últimos años, tras haber pasado muchos otros con diversas estrategias de branching.

Empezando por intentar entender “por qué hacemos lo que hacemos”, qué buscamos realmente con cada práctica y principio, la idea es ver durante la sesión qué ventajas ofrece trabajar con TBD, sus inconvenientes, cuándo considero que tiene sentido hacerlo… y cuándo no.

Tras una primera parte de la charla más “unidireccional”, habrá una segunda parte ya plenamente grupal y participativa, donde todo el mundo pueda compartir sus experiencias, dudas, inquietudes, etc., para reflexionar y aprender juntes :-)

Transcript

1. Pusheando en master, que es gerundio (Trunk-Based Development 101) CSTechHub 9 de julio de 2020 El vídeo de la charla
2. Ego slide Hemos venido a jugar... Desarrollador de software/producto @islomar islomar@gmail.com <titulitis> Teleco, Psicología, Antropología Social y Cultural, Executive MBA… </titulitis>
7. #NoTools #NoCode
8. What is Trunk-Based Development about? Just always push directly to your main (and only) branch Easy-peasy!!
10. Agenda • Continuous X • Basic branching strategies & Pull/Merge Requests • “Why” and “what for”: SW development principles • Trunk-Based Development for the win! • My personal experience • When NOT to use it • How to start with TBD • Recommended people, books and resources • Q&A
11. Continuous X • Continuous Integration • Continuous Delivery • Continuous Deployment • Continuous Release
12. Continuous Integration “Integrate and test changes after no more than a couple of hours.” “The integration step is unpredictable, but can easily take more time than the original programming.” “Continuous integration should be complete enough that the eventual first deployment of the system is no big deal” (“Extreme Programming Explained, Second Edition”, Kent Beck - 2005)
13. Continuous Delivery “Every push” (Isi©) leaves your code continuously ready to be deployed to Production with one single and simple manual step. • Continuous Integration is a prerequisite • Probably you will need feature flags (among other techniques) • Deployments need to be automated
14. Continuous Delivery “Software delivers no revenue until it is in the hands of its users” “How long would it take your organization to deploy a change that involves just one single line of code?” “Branching should, in most circumstances, be avoided” (“Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation”, Jez Humble and David Farley - 2010)
15. Continuous Deployment Every change/push goes through the pipeline and automatically gets put into production. • Continuous Delivery is a prerequisite • No human intervention You need: • Great safety net (different kind of automated tests, statics analysis, monitoring, etc.) • You will certainly need feature flags (plus other parallel changes techniques) • Easy way to rollback
16. https://opensenselabs.com/blog/articles/continuous-deployment-drupal
17. Continuous Release DEPLOYMENT != RELEASE • Released: visible to the users • Deployed: the code is “put” in the Production systems, maybe not visible. ◦ With a feature toggle, or an unknown URL or a hidden HTML element or • Decouple “deployment” and release!
18. Continuous Release Deploying to Production (without releasing) has value in itself: • The whole pipeline works fine (e.g. tests, infrastructure, etc.) • You check that nothing breaks (even not being active). • You might want to partially activate it (e.g. with feature toggles/flags). • You learn about the HOW.
19. How often is “continuous enough”? Online browser editor which triggers the deployment pipeline after clicking a specific button… why not? • E.g. Dark ◦ “A holistic programming language, editor and infrastructure for serverless backends” ◦ “Deployless means that anything you type is instantly deployed and immediately usable in production.” ◦ Native Continuous Delivery, using feature flags ◦ https://darklang.com/ ◦ https://medium.com/darklang/how-dark-deploys-code-in-50ms-771c6dd60671
20. Source Code Management (SCM) tools 1. No Version Control System (VCs) at all. 2. Floppy disks. 3. CVS (no branches at all) 4. SubVersion a. Creating a branch was highly costly 5. Git a. Feature branching: https://martinfowler.com/bliki/FeatureBranch.html b. GitFlow i. Master, Develop, Feature, Release and Hotfix branches
21. Branching strategies (over-simplified) • Feature branching ◦ One branch per feature (sometimes it means per developer) • GitFlow • Other combined strategies ◦ Feature + Main + Release branches ◦ Release trains
22. GitFlow • Develop branch • Feature branches • Release branches • Master branch • Hotfix branches https://nvie.com/posts/a-successful-git-branching-model/
23. https://nvie.com/posts/a-successful-git-branching-model/
24. Why do people do Pull/Merge Requests? • For feeling safer ◦ Peer review: Someone else can take a look to it and detect problems • For rising the code quality (e.g. design solution) • For sharing/spreading knowledge • For getting the control: ◦ "Safe" way to accept contributions from other people/teams ◦ Final review centralized in a few people (e.g. QA, PM, committee, etc.)
25. Problems with Pull/Merge Requests • It completely breaks the flow ◦ Increased lead time: no Continuous Delivery/Deployment ◦ You better wait for that PR to be merged before creating a new one… or fight with Git • Increased risk ◦ It’s too late: someone already worked several hours (days?) on it…. Waste. ◦ More probability of things breaking or not fitting… or not being needed anymore.
26. Problems with Pull/Merge Requests (cont.) • Code reviews: ◦ you don’t have the context of the person for every decision made. ◦ tenths/hundreds of lines changed... are you kidding? SOURCE OF INEFFICIENCIES AND HUMAN CONFLICTS
28. (IMO) When do pull/merge requests make sense? • Distributed team with very different time zones ◦ Not really possible to do pair programming • OSS: very async, worldwide distributed people • Contributions between teams inside a company ◦ Though, even better: pair with someone from the other team • For some exceptional reason, someone worked on their own
29. https://martinfowler.com/articles/branching-patterns.html#continuous-integration This is what it REALLY happens when branching... https://twitter.com/jahnnie/status/937917022247120898
30. But wait… let’s take several steps back...
31. Why and what for do we develop software professionally? • From an individual/personal level: para ganarnos la vida, para divertirnos, para sentirnos útiles, para evitar el horror vacui vital... • From a company perspective: to earn money, to “make the world a better place” (Silicon Valley dixit) Para resolver problemas que no resulta igual de óptimo/eficiente resolver sin software
33. • “Early and continuous” ◦ How “early”? ◦ How “continuous”? (“... from a couple of weeks…”) • “Working software” + “Valuable software” ◦ Does it break? ◦ Fits business requirements? ◦ Is it used? ◦ Does it satisfy the customer/user? ◦ Does it generate “profit”?
34. “agile” is about… <insert here your opinion> • Focus on people ◦ Communication ◦ Needs and expectations • Risk management ◦ “Short” feedback cycles, in order to improve in either: ▪ Business (WHAT): user feedback ▪ Technology (HOW): system feedback ◦ Learn fast (not “fail fast”)
35. Which branching strategy would “better” fit with reducing the risk management, continuous learning and the continuous delivery of value? (and less people suffering!!)
36. Trunk-Based Development for the win!! https://unsplash.com/photos/abkEAOjnY0s
37. https://twitter.com/CodelyTV/status/1248216553138552834
38. https://svgsilh.com/image/2028004.html Technical excellence Continuous delivery of value to the business https://freesvg.org/unicorn-vector-clipart-pdv “Agile” People first! Make the world a better place!
39. Trunk-Based Development is about... Pushing to master
40. https://unsplash.com/photos/MggK54YixfU
41. Trunk-Based Development is about... Keeping one single branch “by default” and pushing your code there
42. Trunk-Based Development is about… (cont.) + Pair/mob programming + Parallel changes techniques + TDD (or at least “test first”, including baby commits) + A proper product management vision
43. Trunk-Based Development is about… (cont.) A “safe”, easy and automated deployment/release pipeline (CD) + Easy way to rollback + “Loosely-coupled” teams/services + An engineering culture which supports it + etc...
44. “A proper product management vision” • Product Manager working in the day by day with the rest of the team • Vertical slicing: ◦ Simplified and small end-to-end chunks ◦ Business (user stories): GREAT (release) > we learn the WHAT ◦ Technical: good (deployment) > we learn the HOW • Continuous prioritization • Focus on value (aligned with “higher company goals”) ◦ Measure the “value” delivered • Pull system (keep it flowing!!) ◦ Guided by the system/team capacity + JIT (last responsible moment)
45. Some other IMPORTANT practices (not only) for TBD • Canary releases ◦ https://martinfowler.com/bliki/CanaryRelease.html ◦ First, deployed only to some representative environment • Feature toggles (a.k.a “feature flags”) ◦ https://martinfowler.com/articles/feature-toggles.html • Other parallel changes techniques ◦ https://martinfowler.com/bliki/ParallelChange.html ◦ Branch by abstraction ◦ Postel’s law: “Be conservative in what you do, be liberal in what you accept from others” ◦ Strangler Application ◦ Anti-Corruption Layer
46. What if I can not do ALL of that? • Keep calm and do not push to master (yet) • It is sometimes a sign of deeper inefficiencies and improvement opportunities, it just makes them “explicit” (if we have enough self-critic and humility) Congratulations! We have a great opportunity of improvement ahead!
47. If TBD is not about “pushing to master”... what is it about? • Lean Product Management ◦ Risk management ◦ “Fast” feedback • REALLY continuous and sustainable delivery of value • Technical excellence (e.g. many XP/DevOps principles and practices)
48. We want a feedback cycle “as short as possible” Longer delay between deployments https://www.goodreads.com/book/show/34695798-release-it Higher risk of bugs and downtime More changes in each deployment Review processes A great vicious cycle
49. Another way of saying it... Reduce your lead time: from a customer making a request to the customer being satisfied. Reduce your product delivery lead time: from code committed to code successfully running in production. https://www.goodreads.com/book/show/35747076-accelerate
50. https://cloud.google.com/blog/products/devops-sre/the-2019-accelerate-state-of-devops-elite-performance-productivity-and-scaling
51. https://cloud.google.com/blog/products/devops-sre/the-2019-accelerate-state-of-devops-elite-performance-productivity-and-scaling
52. Code is not an asset, it is liability! • The best line of code is the one not written. ◦ “Fewest elements”: https://martinfowler.com/bliki/BeckDesignRules.html • Code needs to be maintained. • Code needs to be tested. • Code needs to be understood. • A line of code is one more possible point of failure. • Software has a COST. https://wiki.c2.com/?SoftwareAsLiability Dan North: Microservices software that fits in your head
53. Code is not an asset, it is liability! “Between the time a developer commits code to the repository and the time it runs in production, code is a pure liability. Undeployed code is unfinished inventory. It has unknown bugs. It may break scaling or cause production downtime. It might be a great implementation of a feature nobody wants.” (“Release it! Second Edition”, Michael T. Nygard - 2018)
54. Does it require to work with monorepo? NOT at all Previous session in this Meetup: Monorepo: uno para gobernarlos a todos
55. My personal experience • Two full years at Alea Soluciones ◦ TDD + pairing + feature toggles + vertical slicing simplification (EVERYDAY) ◦ Canary releases ◦ Multiple services (multirepo) ◦ TBD was the way to work (only a couple of branches for spikes or something like that) ◦ Lucky me, I forgot how to use Git beyond commit, pull and push • Six months at Spotify ◦ Context: pure backend. People working with PRs all the time. ◦ I started introducing TDD + pairing + baby-commits. No need for feature toggles. ◦ That enabled to start experimenting TBD in a a couple of simple services (not too critical).
56. My personal experience • Currently (last 3 months) ◦ TDD + pair/mob + baby-commits + vertical slicing + TBD (EVERYDAY) ◦ … but still not released to users (client needs). ◦ Hopefully we’ll start with Continuous Delivery
57. When NOT to do TBD? Any time where you don’t follow “all” of the practices mentioned: • pair/mob programming • parallel changes techniques • TDD (or at least “test first”) • A proper product management vision ◦ Simplified end-to-end “user stories” ◦ Vertical slicing: small end-to-end chunks ▪ Business: GREAT ▪ Technical: good ◦ Business ability to move in baby steps
58. When NOT to do TBD? • Spikes ◦ After the spike, start from scratch in the main branch • Most OSS projects (super-async, no pairing, distributed timezones…) • When someone worked on their own (no pair/mob) • When the team can not work quite “independently” from other teams ◦ 'No Man is an Island' (by John Donne) • When the team doesn’t feel “comfortable enough” ◦ Really hard to achieve if nobody in the team has previous experience.
59. How to start? (I) • Get help (hire people with experience in this approach) • Work in pairs (so much as possible; even better: do mob!!) • Write tests (isolated + sociable, aka “unit and integration”) • Build a continuous integration server that can monitor the main repository and run the tests automatically for every new commits pushed. • Limit the work in progress a.k.a. WIP (e.g. half the number of developers)
60. How to start? (II) • Automate your deployment to Production • Get a staging/test server where to improve the safety net ◦ Acceptance tests ◦ End to end tests ◦ Load tests, etc. • Configure a release/deployment pipeline which breaks if the different checks/validations do not pass (Andon a.k.a. “Stop the line”) • Make your branches ages visible and limit it to 24 hours (e.g. with Panda Pull Reminders)
61. How to start? (III) • Start with TDD (again: get help) • Improve your baby steps and baby commits skills • Improve your vertical slicing and business simplification skills • Reduce the branches lifetime to 2 hours. • Reminder: this is not about tools or technologies!! • Pick up a non-critical service (e.g. a simple API) and start trying Continuous Delivery (push deploy to the staging environment). In the “worst case scenario”, if TBD doesn’t “succeed”, you will have improved in many other things!!!
62. Main key points • TBD is much more than “pushing to master” ◦ Risk management ◦ Reduced lead time ▪ continuously delivering value to the BUSINESS ▪ Fast feedback (users and systems) • TBD is a great excuse to detect many inefficiencies: ◦ why wouldn’t you be able to start working like that? ◦ What should be “fixed” or “improved” to achieve it? ◦ Everything is “connected”!! (systems thinking)
63. Main key points • It requires to live together with many other practices and principles (not only technical ones, but also related with product management). • Keep calm and start in baby steps… but start! ◦ Probably your context or most of your activity is not “superexceptional” ◦ Though often it requires a HUGE change in your engineering culture… everything is connected!
64. Last reflection: renaming “master”? Whatever your first or second thought is… give it a third • IETF terminology proposal (from 2018): https://tools.ietf.org/id/draft-knodel-terminology-00.html#rfc.section.1.1 • Probable naming origin: Bitkeeper and its master/slave ◦ https://git-scm.com/book/en/v2/Getting-Started-A-Short-History-of-Git • Easily rename your default branch from master to main
65. Recommended books
66. Recommended books
67. Resources (I) • https://martinfowler.com/articles/branching-patterns.html • https://martinfowler.com/articles/continuousIntegration.html • https://martinfowler.com/bliki/ContinuousDelivery.html • https://www.atlassian.com/continuous-delivery/principles/continuous-int egration-vs-delivery-vs-deployment • https://wiki.c2.com/?SoftwareAsLiability • Dan North: Microservices software that fits in your head
68. Resources (II) • Workshops about parallel changes: ◦ http://www.eferro.net/2018/03/slides-taller-parallel-changes-we-code.html ◦ https://paucls.wordpress.com/2020/02/26/ci-cd-non-breaking-changes-exercise/
69. Please, give me feedback!!! :-) https://islomar.typeform.com/to/rnZ82aVu