Philosophy, Challenges, and Opportunities of DevOps on Salesforce (with animation)

As presented at the Gearset DevOps Summit on November 2. This version has animations in it, so the diagrams look good, but there are a lot of slides! A version without animations is here https://speakerdeck.com/aidanharding/philosophy-challenges-and-opportunities-of-devops-on-salesforce-no-animation.

In this presentation, we looked at the questions behind DevOps on Salesforce: Why is it hard? What is the real source of truth? What approaches can we take to make progress?

I encourage you to keep things simple and use just enough process/tooling to get what you need.

1. #GearsetDevOpsSummit Philosophy, Challenges, and Opportunities of DevOps on Salesforce Gearset DevOps Summit 2022: Level up your Salesforce DevOps
2. #GearsetDevOpsSummit Aidan Harding Technology Director, Nebula Consulting Over 10 years working on Salesforce A few years messing around on mountain bikes A few years FPGA design PhD in Computer Science
3. #GearsetDevOpsSummit How would we describe Salesforce DevOps?
4. #GearsetDevOpsSummit How would we describe Salesforce DevOps?
5. #GearsetDevOpsSummit Let’s dive in… •It’s not always so simple •Learning Environments •The only truth is production •Don’t “cargo-cult” behaviours from elsewhere •Some ideas, solutions and benefits
6. #GearsetDevOpsSummit The illusion of simplicity
7. #GearsetDevOpsSummit The illusion of simplicity I just want to move my code from A to B without drowning in my own tears
8. #GearsetDevOpsSummit The illusion of simplicity • We “just” want to localise this book from US English to British English I just want to move my code from A to B without drowning in my own tears
9. #GearsetDevOpsSummit The illusion of simplicity • We “just” want to localise this book from US English to British English • So let’s “just” search and replace some terms e.g. pants → trousers I just want to move my code from A to B without drowning in my own tears
10. #GearsetDevOpsSummit The illusion of simplicity • We “just” want to localise this book from US English to British English • So let’s “just” search and replace some terms e.g. pants → trousers • Oops! We’ve made particitrousers I just want to move my code from A to B without drowning in my own tears
11. #GearsetDevOpsSummit The illusion of simplicity • We “just” want to localise this book from US English to British English • So let’s “just” search and replace some terms e.g. pants → trousers • Oops! We’ve made particitrousers I just want to move my code from A to B without drowning in my own tears
12. #GearsetDevOpsSummit Differing Learning Environments
13. #GearsetDevOpsSummit Differing Learning Environments Kind • Feedback is unambiguous and timely • Easy to progress by simple doing the work Wicked • Feedback may be delayed, unclear, or absent • Hard to progress without deliberate experimentation and reflection
14. #GearsetDevOpsSummit Differing Learning Environments Kind • Feedback is unambiguous and timely • Easy to progress by simple doing the work Examples • Sports • Games • Linters and compilers Wicked • Feedback may be delayed, unclear, or absent • Hard to progress without deliberate experimentation and reflection Examples • Parenting • Medicine • DevOps
15. #GearsetDevOpsSummit The real world hurts (not all complexity is accidental)
16. #GearsetDevOpsSummit The real world hurts (not all complexity is accidental) •Salesforce sells itself on being easy to configure, so changes made directly in production are a feature not a bug
17. #GearsetDevOpsSummit The real world hurts (not all complexity is accidental) •Salesforce sells itself on being easy to configure, so changes made directly in production are a feature not a bug •Salesforce prioritises releasing new features over making those features deployable
18. #GearsetDevOpsSummit The real world hurts (not all complexity is accidental) •Salesforce sells itself on being easy to configure, so changes made directly in production are a feature not a bug •Salesforce prioritises releasing new features over making those features deployable •Metadata is often stored as data (especially in packages)
19. #GearsetDevOpsSummit The real world hurts (not all complexity is accidental) •Salesforce sells itself on being easy to configure, so changes made directly in production are a feature not a bug •Salesforce prioritises releasing new features over making those features deployable •Metadata is often stored as data (especially in packages) •Some updates (e.g. changing the type of a field) are not atomic - they require multiple deployments
20. #GearsetDevOpsSummit The real world hurts (not all complexity is accidental) •Salesforce sells itself on being easy to configure, so changes made directly in production are a feature not a bug •Salesforce prioritises releasing new features over making those features deployable •Metadata is often stored as data (especially in packages) •Some updates (e.g. changing the type of a field) are not atomic - they require multiple deployments •Communities (“Experiences”) and metadata god-objects (Profiles, Page Layouts, Record Types)
21. #GearsetDevOpsSummit The real world hurts (not all complexity is accidental) •Salesforce sells itself on being easy to configure, so changes made directly in production are a feature not a bug •Salesforce prioritises releasing new features over making those features deployable •Metadata is often stored as data (especially in packages) •Some updates (e.g. changing the type of a field) are not atomic - they require multiple deployments •Communities (“Experiences”) and metadata god-objects (Profiles, Page Layouts, Record Types) •Consultancies cannot rely on getting buy-in from all stakeholders
22. #GearsetDevOpsSummit Don’t forget the data, dummy!
23. #GearsetDevOpsSummit #GearsetDevOpsSummit The source of truth
24. #GearsetDevOpsSummit #GearsetDevOpsSummit The source of truth •https://developer.salesforce.com/d ocs/metadata-coverage/56
25. #GearsetDevOpsSummit #GearsetDevOpsSummit The source of truth •https://developer.salesforce.com/d ocs/metadata-coverage/56
26. #GearsetDevOpsSummit #GearsetDevOpsSummit The source of truth •https://developer.salesforce.com/d ocs/metadata-coverage/56
27. #GearsetDevOpsSummit #GearsetDevOpsSummit The source of truth •https://developer.salesforce.com/d ocs/metadata-coverage/56 •At Nebula, we sync 24 metadata items in our CI (about 4%)
28. #GearsetDevOpsSummit What is truth? Plato’s cave
29. #GearsetDevOpsSummit What is truth? Plato’s cave
30. #GearsetDevOpsSummit What is truth? Plato’s cave Source control
31. #GearsetDevOpsSummit What is truth? Plato’s cave Source control CI
32. #GearsetDevOpsSummit What is truth? Plato’s cave Source control CI Production
33. #GearsetDevOpsSummit What is truth? Plato’s cave Source control CI Production Devs 😢
34. #GearsetDevOpsSummit Source control is truth?
35. #GearsetDevOpsSummit Source control is truth?
36. #GearsetDevOpsSummit Source control is truth? • It only contains a fraction of the metadata
37. #GearsetDevOpsSummit Source control is truth? • It only contains a fraction of the metadata • It can miss changes made directly in production
38. #GearsetDevOpsSummit Source control is truth? • It only contains a fraction of the metadata • It can miss changes made directly in production • Changes merged into main could still fail to deploy to Production
39. #GearsetDevOpsSummit Source control is truth? • It only contains a fraction of the metadata • It can miss changes made directly in production • Changes merged into main could still fail to deploy to Production • Source control = truth is a leaky abstraction
40. #GearsetDevOpsSummit #GearsetDevOpsSummit Cargo cults
41. #GearsetDevOpsSummit #GearsetDevOpsSummit Cargo cults You ain’t Google (unless you actually are?)
42. #GearsetDevOpsSummit #GearsetDevOpsSummit Cargo cults •Processes become meaningless without the the intentions behind them You ain’t Google (unless you actually are?)
43. #GearsetDevOpsSummit #GearsetDevOpsSummit Cargo cults •Processes become meaningless without the the intentions behind them •“Individuals and interactions over processes and tools” You ain’t Google (unless you actually are?)
44. #GearsetDevOpsSummit #GearsetDevOpsSummit Cargo cults •Processes become meaningless without the the intentions behind them •“Individuals and interactions over processes and tools” •DevOps process, tools, and metrics are a menu, not computer game levels You ain’t Google (unless you actually are?)
45. #GearsetDevOpsSummit WWII in Melanesia
46. #GearsetDevOpsSummit WWII in Melanesia
47. #GearsetDevOpsSummit Copy the symbols
48. #GearsetDevOpsSummit Copy the symbols
49. #GearsetDevOpsSummit Looks about right… but no signal?
50. #GearsetDevOpsSummit e.g. Deployment frequency
51. #GearsetDevOpsSummit e.g. Deployment frequency
52. #GearsetDevOpsSummit e.g. Deployment frequency •You deploy multiple times per day? But what does that even mean?
53. #GearsetDevOpsSummit e.g. Deployment frequency •You deploy multiple times per day? But what does that even mean? •If a feature is “in production” but no-one is using it, is it even in production?
54. #GearsetDevOpsSummit e.g. Deployment frequency •You deploy multiple times per day? But what does that even mean? •If a feature is “in production” but no-one is using it, is it even in production? •“When a measure becomes a target, it ceases to be a good measure.” [1] ◦ [1] “Goodhart’s Law” Strathern, Marilyn (1997). "'Improving ratings': audit in the British University system". European Review. John Wiley & Sons. 5 (3): 305–321
55. #GearsetDevOpsSummit Well, this is depressing How about something more constructive?
56. #GearsetDevOpsSummit Unlocked Packaging
57. #GearsetDevOpsSummit Unlocked Packaging
58. #GearsetDevOpsSummit Unlocked Packaging Clean separation of functional areas Package development really is source- driven All Salesforce tooling Org-dependent can build on-top of messy code Parallel work is easy
59. #GearsetDevOpsSummit Unlocked Packaging Clean separation of functional areas Package development really is source- driven All Salesforce tooling Org-dependent can build on-top of messy code Parallel work is easy Not everything is packageable Not great for customising standard clouds Hard to migrate to from a messy codebase Can be hard to manage dependencies Sometimes requires dependency injection Time to start working can be long
60. #GearsetDevOpsSummit #GearsetDevOpsSummit Production is truth
61. #GearsetDevOpsSummit #GearsetDevOpsSummit Production is truth So let’s work with it
62. #GearsetDevOpsSummit Production is truth
63. #GearsetDevOpsSummit Production is truth
64. #GearsetDevOpsSummit Production is truth
65. #GearsetDevOpsSummit Production is truth
66. #GearsetDevOpsSummit Production is truth
67. #GearsetDevOpsSummit Production is truth
68. #GearsetDevOpsSummit Production is truth
69. #GearsetDevOpsSummit Delta CI to the rescue
70. #GearsetDevOpsSummit Delta CI to the rescue
71. #GearsetDevOpsSummit Delta CI to the rescue
72. #GearsetDevOpsSummit Trunk based development
73. #GearsetDevOpsSummit Trunk based development
74. #GearsetDevOpsSummit Trunk based development
75. #GearsetDevOpsSummit Trunk based development
76. #GearsetDevOpsSummit Trunk based development
77. #GearsetDevOpsSummit Trunk based development
78. #GearsetDevOpsSummit Trunk based development Simple Features are small, so selective promotion is easy CI tool requirements are simple
79. #GearsetDevOpsSummit Trunk based development Simple Features are small, so selective promotion is easy CI tool requirements are simple Collaboration on unfinished code happens via prod No real data for UAT testing outside prod Sometimes user actually want fewer releases (e.g. when they require training to use) May require feature flags
80. #GearsetDevOpsSummit Feature branches
81. #GearsetDevOpsSummit Feature branches
82. #GearsetDevOpsSummit Feature branches
83. #GearsetDevOpsSummit Feature branches Allows collaboration in a sandbox/feature branch UAT in a full/partial sandbox is easy Allows larger releases Less frequent releases are mitigated by updating from Prod
84. #GearsetDevOpsSummit Feature branches Allows collaboration in a sandbox/feature branch UAT in a full/partial sandbox is easy Allows larger releases Less frequent releases are mitigated by updating from Prod Requires more advanced CI tooling/procedures No selective deployment from Integration org – it’s all or nothing Larger releases are more prone to failure
85. #GearsetDevOpsSummit Yet still we DevOp
86. #GearsetDevOpsSummit Yet still we DevOp • It may be hard, but remember life before SFDX?
87. #GearsetDevOpsSummit Yet still we DevOp • It may be hard, but remember life before SFDX? • Understand your goals in each situation
88. #GearsetDevOpsSummit Yet still we DevOp • It may be hard, but remember life before SFDX? • Understand your goals in each situation • Use least amount of process possible
89. #GearsetDevOpsSummit Yet still we DevOp • It may be hard, but remember life before SFDX? • Understand your goals in each situation • Use least amount of process possible • Don’t fixate on a single metric - it’s a system
90. #GearsetDevOpsSummit Yet still we DevOp • It may be hard, but remember life before SFDX? • Understand your goals in each situation • Use least amount of process possible • Don’t fixate on a single metric - it’s a system • Quality and timely feedback improves software
91. #GearsetDevOpsSummit Yet still we DevOp • It may be hard, but remember life before SFDX? • Understand your goals in each situation • Use least amount of process possible • Don’t fixate on a single metric - it’s a system • Quality and timely feedback improves software • Tools can’t solve all your problems, but they are much better than no tools
92. #GearsetDevOpsSummit Thank you! aidan.harding@nebulaconsulting.co.uk https://nebulaconsulting.co.uk/insights/ @AidanHarding