The Cutting Edge Of Versioning (LambdaConf 2024)

Versioning sits at the intersection of contract and communications: what our languages and tools can enforce, and what we want to tell the people using our software.

Semantic Versioning is one answer to this: a sociotechnical contract which lets us define breaking changes. Because it is social as well as technical, though, it is ambiguous. Members of the Rust and TypeScript communities offer one kind of answer to this challenge: specification and tooling. Elm has offered another: baking it into a language-aware package manager. Unison lets evolution and versioning coexist. One group of researchers even baked type evolution into a functional programming language. Other languages and ecosystems have just thrown up their hands in the face of the inevitable edge cases and failure modes. We can do better!

How can new languages include versioning semantics in their design constraints? What tooling should we build for existing ecosystems? Above all, what should you do in your own projects?

Transcript

1. Chris Krycho – LambdaConf 2024 Semantic Versioning, library & framework evolution, programming language design, type systems, and you. The Cutting Edge of Versioning
2. A little bit about me Hello! • 15 years in software engineering • Avionics, energy industry & physics, Bib l es, restaurant ordering, LinkedIn • Now writing new materia l s for The Rust Programming Language book • Recent work at LinkedIn on versions, ecosystem evo l ution, PLs, and more
3. What should you know when we’re done? What is versioning? What does it l ook l ike to take versioning serious l y as a kind of programming? What shou l d you be doing as a programmer who is using versioning? Key goals 3
4. What is versioning?
5. Versioning is a communication tool. What is versioning?
6. Versioning is a communication tool. What is versioning? Something changed: • New features? • Add support for my operating system version? • Fix that annoying bug? 6
7. Does the upgrade get me something I want? 7
8. Am I willing to pay for it? 8
9. Versioning Strategies 9
10. The de facto standard for the past decade. SemVer (Semantic Versioning) <Major>.<Minor>.<Patch>(-<Pre-release>)?(+<Metadata>)? 10
11. The de facto standard for the past decade. SemVer Given the version 5.2.3: • Major: 5 • Minor: 2 • Patch: 3 • Pre-re l ease: N / A • Bui l d or other metadata: N / A 11
12. The de facto standard for the past decade. SemVer Given the version 5.2.3-beta.1+fun: • Major: 5 • Minor: 2 • Patch: 3 • Pre-re l ease: beta.1 • Bui l d or other metadata: fun 12
13. The de facto standard for the past decade. SemVer <Major>.<Minor>.<Patch>(-<Pre-release>)?(+<Metadata>)? • Patch: backwards-compatib l e bug fi xes on l y • Minor: features or deprecations (and possib l y backwards-compatib l e bug fi xes) • Major: breaking changes (and possib l y features and/or backwards-compatib l e bug fi xes) Extras: • Pre-re l ease: optiona l pre-re l ease number • Metadata: optiona l bui l d or other metadata 13
14. …is not without its issues. SemVer • What is a bug? • What is a breaking change? • Observab l e behavior • Performance characteristics • Private API (…but used by l itera l l y everyone) → “SemVer l awyering” 14
15. —SoloVer v2 ∞ “We intentionally do not try to communicate “backward compatibility” as there is no objective and satisfying de fi nition anyways. As a provider, you should document changes properly. As a user, you should test anyways. SoloVer &c. Skip the arguing: just increment a number! 15
16. Dates always go up. (Or close enough!) CalVer &c. Common variations: • YYYY.MM, e.g. 2024.05 • Ubuntu-sty l e: YY.MM, e.g. 24.05 • Patches a l l owed/supported: YY.MM.<patch>, e.g.: • 24.05.1 • 24.05.2 16
17. Just count to 9. TypeScriptVer! • 1.0.0, 1.0.1, …, 1.0.999, … • 1.1.0, 1.1.1, … • … • 1.9.0, 1.9.1, … • 2.0.0, 2.0.1, … 17
18. But why?!? TypeScriptVer! • Marketing: Microsoft muckety-mucks said so. • Phi l osophy: “A l l changes to a compi l er are breaking.” 18
19. —Hyrum Wright ∞ Hyrum’s Law “With a su ffi cient number of users of an A P I , it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody. 19
20. “…a su ffi cient number of users…” Hyrum’s Law How many? Tens. Maybe dozens. 20
21. Versioning is a communication tool.
22. Each versioning scheme communicates something. Versioning is a communication tool. • SemVer: What kinds of changes happened? • Ca l Ver: When did this version happen? • So l oVer etc.: …things happened! • TypeScript: the decima l system!
23. Versioning as Contract 23
24. Speci fi cally: a socio-technical contract. Versioning as Contract At the intersection of peop l e and computers: • Not pure l y socia l . • Not pure l y technica l . • Often imp l icit. Contracts can be enforced!
25. Versioning as Programming 25
26. What are the basic rules for making this work? Versioning as Programming 1. De fi ne your versioning system in a machine-readab l e way. 2. Give semantics to that machine-readab l e format. 3. Map changes in your code to those semantics.
27. —Postel’s Law “Be conservative in what you do, be liberal in what you accept from others. Versioning as Programming De fi ning breaking changes: a shorthand.
28. De fi ning breaking changes: a mental model. Versioning as Programming Non-brea k ing Brea k ing Accept “looser” input Require “stricter” input Provide “stricter” output Provide “looser” output Add a new API Remove an existing API
29. —semver-ts.org ∞ “Note that this summary elides many important details, and those details may surprise you! Versioning as Programming De fi ning breaking changes.
30. Versioning as Programming Checking the contract.
31. Checking the contract: vibes. Versioning as Programming If we on l y cared about communicating with humans… Does it fee l l ike a breaking change? Then it is.
32. Checking the contract: tests. Versioning as Programming • Pioneers: Ruby and Node.js • Changing tests te l l s you something: • Fai l ing tests ⟹ bug or breaking change • New tests ⟹ bug fi x or feature • Removed tests ⟹ removed (deprecated) code • Performance and end-to-end re l ationships
33. Checking the contract: types. Versioning as Programming Types communicate the contract for a piece of code— to both humans and computers!
34. Checking the contract: types in Elm. Versioning as Programming • Run the compi l er when pub l ishing! • On l y at the type l eve l . → Necessary but insu ffi cient.
35. Checking the contract: types in Rust. Versioning as Programming • Uses SemVer for version reso l ution in Cargo. • Documents breaking changes in great detai l .
36. Checking the contract: types in Rust. Versioning as Programming
37. Checking the contract: types in Rust. Versioning as Programming • Uses SemVer for version reso l ution in Cargo. • Documents breaking changes in great detai l . • Growing support for automation: cargo-semver-checks • This is hard!
38. Checking the contract: types in Rust. Versioning as Programming mod example { pub struct Person { pub age: u8, pub name: String, } } let me = example :: Person { name: String :: from("Chris"), age: 36, };
39. Checking the contract: types in Rust. Versioning as Programming mod example { pub struct Person { pub age: u8, pub name: String, id: Uuid, } } let me = example :: Person { name: String :: from("Chris"), age: 36, }; error: cannot construct `Person` with struct literal syntax due to private fields --> src/main.rs:1:14 | 1 | let me = example :: Person { | ^^^^^^^^^^^^^^^ | = note: ... and other private field `id` that was not provided
40. Checking the contract: types in TypeScript. Versioning as Programming • More comp l icated type system ⟹ SemVer checking is harder
41. Checking the contract: types in TypeScript. Versioning as Programming
42. Checking the contract: types in TypeScript. Versioning as Programming • More comp l icated type system ⟹ SemVer checking is harder • Examp l e: • JS co ll ection types are mutab l e • TS has l oca l type inference • TS has untagged union types: string | number
43. Checking the contract: types in TypeScript. Versioning as Programming function example(): string | number; let myArray = [example()]; // Array<string | number> myArray.push(123); // ✅ myArray.push("hello"); // ✅
44. Checking the contract: types in TypeScript. Versioning as Programming function example(): number;
45. De fi ning breaking changes: a mental model. Versioning as Programming Non-brea k ing Brea k ing Accept “looser” input Require “stricter” input Provide “stricter” output Provide “looser” output Add a new API Remove an existing API
46. Checking the contract: types in TypeScript. Versioning as Programming function example(): number; let myArray = [example()]; // Array<number> myArray.push(123); // ✅ myArray.push("hello"); // ❌ string not assignable to number
47. Checking the contract: types in TypeScript. Versioning as Programming function example(): number; let myArray: Array<string | number> = [example()]; myArray.push(123); // ✅ myArray.push("hello"); // ✅
48. Checking the contract: types in Unison. Versioning as Programming • Generate an AST • Norma l ize the AST into a canonica l form • Hash the norma l ized AST • Save the hash, norma l ized AST, and name to SQLite
49. Checking the contract: versions as types. Versioning as Programming version init class Expr extends Object { @init Expr() { super() } @init int eval() { return 0 ; }; } class Num extends Expr { @init int n; @init Num(int n) { super(); this.n = n; } @init int eval() { return this.n; } } class Example { @init Expr expr() { return new Num(4); } } @init((new Example().expr())
50. Checking the contract: versions as types. Versioning as Programming version v1 upgrades init class Add extends Expr { @v1 Expr a, Expr b; @v1 Add(Expr a, Expr b) { super(); this.a = a; this.b = b; } @v1 int eval() { return this.a.eval() + this.b.eval(); } } @v1(new Add((new Example).expr(), (new Example).expr()))
51. Checking the contract: versions as types. Versioning as Programming version v2 upgrades init class Expr extends Object { @v2 string print() { return ""; } } class Num extends Expr { @v2 string print() { return "" + this.n; } } @v1(new Add((new Example).expr(), (new Example).expr())) ✅ @v1((new Example).expr().print()) ❌
52. Checking the contract: versions as types. Versioning as Programming version v3 upgrades v1, v2 replaces init class Add extends Expr { @v3 string print() { return this.a.print() + " + " this.b.print(); } } @v1((new Example).expr().print()) ✅
53. What should you do? 53
54. …as an application developer? What should you do? • Use something SemVer-ish for your user facing versions. • Think about l ibraries and too l s: What are their versioning po l icies? • Imp l icit? Look at history! • Exp l icit? Look at history!
55. …as a library author? What should you do? • Think about—and communicate about—versioning! • Use the too l s: • elm bump, cargo semver-checks, etc. • Bui l d them if they don’t exist! • Keep upgrades l ow-coup l ing. → Support mu l tip l e majors of your own core dependencies
56. …as a framework author? What should you do? Your choices imp l icit l y constrain the who l e ecosystem. So: • Be extra exp l icit about your versioning po l icies. • Design your po l icies according l y: → A l l ow l ibraries to support more than one major version at once. → Make breaking changes infrequent and predictab l e. → Incorporate migration strategies and too l ing. (Codemods!) • Advise l ibraries to treat you as a “peer dependency”.
57. Aside: peer dependency semantics What should you do? • Frameworks are peer dependencies of l ibraries. • Libraries shou l d on l y require the l owest version they support • …and test against the who l e range they support in CI • (This can be comp l icated in many ecosystems!) • In Node, use pnpm or yarn, or use va l idate-peer-dependencies
58. …as a programming language author? What should you do? • Consider versioning in your l anguage design. • Bui l d version-aware-too l ing as soon as possib l e. • Bake “peer dependency” hand l ing into package too l ing. → Make it easy for l ibrary authors to check compatibi l ity. → Make it easy for app l ication authors to get compatib l e versions. → Enab l e automation, for CI, upgrades, etc.
59. What did we learn? Versioning is a form of communication. Versioning is a socio-technica l contract. So: Summing up 59 • Pick a po l icy (preferab l y SemVer!), and state it c l ear l y. • Enforce your po l icy with whatever too l s you can. • Keep upgrade coup l ing l ow! • Bui l d new things to push this forward: peer deps, l anguage features, etc.
60. I appreciate your attention. Thank you! • Fo l l ow me: chriskrycho.com • Emai l me: he l l o@chriskrycho.com • Hire me! • front-end technica l strategy • TS adoption & conversion • Rust workshops • hard prob l em ana l ysis • bui l ding “ratchets” to improve software qua l ity 60