Slow Down, Compose Yourself (PyCon AU 2015)

As your codebase grows, you will have instances where you have code that needs to be reused. Then you think, I can do this with subclassing! Now you have a diamond-tree of problems.

Here, we’ll talk about a better way of code reuse and introducing alternate behaviour into existing classes — composition. It’ll improve how you test your code — making your test runs faster, making it easier to test exceptional cases, and making everything easier to understand.

Transcript

1. Slow Down, Compose Yourself PyCon Australia, 2015
2. Hello, I’m Amber Brown (HawkOwl)
5. No software is unchanging
6. Extend, improve, replace
7. Some software are libraries
8. Libraries don’t do anything on their own, but are commonly extended to make applications
9. One of the most common ways to do this is through inheritance and subclassing
10. A library provides a class, you subclass it and extend it to implement your behaviour
11. You write mixins to alter behavour of subclasses, and reduce duplication
12. However, this can lead to problems down the line
13. Multiple inheritance can lead to confusing situations, such as the order of the mixin declarations affecting behavour
14. Large inheritance trees can be confusing to navigate and find what actually does what
15. Say “Slow Down, Compose Yourself”
16. Why you should slow down
17. Software can only be designed once
18. Desirable traits can only be designed in
19. Security, performance, modularity, or testability cannot be “bolted on”
20. Modularity and testability are two very desirable traits
21. Modularity nearly always comes paired with extensibility
22. Modularity helps testability
23. Tests are the only way to validate something works and keeps working
24. When designing a system, you might want to aim for these traits
25. Why you should compose yourself
26. Composition is when logic is made up of smaller blocks of logic
27. This is in contrast to inheritance, where logic is built on top of other logic
28. These blocks are down “component” lines
29. For example, a blog has multiple components
30. Blog posts, user accounts, comments
31. At runtime, each of these components will be taken care of by an implementation
32. During testing, portions will be mocked, stubbed, or faked out
33. Replacing the real components with fake ones reduces the surface area to test
34. Fake components can be hardcoded to return otherwise hard-to-reproduce results
35. e.g. a database component returning “No More Space”
36. The benefits of composing yourself
37. Inheritance, especially multiple inheritance, can be confusing
38. In non-trivial software, the inheritance trees can become large
39. e.g. Django Generic Class Based Views
40. Modifying the MRO is possible (on CPython at least), but it’s fragile and “magic”
41. Composition gives you equally large (but less strongly tied) trees, where any portion can be swapped out
42. Testing without these switchable components commonly requires monkey patching
43. Monkey patching is also fragile
44. It commonly relies on implementation details
45. …and it’s very messy
46. Tracking down what exactly does what requires something like ccbv.co.uk for Django
47. Then you have to make sure the monkey patching doesn’t affect any internal state
48. Because inheritance leads to objects that are simply too big
49. Mixins can override other mixins which can get overridden by subclasses
50. How to
51. Write an example of your code being used
53. Ideally, once you’ve written this code, this “API sketch” will actually run
54. Find defects
55. Design the API “experience”
56. Begin working down from there
57. Example: Blog
58. Three top-level components
59. Blog posts Accounts (for authors) Comments (for civil discourse)
60. Write interfaces for all three — if you don’t cover everything, you can expand later!
61. But all three share a common component — storage!
62. So that needs an interface too!
67. Now we have these interfaces, we can write implementations for them
68. We don’t have to write against the production database immediately
69. Let’s make an in- memory one!
71. Then let’s make a blog!
73. Let’s make a web interface, as well!
74. For this example, we’ll just add some posts at startup
76. And then add the web UI
78. Let’s run it, and let’s see what we get!
80. Because it’s composition, this in-memory database can be swapped out for the real one later
81. Or Blog can be further specialised
82. But isn’t it overkill to make an interface that has only one implementation?
83. It doesn’t seem like there could be more implementations of IBlog?
84. Adhering to interfaces brings a new way of thinking
85. With some suitable abstractions, you could have many implementations of IBlog
86. A yearly archive could be implemented as an IBlog that only looks at the year it was instantiated with, rather than making more complex queries
87. The composition mindset leads to lots of simple objects that fit together like LEGO bricks
88. Testing with Composition
89. Composition makes things easier to test
90. Make simple fakes in testing that give you just the result you want
91. Test the components in isolation
93. Test with the parts integrated
94. Correctness at every level
95. Object Composition & Function Composition
96. So far I’ve talked about Object Composition
97. For lots of tasks, just passing a function will suffice
98. This is Function Composition
99. This is essentially Lisp
100. Really, that’s all Lisp is
101. sorted()
102. filter()
103. Examples to follow
104. Cryptography.io
105. Python stdlib
106. Twisted (the newer parts)
107. Lisp
108. Questions! (non-questions will be ignored)