Slide 1

Slide 1 text

WRITING MAINTAINABLE SOFTWARE ANDREW GODWIN // @andrewgodwin AT SCALE

Slide 2

Slide 2 text

Andrew Godwin / @andrewgodwin Hi, I’m Andrew Godwin • Principal Engineer at Astronomer (w/on Airflow) • Django Migrations, Channels & Async • Doing bad things with Python since 2005

Slide 3

Slide 3 text

Andrew Godwin / @andrewgodwin

Slide 4

Slide 4 text

Andrew Godwin / @andrewgodwin Why am I still writing Python? 17 years is quite a long time, though not as long as some

Slide 5

Slide 5 text

Andrew Godwin / @andrewgodwin I mostly work on large new projects The kind where you only know roughly what you want

Slide 6

Slide 6 text

Andrew Godwin / @andrewgodwin I rarely know exactly where I'm going …and so I use Python, as it's really good at that

Slide 7

Slide 7 text

Andrew Godwin / @andrewgodwin A lot of maintenance advice is "do it right first time"

Slide 8

Slide 8 text

Andrew Godwin / @andrewgodwin But we will all get it wrong sometimes So, how do we build knowing we can make mistakes?

Slide 9

Slide 9 text

Andrew Godwin / @andrewgodwin Software is almost never complete It will spend far longer running and being maintained than you spend creating it

Slide 10

Slide 10 text

Andrew Godwin / @andrewgodwin Nobody knows what they want Which is also why we're safe from AIs writing code, by the way

Slide 11

Slide 11 text

Andrew Godwin / @andrewgodwin This is mostly about systems, not libraries (though some of the lessons are similar!)

Slide 12

Slide 12 text

Andrew Godwin / @andrewgodwin Writing code Understanding code

Slide 13

Slide 13 text

Andrew Godwin / @andrewgodwin What makes maintenance hard? There's some obvious ones, of course…

Slide 14

Slide 14 text

Andrew Godwin / @andrewgodwin Obfuscation Or to echo a bad joke, "job security"

Slide 15

Slide 15 text

Andrew Godwin / @andrewgodwin q=lambda x,_=('c9*6iv"&s1[Y/`Oh7_|pEW:=!uT4+zeNl;Im\'X\\<-wabDMZ8ykgR@{r>B)~qKFd C3H0Q%,S}xVG](?^2#oPjJL.A$U fnt5'): type('').__dict__['}VJ|(@J}>'. translate(('_'*32+_+('__'*len(_))[:(1<<2)-1-(1<<8-2)]))](x,('_'*32+_+(' __'* len(_))[:(1<<2)-1-(1<<8-2)]));globals()[q('11$C3qV}11')] = lambda _,q=q,__builtins__=__builtins__: __builtins__.__dict__[q('11$C3qV}11')](q(_));q2=( lambda globals=(lambda q=(lambda x: getattr(__import__('q('),x)): q): (lambda os, __import__=q: (globals()(__import__(os)))))(); q=[q2('q3>|')(__import__( '(P(').__dict__[q('JVR%')][0],0),q];q2('^93o')(3,0);raw_input();input()

Slide 16

Slide 16 text

Andrew Godwin / @andrewgodwin No documentation The code! It's self-documenting!

Slide 17

Slide 17 text

Andrew Godwin / @andrewgodwin One giant file First one to 100,000 lines wins

Slide 18

Slide 18 text

Andrew Godwin / @andrewgodwin But what are the root causes? At least, the three I think are most important

Slide 19

Slide 19 text

Andrew Godwin / @andrewgodwin What Is this tech/design common? Can I research elsewhere? Where What's the flow of execution? How do I find side effects? Why What was the context? The other options?

Slide 20

Slide 20 text

Andrew Godwin / @andrewgodwin The What

Slide 21

Slide 21 text

Andrew Godwin / @andrewgodwin Boring technology is good! I am very proud to call Django "boring" these days

Slide 22

Slide 22 text

Andrew Godwin / @andrewgodwin Don't overdesign from the start It is very likely you do not need to work at "Google scale"

Slide 23

Slide 23 text

Andrew Godwin / @andrewgodwin You probably don't have big data If it is less than 20TB, it fits on a single machine

Slide 24

Slide 24 text

Andrew Godwin / @andrewgodwin The Principle Of Least Surprise Write it the more obvious way, even if it's 1% slower

Slide 25

Slide 25 text

Andrew Godwin / @andrewgodwin t = sum([p for g, es in gs for e, p in es.items()]) total = 0 for group, entries in gs: for entry, price in entries.item(): total += price

Slide 26

Slide 26 text

Andrew Godwin / @andrewgodwin The Where

Slide 27

Slide 27 text

Andrew Godwin / @andrewgodwin Ever opened 20 files following an execution path? Isn't it the worst?

Slide 28

Slide 28 text

Andrew Godwin / @andrewgodwin It's OK to repeat yourself! Only factor things out into a library if it makes it easier to read

Slide 29

Slide 29 text

Andrew Godwin / @andrewgodwin Code should be easy to follow More people will read it and understand it than will write it

Slide 30

Slide 30 text

Andrew Godwin / @andrewgodwin def sell_ticket(): if check_availability(): amount = calculate_cost() auth_code = preauthorize_payment(amount) allocate_inventory() send_confirmation() complete_payment(auth_code) else: raise CannotSellError() def check_availability(): ...

Slide 31

Slide 31 text

Andrew Godwin / @andrewgodwin class TicketSeller(PaymentMixin, EmailMixin, AvailMixin): def sell(self): super().sell() callback = getattr(self, f"start_{self.type}").call() self.allocate() callback()

Slide 32

Slide 32 text

Andrew Godwin / @andrewgodwin def sell_ticket(): if check_availability(): amount = calculate_cost() auth_code = preauthorize_payment(amount) allocate_inventory() send_confirmation() complete_payment(auth_code) else: raise CannotSellError() def check_availability(): ...

Slide 33

Slide 33 text

Andrew Godwin / @andrewgodwin Never use multiple inheritance OK, there are uses, but don't do anything that depends on understanding MRO

Slide 34

Slide 34 text

Andrew Godwin / @andrewgodwin Never use multiple inheritance OK, there are uses, but don't do anything that depends on understanding MRO Try not to

Slide 35

Slide 35 text

Andrew Godwin / @andrewgodwin Microservices are almost never worth it Unless there's matching ownership… more on that later

Slide 36

Slide 36 text

Andrew Godwin / @andrewgodwin The Why

Slide 37

Slide 37 text

Andrew Godwin / @andrewgodwin Reading what code does is mostly easy Unless it's some of my code from 2005-2006

Slide 38

Slide 38 text

Andrew Godwin / @andrewgodwin Guessing why it works that way is not There is rarely just one way to do something

Slide 39

Slide 39 text

Andrew Godwin / @andrewgodwin # Loop forever while True: # Check to see if we should run allocate if self.allocate_timer.check(): self.allocate() # Check to see if we should run clean if self.clean_timer.check(): self.clean() # Sleep for 0.01 seconds time.sleep(0.01)

Slide 40

Slide 40 text

Andrew Godwin / @andrewgodwin # Main loop - only exit when Ctrl-C or SIGTERM is received while True: # Run workloads if it's time if self.allocate_timer.check(): self.allocate() if self.clean_timer.check(): self.clean() # Sleep so we don't busy-loop if nothing is ready time.sleep(0.01)

Slide 41

Slide 41 text

Andrew Godwin / @andrewgodwin If it took you ages to work out, write why! You'll help someone else or maybe even your future self

Slide 42

Slide 42 text

Andrew Godwin / @andrewgodwin Software Engineering is more than coding It's the skill of making something people want or need

Slide 43

Slide 43 text

Andrew Godwin / @andrewgodwin Splitting Up Problems Communication Documentation Decision-making

Slide 44

Slide 44 text

Andrew Godwin / @andrewgodwin Big projects must have multiple teams Unless you want them to take years to ship

Slide 45

Slide 45 text

“ Andrew Godwin / @andrewgodwin Conway's Law Organizations, who design systems, are constrained to produce designs which are copies of the communication structures of these organizations.

Slide 46

Slide 46 text

Andrew Godwin / @andrewgodwin It works in reverse, too Don't try to build a system that goes against your organisational structure.

Slide 47

Slide 47 text

Andrew Godwin / @andrewgodwin One service/domain per team … and microservices only work when every service has a defined owner

Slide 48

Slide 48 text

Andrew Godwin / @andrewgodwin Reduce communication overheads! Don't make every team talk to every other team every day

Slide 49

Slide 49 text

Andrew Godwin / @andrewgodwin Consider your incentives If you only promote for launching new things, you'll never get anything maintained.

Slide 50

Slide 50 text

Andrew Godwin / @andrewgodwin Iteration speed is king The cost of improving and fixing matters more than of launching

Slide 51

Slide 51 text

Andrew Godwin / @andrewgodwin Progressive Typing Add types as you are more sure of code layouts Progressive Testing Tests are really helpful for refactors, but can be fragile early on Don't Optimise Early Until you know what the bottlenecks are

Slide 52

Slide 52 text

Andrew Godwin / @andrewgodwin The most important metric is "How long from code fix to production?"

Slide 53

Slide 53 text

Andrew Godwin / @andrewgodwin But Andrew! I work on planes/reactors/life support! Well, first of all, thank you! That's not easy! Also… have you considered Rust?

Slide 54

Slide 54 text

Andrew Godwin / @andrewgodwin You can't always iterate in production It's a careful balance as to how much you can get wrong

Slide 55

Slide 55 text

Andrew Godwin / @andrewgodwin All software needs maintenance If you somehow find a perfectly solvable problem, do let me know

Slide 56

Slide 56 text

Andrew Godwin / @andrewgodwin Simple, understandable designs It makes it much easier for others to work with Capture your context and reasons If the context changes, people know it's safe to change your code! Design for iteration It is never, ever done after the initial launch.

Slide 57

Slide 57 text

Andrew Godwin / @andrewgodwin Thank your maintainers. Coworkers. Open Source. Real-world infrastructure.

Slide 58

Slide 58 text

Aitäh! Andrew Godwin @andrewgodwin // aeracode.org